Search

The real anisotropic Littlewood's $4 / 3$ inequality is an extension of a famous result obtained in 1930 by J. E. Littlewood. It asserts that, for $a , b \in ( 0 , \infty )$, the following conditions are equivalent: $\bullet$ There is an optimal constant $\mathsf{L}_{ a , b }^{ \mathbb{R} } \in [ 1 , \infty )$ such that \[ \Biggl ( \, \sum_{ k = 1 }^{ \infty } \biggl ( \, \sum_{ j = 1 }^{ \infty } \bigl \lvert A \bigl ( \boldsymbol{e}^{ (k) } , \boldsymbol{e}^{ (j) } \bigr ) \bigr \rvert^a \biggr )^{ \frac{b}{a} } \Biggr )^{ \frac{1}{b} } \leq \mathsf{L}_{ a , b }^{ \mathbb{R} } \cdot \lVert A \rVert \] for every continuous bilinear form $A \colon c_0 \times c_0 \to \mathbb{R}$. $\bullet$ The values $a , b$ satisfy $a , b \geq 1$ and $\frac{1}{a} + \frac{1}{b} \leq \frac{3}{2}$. Several authors have obtained the values of $\mathsf{L}_{ a , b }^{ \mathbb{R} }$ for diverse pairs $( a , b )$. In this paper we provide the complete list of such optimal values, as well as new estimates for $\mathsf{L}_{ a , b }^{ \mathbb{C} }$ (the analog for continuous $\mathbb{C}$-bilinear forms), which are exact in several cases. As an application we prove, in terms of the values $\mathsf{L}_{ 1 , r }^{ \mathbb{C} }$, a variant of Khinchin's inequality for Steinhaus variables, and we determine all the optimal $( q , s )$-cotype constants of the spaces $\ell_1 ( \mathbb{K} )$ (with $\mathbb{K} = \mathbb{R}$ or $\mathbb{C}$) in terms of the values $\mathsf{L}_{ 1 , q }^{ \mathbb{R} }$., Comment: Corrected typos and improved presentation and formatting. Added the section about cotype

Contrastive learning has emerged as a powerful method in deep learning, excelling at learning effective representations through contrasting samples from different distributions. However, neural collapse, where embeddings converge into a lower-dimensional space, poses a significant challenge, especially in semi-supervised and self-supervised setups. In this paper, we first theoretically analyze the effect of large learning rates on contrastive losses that solely rely on the cosine similarity metric, and derive a theoretical bound to mitigate this collapse. {Building on these insights, we propose CLOP, a novel semi-supervised loss function designed to prevent neural collapse by promoting the formation of orthogonal linear subspaces among class embeddings.} Unlike prior approaches that enforce a simplex ETF structure, CLOP focuses on subspace separation, leading to more distinguishable embeddings. Through extensive experiments on real and synthetic datasets, we demonstrate that CLOP enhances performance, providing greater stability across different learning rates and batch sizes., Comment: 17 pages, 8 figures

Contrastive Learning (CL) has emerged as a powerful method for training feature extraction models using unlabeled data. Recent studies suggest that incorporating a linear projection head post-backbone significantly enhances model performance. In this work, we investigate the use of a transformer model as a projection head within the CL framework, aiming to exploit the transformer's capacity for capturing long-range dependencies across embeddings to further improve performance. Our key contributions are fourfold: First, we introduce a novel application of transformers in the projection head role for contrastive learning, marking the first endeavor of its kind. Second, our experiments reveal a compelling "Deep Fusion" phenomenon where the attention mechanism progressively captures the correct relational dependencies among samples from the same class in deeper layers. Third, we provide a theoretical framework that explains and supports this "Deep Fusion" behavior. Finally, we demonstrate through experimental results that our model achieves superior performance compared to the existing approach of using a feed-forward layer., Comment: 10 pages, 2 figures