Search

Recently, vision-language instruct-tuning models have made significant progress due to their more comprehensive understanding of the world. In this work, we discovered that large-scale 3D parallel training on those models leads to an imbalanced computation load across different devices. The vision and language parts are inherently heterogeneous: their data distribution and model architecture differ significantly, which affects distributed training efficiency. We rebalanced the computational loads from data, model, and memory perspectives to address this issue, achieving more balanced computation across devices. These three components are not independent but are closely connected, forming an omniverse balanced training framework. Specifically, for the data, we grouped instances into new balanced mini-batches within and across devices. For the model, we employed a search-based method to achieve a more balanced partitioning. For memory optimization, we adaptively adjusted the re-computation strategy for each partition to utilize the available memory fully. We conducted extensive experiments to validate the effectiveness of our method. Compared with the open-source training code of InternVL-Chat, we significantly reduced GPU days, achieving about 1.8x speed-up. Our method's efficacy and generalizability were further demonstrated across various models and datasets. Codes will be released at https://github.com/ModelTC/OmniBal.

The goal of stock trend prediction is to forecast future market movements for informed investment decisions. Existing methods mostly focus on predicting stock trends with supervised models trained on extensive annotated data. However, human annotation can be resource-intensive and the annotated data are not readily available. Inspired by the impressive few-shot capability of Large Language Models (LLMs), we propose using LLMs in a few-shot setting to overcome the scarcity of labeled data and make prediction more feasible to investors. Previous works typically merge multiple financial news for predicting stock trends, causing two significant problems when using LLMs: (1) Merged news contains noise, and (2) it may exceed LLMs' input limits, leading to performance degradation. To overcome these issues, we propose a two-step method 'denoising-then-voting'. Specifically, we introduce an `Irrelevant' category, and predict stock trends for individual news instead of merged news. Then we aggregate these predictions using majority voting. The proposed method offers two advantages: (1) Classifying noisy news as irrelevant removes its impact on the final prediction. (2) Predicting for individual news mitigates LLMs' input length limits. Our method achieves 66.59% accuracy in S&P 500, 62.17% in CSI-100, and 61.17% in HK stock prediction, outperforming the standard few-shot counterparts by around 7%, 4%, and 4%. Furthermore, our proposed method performs on par with state-of-the-art supervised methods.

Human de-occlusion, which aims to infer the appearance of invisible human parts from an occluded image, has great value in many human-related tasks, such as person re-id, and intention inference. To address this task, this paper proposes a dynamic mask-aware transformer (DMAT), which dynamically augments information from human regions and weakens that from occlusion. First, to enhance token representation, we design an expanded convolution head with enlarged kernels, which captures more local valid context and mitigates the influence of surrounding occlusion. To concentrate on the visible human parts, we propose a novel dynamic multi-head human-mask guided attention mechanism through integrating multiple masks, which can prevent the de-occluded regions from assimilating to the background. Besides, a region upsampling strategy is utilized to alleviate the impact of occlusion on interpolated images. During model learning, an amodal loss is developed to further emphasize the recovery effect of human regions, which also refines the model's convergence. Extensive experiments on the AHP dataset demonstrate its superior performance compared to recent state-of-the-art methods.

The effective assessment of the instruction-following ability of large language models (LLMs) is of paramount importance. A model that cannot adhere to human instructions might be not able to provide reliable and helpful responses. In pursuit of this goal, various benchmarks have been constructed to evaluate the instruction-following capacity of these models. However, these benchmarks are limited to a single language and are constructed using automated approaches, which restricts their applicability and the quality of the test examples they contain. To bridge this gap, we introduce the FollowEval benchmark in this paper. This benchmark is composed of instances in both English and Chinese, and all test examples are crafted by human experts. Furthermore, the FollowEval benchmark is designed to assess LLMs across five critical dimensions of instruction following: string manipulation, commonsense reasoning, logical reasoning, spatial reasoning, and response constraints. To enhance the complexity and present a sufficient challenge, each test example is designed to evaluate more than one dimension. We have evaluated various LLMs using the FollowEval benchmark and found that their performance significantly lags behind that of humans. This highlights the considerable room for improvement in the instruction-following ability of these models., Comment: Work in progress

For a compact smooth manifold with corners (or finite CW-complex) $X$, we can prescribe a finite set of spin or spin$^h$ manifolds (possibly with boundary) mapping into it so that every real vector bundle over $X$ is determined, up to stable equivalence, by the Dirac indices of the real vector bundle when pulled-back onto those prescribed spin or spin$^h$ manifolds., Comment: 83 pages, 4 figures, 3 tables. Version 2 improved exposition, added new figures and a new appendix

Purpose: To develop a truly calibrationless reconstruction method that derives ESPIRiT maps from uniformly-undersampled multi-channel MR data by deep learning. Methods: ESPIRiT, one commonly used parallel imaging reconstruction technique, forms the images from undersampled MR k-space data using ESPIRiT maps that effectively represents coil sensitivity information. Accurate ESPIRiT map estimation requires quality coil sensitivity calibration or autocalibration data. We present a U-Net based deep learning model to estimate the multi-channel ESPIRiT maps directly from uniformly-undersampled multi-channel multi-slice MR data. The model is trained using fully-sampled multi-slice axial brain datasets from the same MR receiving coil system. To utilize subject-coil geometric parameters available for each dataset, the training imposes a hybrid loss on ESPIRiT maps at the original locations as well as their corresponding locations within the standard reference multi-slice axial stack. The performance of the approach was evaluated using publicly available T1-weighed brain and cardiac data. Results: The proposed model robustly predicted multi-channel ESPIRiT maps from uniformly-undersampled k-space data. They were highly comparable to the reference ESPIRiT maps directly computed from 24 consecutive central k-space lines. Further, they led to excellent ESPIRiT reconstruction performance even at high acceleration, exhibiting a similar level of errors and artifacts to that by using reference ESPIRiT maps. Conclusion: A new deep learning approach is developed to estimate ESPIRiT maps directly from uniformly-undersampled MR data. It presents a general strategy for calibrationless parallel imaging reconstruction through learning from coil and protocol specific data.

We prove differential K-theory is determined by the character diagram, thus giving an affirmative answer to a question asked by Simons and Sullivan. Our proof combines the techniques of Bunke and Schick with our key observation that there are canonical topologies, induced from the smooth structure of the manifold, on the groups in the character diagram for differential K-theory., Comment: 17 pages, comments welcome!

In this note, we determine the structure of the associative algebra generated by the differential operators $\overline{\mu}, \overline{\partial}, \partial, \mu$ that act on complex-valued differential forms of almost complex manifolds. This is done by showing it is the universal enveloping algebra of the graded Lie algebra generated by these operators and determining the structure of the corresponding graded Lie algebra. We then determine the cohomology of this graded Lie algebra with respect to its canonical inner differential $[d,-]$, as well as its cohomology with respect to all its inner differentials., Comment: To appear in Complex Manifolds. Minor improvements. 9 pages

In this paper, a new structure is proposed which is expected to realize dual-band electromagnetically induced transparency (EIT)-like effect in terahertz spectrum. It can be regarded as a metamaterial of grating-like elements composed of dual-band U-shaped resonators. The near-field coupling between bright modes can produce EIT-like effect. After optimizing the parameters, our numerical simulated results are in good agreement with the theoretical analysis. The EIT-like effect can significantly reduce the group speed near the transparent window, may gain more significant potential applications in slow-light transmission and optical storage., Comment: 7page,7figures,46references

We show that the infinite symmetric product of a connected graded-commutative algebra over the rationals is naturally isomorphic to the free graded-commutative algebra on the positive degree subspace of the original algebra. In particular, the infinite symmetric product of a connected commutative (in the usual sense) graded algebra over the rationals is a polynomial algebra. Applied to topology, we obtain a quick proof of the Dold-Thom theorem in rational homotopy theory for connected spaces of finite type. We also show that finite symmetric products of certain simple free graded commutative algebras are free; this allows us to determine minimal Sullivan models for finite symmetric products of complex projective spaces., Comment: Minor revisions according to referee comments. Accepted to Comptes Rendus - Math\'ematique. 11 pages

We show the minimal total Betti number of a closed almost complex manifold of dimension $2n\ge 8$ is four, thus confirming a conjecture of Sullivan except for dimension $6$. Along the way, we prove the only simply connected closed complex manifold having total Betti number three is the complex projective plane., Comment: 6 pages