Search

As the demand for high-resolution image processing in Large Vision-Language Models (LVLMs) grows, sub-image partitioning has become a popular approach for mitigating visual information loss associated with fixed-resolution processing. However, existing partitioning methods uniformly process sub-images, resulting in suboptimal image understanding. In this work, we reveal that the sub-images with higher semantic relevance to the entire image encapsulate richer visual information for preserving the model's visual understanding ability. Therefore, we propose the Global Semantic-guided Weight Allocator (GSWA) module, which dynamically allocates weights to sub-images based on their relative information density, emulating human visual attention mechanisms. This approach enables the model to focus on more informative regions, overcoming the limitations of uniform treatment. We integrate GSWA into the InternVL2-2B framework to create SleighVL, a lightweight yet high-performing model. Extensive experiments demonstrate that SleighVL outperforms models with comparable parameters and remains competitive with larger models. Our work provides a promising direction for more efficient and contextually aware high-resolution image processing in LVLMs, advancing multimodal system development., Comment: 10 pages, 10 figures and tables

We introduce our first-generation reasoning models, DeepSeek-R1-Zero and DeepSeek-R1. DeepSeek-R1-Zero, a model trained via large-scale reinforcement learning (RL) without supervised fine-tuning (SFT) as a preliminary step, demonstrates remarkable reasoning capabilities. Through RL, DeepSeek-R1-Zero naturally emerges with numerous powerful and intriguing reasoning behaviors. However, it encounters challenges such as poor readability, and language mixing. To address these issues and further enhance reasoning performance, we introduce DeepSeek-R1, which incorporates multi-stage training and cold-start data before RL. DeepSeek-R1 achieves performance comparable to OpenAI-o1-1217 on reasoning tasks. To support the research community, we open-source DeepSeek-R1-Zero, DeepSeek-R1, and six dense models (1.5B, 7B, 8B, 14B, 32B, 70B) distilled from DeepSeek-R1 based on Qwen and Llama.

Context. The DESI Legacy Imaging Survey DR9, with its extensive dataset of galaxy locations and photometric redshifts, presents an opportunity to study Baryon Acoustic Oscillations (BAO) in the region covered by the ongoing DESI spectroscopic survey. Aims. We aim to investigate differences between different parts of the DR9 footprint. Furthermore, we want to measure the BAO scale for luminous red galaxies within them. Our selected redshift range of 0.6 to 0.8 corresponds to the bin in which a tension between DESI Y1 and eBOSS was found. Methods. We calculated the anisotropic two-point correlation function in a modified binning scheme to detect the BAO in DR9 data. We then use template fits based on simulations to measure the BAO scale in the imaging data. Results. Our analysis revealed the expected correlation function shape in most of the footprint areas, showing a BAO scale consistent with Planck's observations. Aside from identified mask-related data issues in the southern region of the South Galactic Cap, we also find a notable variance between the different footprints. Conclusions. We find that this variance is consistent with the difference between the DESI Y1 and eBOSS data and it supports the argument that that tension is caused by sample variance. Additionally, we also uncovered systematic biases not previously accounted for in photometric BAO studies. We emphasize the necessity of adjusting for the systematic shift in the BAO scale associated with typical photometric redshift uncertainties to ensure accurate measurements., Comment: 20 pages, 14 figures, 3 tables, accepted for publication in Astronomy and Astrophysics

The rapid increase in the size of large language models (LLMs) has significantly escalated their computational and memory demands, posing challenges for efficient deployment, especially on resource-constrained devices. Structured pruning has emerged as an effective model compression method that can reduce these demands while preserving performance. In this paper, we introduce FASP (Fast and Accurate Structured Pruning), a novel structured pruning framework for LLMs that emphasizes both speed and accuracy. FASP employs a distinctive pruning structure that interlinks sequential layers, allowing for the removal of columns in one layer while simultaneously eliminating corresponding rows in the preceding layer without incurring additional performance loss. The pruning metric, inspired by Wanda, is computationally efficient and effectively selects components to prune. Additionally, we propose a restoration mechanism that enhances model fidelity by adjusting the remaining weights post-pruning. We evaluate FASP on the OPT and LLaMA model families, demonstrating superior performance in terms of perplexity and accuracy on downstream tasks compared to state-of-the-art methods. Our approach achieves significant speed-ups, pruning models such as OPT-125M in 17 seconds and LLaMA-30B in 15 minutes on a single NVIDIA RTX 4090 GPU, making it a highly practical solution for optimizing LLMs.

Current multimodal large language models (MLLMs) often underperform on mathematical problem-solving tasks that require fine-grained visual understanding. The limitation is largely attributable to inadequate perception of geometric primitives during image-level contrastive pre-training (e.g., CLIP). While recent efforts to improve math MLLMs have focused on scaling up mathematical visual instruction datasets and employing stronger LLM backbones, they often overlook persistent errors in visual recognition. In this paper, we systematically evaluate the visual grounding capabilities of state-of-the-art MLLMs and reveal a significant negative correlation between visual grounding accuracy and problem-solving performance, underscoring the critical role of fine-grained visual understanding. Notably, advanced models like GPT-4o exhibit a 70% error rate when identifying geometric entities, highlighting that this remains a key bottleneck in visual mathematical reasoning. To address this, we propose a novel approach, SVE-Math (Selective Vision-Enhanced Mathematical MLLM), featuring a geometric-grounded vision encoder and a feature router that dynamically adjusts the contribution of hierarchical visual feature maps. Our model recognizes accurate visual primitives and generates precise visual prompts tailored to the language model's reasoning needs. In experiments, SVE-Math-Qwen2.5-7B outperforms other 7B models by 15% on MathVerse and is compatible with GPT-4V on MathVista. Despite being trained on smaller datasets, SVE-Math-7B achieves competitive performance on GeoQA, rivaling models trained on significantly larger datasets. Our findings emphasize the importance of incorporating fine-grained visual understanding into MLLMs and provide a promising direction for future research.

Multi-Agent Path Finding (MAPF), which focuses on finding collision-free paths for multiple robots, is crucial in autonomous warehouse operations. Lifelong MAPF (L-MAPF), where agents are continuously reassigned new targets upon completing their current tasks, offers a more realistic approximation of real-world warehouse scenarios. While cache storage systems can enhance efficiency and reduce operational costs, existing approaches primarily rely on expectations and mathematical models, often without adequately addressing the challenges of multi-robot planning and execution. In this paper, we introduce a novel mechanism called Lifelong MAPF with Cache Mechanism (L-MAPF-CM), which integrates high-level cache storage with low-level path planning. We have involved a new type of map grid called cache for temporary item storage. Additionally, we involved a task assigner (TA) with a locking mechanism to bridge the gap between the new cache grid and L-MAPF algorithm. The TA dynamically allocates target locations to agents based on their status in various scenarios. We evaluated L-MAPF-CM using different cache replacement policies and task distributions. L-MAPF-CM has demonstrated performance improvements particularly with high cache hit rates and smooth traffic conditions., Comment: arXiv admin note: substantial text overlap with arXiv:2403.13421

Large Vision-Language Models (LVLMs) have achieved remarkable success in a wide range of multimodal tasks by integrating pre-trained vision encoders and large language models. However, current LVLMs primarily rely on visual features extracted from the final layers of the vision encoder, overlooking the complementary information available in shallower layers. While recent approaches have explored the use of multilayer visual features in LVLMs, they tend to be task-agnostic and fail to examine the dependencies of hierarchical visual features on specific tasks. To address these gaps, we systematically investigate the contributions of visual features from different encoder layers using 18 benchmarks spanning 6 task categories. Our findings reveal that multilayer features provide complementary strengths with varying task dependencies, and uniform fusion leads to suboptimal performance. Building on these insights, we propose the instruction-guided vision aggregator, a module that dynamically integrates multi-layer visual features based on textual instructions, without increasing the number of visual tokens. Extensive evaluations demonstrate the superior performance of our method. Additionally, an in-depth analysis of the aggregator's behavior highlights the dominance of mid-to-high-level features in semantic-rich tasks and the critical role of low-level features in fine-grained perception.

We present DeepSeek-V3, a strong Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token. To achieve efficient inference and cost-effective training, DeepSeek-V3 adopts Multi-head Latent Attention (MLA) and DeepSeekMoE architectures, which were thoroughly validated in DeepSeek-V2. Furthermore, DeepSeek-V3 pioneers an auxiliary-loss-free strategy for load balancing and sets a multi-token prediction training objective for stronger performance. We pre-train DeepSeek-V3 on 14.8 trillion diverse and high-quality tokens, followed by Supervised Fine-Tuning and Reinforcement Learning stages to fully harness its capabilities. Comprehensive evaluations reveal that DeepSeek-V3 outperforms other open-source models and achieves performance comparable to leading closed-source models. Despite its excellent performance, DeepSeek-V3 requires only 2.788M H800 GPU hours for its full training. In addition, its training process is remarkably stable. Throughout the entire training process, we did not experience any irrecoverable loss spikes or perform any rollbacks. The model checkpoints are available at https://github.com/deepseek-ai/DeepSeek-V3.

Reliable flood forecasting remains a critical challenge due to persistent underestimation of peak flows and inadequate uncertainty quantification in current approaches. We present DRUM (Diffusion-based Runoff Model), a generative AI solution for probabilistic runoff prediction. DRUM builds up an iterative refinement process that generates ensemble runoff estimates from noise, guided by past meteorological conditions, present meteorological forecasts, and static catchment attributes. This framework allows learning complex hydrological behaviors without imposing explicit distributional assumptions, particularly benefiting extreme event prediction and uncertainty quantification. Using data from 531 representative basins across the contiguous United States, DRUM outperforms state-of-the-art deep learning methods in runoff forecasting regarding both deterministic and probabilistic skills, with particular advantages in extreme flow (0.1%) predictions. DRUM demonstrates superior flood early warning skill across all magnitudes and lead times (1-7 days), achieving F1 scores near 0.4 for extreme events under perfect forecasts and maintaining robust performance with operational forecasts, especially for longer lead times and high-magnitude floods. When applied to climate projections through the 21st century, DRUM reveals increasing flood vulnerability in 47.8-57.1% of basins across emission scenarios, with particularly elevated risks along the West Coast and Southeast regions. These advances demonstrate significant potential for improving both operational flood forecasting and long-term risk assessment in a changing climate., Comment: 40 pages, 12 figures

The design of metallic contact grids on the front side of thermophotovoltaic cells is critical since it can cause significant optical and electrical resistive losses, particularly in the near field. However, from the theoretical point of view, this effect has been either discarded or studied by means of extremely simplified models like the shadowing methods, that consist in simply ignoring the fraction of the semiconductor surface covered by metal. Our study, based on a rigorous three-body theoretical framework and implemented using the scattering matrix approach with the Fourier modal method augmented with adaptive spatial resolution, provides deeper insight into the influence of the front metal contact grid. This approach allows direct access to the radiative power absorbed by the semiconductor, enabling the proposal of an alternative definition for the thermophotovoltaic cell efficiency. By modeling this grid as a metallic grating, we demonstrate its significant impact on the net radiative power absorbed by the cell and, consequently, on the generated electrical power. Our analysis reveals behaviors differing substantially from those predicted by previous simplistic approaches., Comment: 8 pages, 7 figures

Klyshko's advanced-wave picture (AWP) is mainly interpreted by replacing the nonlinear crystal producing biphotons via spontaneous parametric down-conversion (SPDC) by a mirror in quantum imaging protocols with thin crystals, where the biphotons are perfectly correlated in position at the crystal. To better explain the biphoton spatial states produced by arbitrary crystals and pump beams, we develop a formal theory of AWP with monochromatic lights that the conditional wave function of one photon is calculated by propagation, multiplication, and another propagation. The case of more general photon postselection or no detection and the inclusion of polarization are studied. Then, we explain the form of the biphoton state from SPDC with a bulk crystal and its free-space propagation. By treating the biphoton wave function as an impulse response function of a classical optical setup, we analyze quantum imaging with undetected photons and quantum holography with polarization entanglement, where properties like the spatial resolution can be concisely deduced. This method can be employed to design nonlinear materials or novel quantum imaging techniques. Finally, we discuss Klyshko's original proposal beyond monochromatic lights with the Hong-Ou-Mandel effect as an example., Comment: 18 pages, 7 figures. (c) 2024 American Physical Society

Thouless pumping with nontrivial topological phases provides a powerful means for the manipulation of matter waves in one-dimensional lattice systems. The band topology is revealed by the quantization of pumped charge. In the context of Thouless pumping, we present a graphical representation for the topological phases characterized by the Chern number of an effective two-dimensional band. We illustrate how the two topological phases with distinct Zak phase is connected in the pumping process. Such a visual depiction exhibits typical patterns that is directly related to a linking number and to the Chern number, allowing for the construction of Thouless pumping schemes in a practical way. As a demonstration, we present a generalized Creutz model with tunable Peierls phase, inter-leg imbalance and diagonal hopping. Various modulation schemes for Thouless pumping are studied, focusing on their graphical representations in Bloch space, as well as the quantized pumping phenomenon in real space., Comment: 10 pages, 14 figures

We study the dynamic pricing and replenishment problems under inconsistent decision frequencies. Different from the traditional demand assumption, the discreteness of demand and the parameter within the Poisson distribution as a function of price introduce complexity into analyzing the problem property. We demonstrate the concavity of the single-period profit function with respect to product price and inventory within their respective domains. The demand model is enhanced by integrating a decision tree-based machine learning approach, trained on comprehensive market data. Employing a two-timescale stochastic approximation scheme, we address the discrepancies in decision frequencies between pricing and replenishment, ensuring convergence to local optimum. We further refine our methodology by incorporating deep reinforcement learning (DRL) techniques and propose a fast-slow dual-agent DRL algorithm. In this approach, two agents handle pricing and inventory and are updated on different scales. Numerical results from both single and multiple products scenarios validate the effectiveness of our methods.

Hallucinations in Large Language Models (LLMs) remain a major obstacle, particularly in high-stakes applications where factual accuracy is critical. While representation editing and reading methods have made strides in reducing hallucinations, their heavy reliance on specialised tools and training on in-domain samples, makes them difficult to scale and prone to overfitting. This limits their accuracy gains and generalizability to diverse datasets. This paper presents a lightweight method, Norm Voting (NoVo), which harnesses the untapped potential of attention head norms to dramatically enhance factual accuracy in zero-shot multiple-choice questions (MCQs). NoVo begins by automatically selecting truth-correlated head norms with an efficient, inference-only algorithm using only 30 random samples, allowing NoVo to effortlessly scale to diverse datasets. Afterwards, selected head norms are employed in a simple voting algorithm, which yields significant gains in prediction accuracy. On TruthfulQA MC1, NoVo surpasses the current state-of-the-art and all previous methods by an astounding margin -- at least 19 accuracy points. NoVo demonstrates exceptional generalization to 20 diverse datasets, with significant gains in over 90\% of them, far exceeding all current representation editing and reading methods. NoVo also reveals promising gains to finetuning strategies and building textual adversarial defence. NoVo's effectiveness with head norms opens new frontiers in LLM interpretability, robustness and reliability.

We propose a geometric series modulated non-Hermitian quasiperiodic lattice model, and explore its localization and topological properties. The results show that with the ever-increasing summation terms of the geometric series, multiple mobility edges and non-Hermitian point gaps with high winding number can be induced in the system. The point gap spectrum of the system has a Matryoshka doll-like structure in the complex plane, resulting in a high winding number. In addition, we analyze the limit case of summation of infinite terms. The results show that the mobility edges merge together as only one mobility edge when summation terms are pushed to the limit. Meanwhile, the corresponding point gaps are merged into a ring with winding number equal to one. Through Avila's global theory, we give an analytical expression for mobility edges in the limit of infinite summation, reconfirming that mobility edges and point gaps do merge and will result in a winding number that is indeed equal to one., Comment: main 6 pages, 7 figures

We explore a new security loophole in a practical continuous-variable quantum key distribution (CVQKD) system, which is opened by the photorefractive effect of lithium niobate-based (LN-based) modulators. By exploiting this loophole, we propose a quantum hacking strategy, i.e., the induced-photorefraction attack, which utilizes the induced photorefraction on the LN-based modulators to hide the classical intercept-resend attack. Specifically, we show that the induced photorefraction can bias the response curve of the LN-based modulator, which will affect the intensity of the modulated signal. Based on the investigation of the channel parameter estimation under above influence, we further analyze the secret key rate of the practical CVQKD system. The simulation results indicate that the communication parties will overestimate the secret key rate, which reveals that Eve can actively open the above loophole by launching the induced-photorefraction attack to successfully obtain the secret key information. To defend against this attack, we can use a random monitoring scheme for modulation variance to determine this attack, and use an improving optical power limiter to effectively mitigate the irradiation beam. Apart from these countermeasures, we also propose using the Sagnac-based IM to stabilize the practical CVQKD system, which can minimize the above effects.

Tilting planar samples for multi-zone-axes observation is a routine procedure in electron microscopy. However, this process invariably introduces optical path differences in the electron beam across different sample positions, significantly compromising image quality, particularly over large fields of view. To address this challenge, we developed path difference ptychography (PDP), a method capable of decoupling path differences from the four-dimensional data during reconstruction. This enables the acquisition of high-quality, large-scale images, facilitating a more comprehensive understanding and analysis of materials microstructure. Moreover, PDP has the potential to promote the widespread application of ptychographic tomography in the analysis of planar samples.

By using PTP and DEA methods, the study of marine fishing capacity in China revealed that it's more accurate to consider the input index as total power rather than the number of ships. The analysis clarified that the main issue with marine fishing capacity in China is the excess total power of fishing ships. The study suggested reducing the number of ships by 35.2%, the gross tonnage by 29.8%, and the total power by 37.3% to align with the catch levels of 1999. It also proposed the idea of supplementing peak years. The PTP methodology was found to be suitable for longitudinal analysis over time, while the DEA approach is better for comparing fishing capacity across different regions at the same time. Additionally, the study indicated that practical fishing capacity tends to be underestimated, so actual reductions in capacity are usually larger than calculated values.

For promoting the development of the marine economy more sustainably, based on the data envelopment analysis method and combined with the impact of the marine environment, the environmental performance of the marine economy was evaluated for Chinese coastal provinces. Firstly, the classical CCR model was used. Then, a model that considered undesirable outputs was developed to suit the Chinese marine economy. Using the two models, the economic efficiencies without environmental consideration and the environmental performance index were calculated and compared. According to the results, the empirical relationship between EPI and EE, per capita GDP, and the industrial structure was analyzed. It is useful for guiding the coastal local economy of China to a healthy way.

The fishing capacity and capacity utilization for Chinese inshore fleets over the latest 13 years were measured using the DEA method. Relevant models were then established to analyze the relationships between capacity output, capacity utilization, and income, and the function of collecting taxes to control fishing capacity was quantitatively simulated. It was pointed out that the tax system would be effective for curtailing fishing capacity and improving the efficiency of the entire fishing industry in China, provided that the tax rate is not too low. Finally, it was suggested that collecting taxes at a proper rate be implemented for Chinese inshore fishing fleets.

For controlling pollution of the marine environment while developing coastal economy, the coastal environmental performance was proposed and measured in static and dynamic methods combined with DEA and efficiency theory in this paper. With the two methods, 16 harbor cities were evaluated. The results showed the index designed in this paper can better reflect the effect to the marine environment for economy of the coastal cities.

Firstly, by establishing a prediction model for global sea-level rise and calculating with Maple, it is shown that the global sea-level rise rate in 2009 is 2.68 mm/a. The height and rate of global sea-level rise will be about 9.11 cm and 3.22 mm/a in 2020. Based on the study and the actual land subsidence in Shanghai Lingang New City, the rate of relative sea-level rise near Lingang New City is calculated to be 12.68 mm/a in 2009. Then, through setting up the extrapolation prediction model with a linear trend term and a significant tidal cycle, the rise rate of average sea-level near Lingang New City was predicted. The result showed it will be 0.33 mm/a in 2020.

Pruning is a critical strategy for compressing trained large language models (LLMs), aiming at substantial memory conservation and computational acceleration without compromising performance. However, existing pruning methods often necessitate inefficient retraining for billion-scale LLMs or rely on heuristic methods such as the optimal brain surgeon framework, which degrade performance. In this paper, we introduce FISTAPruner, the first post-training pruner based on convex optimization models and algorithms. Specifically, we propose a convex optimization model incorporating $\ell_1$ norm to induce sparsity and utilize the FISTA solver for optimization. FISTAPruner incorporates an intra-layer cumulative error correction mechanism and supports parallel pruning. We comprehensively evaluate FISTAPruner on models such as OPT, LLaMA, LLaMA-2, and LLaMA-3 with 125M to 70B parameters under unstructured and 2:4 semi-structured sparsity, demonstrating superior performance over existing state-of-the-art methods across various language benchmarks.

To empirically study the policy impact of a National New Area on the local economy, this paper evaluates the effect of the Zhoushan Archipelago New Area on local GDP growth rate and economic efficiency. By collecting input and output data from 20 prefectural-level cities in Jiangsu, Zhejiang, and Anhui provinces from 1995 to 2015, we estimate the economic efficiency of these cities using data envelopment analysis. Subsequently, we construct counterfactuals for Zhoushan by selecting comparable cities from the dataset, excluding Zhoushan, and applying a panel data approach. The difference between the actual and counterfactual values for GDP growth rate and economic efficiency in Zhoushan is analyzed to determine the treatment effect of the National New Area policy. The research reveals that in the initial four years, the New Area policy enhanced Zhoushan's economic efficiency but negatively affected its GDP growth rate. This influence gradually disappeared after four years. Further analysis suggests that the policy's effect on GDP growth rate varies with the level of economic development in different regions, having a more substantial impact in less developed areas. Therefore, we conclude that establishing a New Area in relatively undeveloped zones is more advantageous.

The photorealistic reconstruction and rendering of architectural scenes have extensive applications in industries such as film, games, and transportation. It also plays an important role in urban planning, architectural design, and the city's promotion, especially in protecting historical and cultural relics. The 3D Gaussian Splatting, due to better performance over NeRF, has become a mainstream technology in 3D reconstruction. Its only input is a set of images but it relies heavily on geometric parameters computed by the SfM process. At the same time, there is an existing abundance of raw 3D models, that could inform the structural perception of certain buildings but cannot be applied. In this paper, we propose a straightforward method to harness these raw 3D models to guide 3D Gaussians in capturing the basic shape of the building and improve the visual quality of textures and details when photos are captured non-systematically. This exploration opens up new possibilities for improving the effectiveness of 3D reconstruction techniques in the field of architectural design.

Large model training has been using recomputation to alleviate the memory pressure and pipelining to exploit the parallelism of data, tensor, and devices. The existing recomputation approaches may incur up to 40% overhead when training real-world models, e.g., the GPT model with 22B parameters. This is because they are executed on demand in the critical training path. In this paper, we design a new recomputation framework, Lynx, to reduce the overhead by overlapping the recomputation with communication occurring in training pipelines. It consists of an optimal scheduling algorithm (OPT) and a heuristic-based scheduling algorithm (HEU). OPT achieves a global optimum but suffers from a long search time. HEU was designed based on our observation that there are identical structures in large DNN models so that we can apply the same scheduling policy to all identical structures. HEU achieves a local optimum but reduces the search time by 99% compared to OPT. Our comprehensive evaluation using GPT models with 1.3B-20B parameters shows that both OPT and HEU outperform the state-of-the-art recomputation approaches (e.g., Megatron-LM and Checkmake) by 1.02-1.53x. HEU achieves a similar performance as OPT with a search time of 0.16s on average., Comment: 13 pages

With an extremely high dimensionality, the spatial degree of freedom of entangled photons is a key tool for quantum foundation and applied quantum techniques. To fully utilize the feature, the essential task is to experimentally characterize the multiphoton spatial wave function including the entangled amplitude and phase information at different evolutionary stages. However, there is no effective method to measure it. Quantum state tomography is costly, and quantum holography requires additional references. Here we introduce quantum Shack-Hartmann wavefront sensing to perform efficient and reference-free measurement of the biphoton spatial wave function. The joint probability distribution of photon pairs at the back focal plane of a microlens array is measured and used for amplitude extraction and phase reconstruction. In the experiment, we observe that the biphoton amplitude correlation becomes weak while phase correlation shows up during free-space propagation. Our work is a crucial step in quantum physical and adaptive optics and paves the way for characterizing quantum optical fields with high-order correlations or topological patterns., Comment: Main text: 6 pages, 4 figures; Supplemental Material: 13 pages, 11 figures. (c) 2024 American Physical Society

The expanding applications call for novel new-generation wide-bandgap semiconductors. Here, we show that a compound only composed of the ordinary elements Na and Cl, namely three-dimensional NaCl2 crystal, is a wide-bandgap semiconductor. This finding benefits from the breaking of conventional stoichiometry frameworks in the theoretical design, leading to the discovery of three-dimensional XY2 (X = Na, Li, K; Y = Cl, F, Br, I) crystals, with covalent bonds of Y pairs inducing the wide bandgap from 2.24 to 4.45 eV. Crucially, such an unexpected NaCl2 crystal was successfully synthesized under ambient conditions. The unconventional stoichiometric strategy with other chemical elements potentially yields more wide-bandgap semiconductors, offering the capability for bandgap tuning. These unconventional stoichiometric materials may also exhibit superconductivity, transparent inorganic electrides, high-energy-density, and beyond.

We present DeepSeek-V2, a strong Mixture-of-Experts (MoE) language model characterized by economical training and efficient inference. It comprises 236B total parameters, of which 21B are activated for each token, and supports a context length of 128K tokens. DeepSeek-V2 adopts innovative architectures including Multi-head Latent Attention (MLA) and DeepSeekMoE. MLA guarantees efficient inference through significantly compressing the Key-Value (KV) cache into a latent vector, while DeepSeekMoE enables training strong models at an economical cost through sparse computation. Compared with DeepSeek 67B, DeepSeek-V2 achieves significantly stronger performance, and meanwhile saves 42.5% of training costs, reduces the KV cache by 93.3%, and boosts the maximum generation throughput to 5.76 times. We pretrain DeepSeek-V2 on a high-quality and multi-source corpus consisting of 8.1T tokens, and further perform Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) to fully unlock its potential. Evaluation results show that, even with only 21B activated parameters, DeepSeek-V2 and its chat versions still achieve top-tier performance among open-source models.