Search

Large language models (LLMs) show remarkable abilities with instruction tuning. However, they fail to achieve ideal tasks when lacking high-quality instruction tuning data on target tasks. Multi-Aspect Controllable Text Generation (MCTG) is a representative task for this dilemma, where aspect datasets are usually biased and correlated. Existing work exploits additional model structures and strategies for solutions, limiting adaptability to LLMs. To activate MCTG ability of LLMs, we propose a lightweight MCTG pipeline based on data augmentation. We analyze bias and correlations in traditional datasets, and address these concerns with augmented control attributes and sentences. Augmented datasets are feasible for instruction tuning. In our experiments, LLMs perform better in MCTG after data augmentation, with a 20% accuracy rise and less aspect correlations.

Let $J_{1,m}(N)$ be the vector space of Jacobi forms of weight one and index $m$ on $\Gamma_0(N)$. In 1985, Skoruppa proved that $J_{1,m}(1)=0$ for all $m$. In 2007, Ibukiyama and Skoruppa proved that $J_{1,m}(N)=0$ for all $m$ and all squarefree $N$ with $\mathrm{gcd}(m,N)=1$. This paper aims to extend their results. We determine all levels $N$ separately, such that $J_{1,m}(N)=0$ for all $m$; or $J_{1,m}(N)=0$ for all $m$ with $\mathrm{gcd}(m,N)=1$. We also establish explicit dimension formulas of $J_{1,m}(N)$ when $m$ and $N$ are relatively prime or $m$ is squarefree. These results are obtained by refining Skoruppa's method and analyzing local invariants of Weil representations. As applications, we prove the vanishing of Siegel modular forms of degree two and weight one in some cases., Comment: 39 pages, comments welcome

Deep learning models are increasingly deployed on resource-constrained edge devices for real-time data analytics. In recent years, Vision Transformer models and their variants have demonstrated outstanding performance across various computer vision tasks. However, their high computational demands and inference latency pose significant challenges for model deployment on resource-constraint edge devices. To address this issue, we propose a novel Vision Transformer splitting framework, ED-ViT, designed to execute complex models across multiple edge devices efficiently. Specifically, we partition Vision Transformer models into several sub-models, where each sub-model is tailored to handle a specific subset of data classes. To further minimize computation overhead and inference latency, we introduce a class-wise pruning technique that reduces the size of each sub-model. We conduct extensive experiments on five datasets with three model structures, demonstrating that our approach significantly reduces inference latency on edge devices and achieves a model size reduction of up to 28.9 times and 34.1 times, respectively, while maintaining test accuracy comparable to the original Vision Transformer. Additionally, we compare ED-ViT with two state-of-the-art methods that deploy CNN and SNN models on edge devices, evaluating accuracy, inference time, and overall model size. Our comprehensive evaluation underscores the effectiveness of the proposed ED-ViT framework., Comment: 14 pages, 8 figures

In the field of industrial inspection, Multimodal Large Language Models (MLLMs) have a high potential to renew the paradigms in practical applications due to their robust language capabilities and generalization abilities. However, despite their impressive problem-solving skills in many domains, MLLMs' ability in industrial anomaly detection has not been systematically studied. To bridge this gap, we present MMAD, the first-ever full-spectrum MLLMs benchmark in industrial Anomaly Detection. We defined seven key subtasks of MLLMs in industrial inspection and designed a novel pipeline to generate the MMAD dataset with 39,672 questions for 8,366 industrial images. With MMAD, we have conducted a comprehensive, quantitative evaluation of various state-of-the-art MLLMs. The commercial models performed the best, with the average accuracy of GPT-4o models reaching 74.9%. However, this result falls far short of industrial requirements. Our analysis reveals that current MLLMs still have significant room for improvement in answering questions related to industrial anomalies and defects. We further explore two training-free performance enhancement strategies to help models improve in industrial scenarios, highlighting their promising potential for future research., Comment: The code and data are available at https://github.com/jam-cc/MMAD

Controllable generation, which enables fine-grained control over generated outputs, has emerged as a critical focus in visual generative models. Currently, there are two primary technical approaches in visual generation: diffusion models and autoregressive models. Diffusion models, as exemplified by ControlNet and T2I-Adapter, offer advanced control mechanisms, whereas autoregressive models, despite showcasing impressive generative quality and scalability, remain underexplored in terms of controllability and flexibility. In this study, we introduce Controllable AutoRegressive Modeling (CAR), a novel, plug-and-play framework that integrates conditional control into multi-scale latent variable modeling, enabling efficient control generation within a pre-trained visual autoregressive model. CAR progressively refines and captures control representations, which are injected into each autoregressive step of the pre-trained model to guide the generation process. Our approach demonstrates excellent controllability across various types of conditions and delivers higher image quality compared to previous methods. Additionally, CAR achieves robust generalization with significantly fewer training resources compared to those required for pre-training the model. To the best of our knowledge, we are the first to propose a control framework for pre-trained autoregressive visual generation models., Comment: Code available at: https://github.com/MiracleDance/CAR

Ensuring safety is a key aspect in sequential decision making problems, such as robotics or process control. The complexity of the underlying systems often makes finding the optimal decision challenging, especially when the safety-critical system is time-varying. Overcoming the problem of optimizing an unknown time-varying reward subject to unknown time-varying safety constraints, we propose TVSafeOpt, a new algorithm built on Bayesian optimization with a spatio-temporal kernel. The algorithm is capable of safely tracking a time-varying safe region without the need for explicit change detection. Optimality guarantees are also provided for the algorithm when the optimization problem becomes stationary. We show that TVSafeOpt compares favorably against SafeOpt on synthetic data, both regarding safety and optimality. Evaluation on a realistic case study with gas compressors confirms that TVSafeOpt ensures safety when solving time-varying optimization problems with unknown reward and safety functions., Comment: Accepted to NeurIPS 2024

Categorization, a core cognitive ability in humans that organizes objects based on common features, is essential to cognitive science as well as computer vision. To evaluate the categorization ability of visual AI models, various proxy tasks on recognition from datasets to open world scenarios have been proposed. Recent development of Large Multimodal Models (LMMs) has demonstrated impressive results in high-level visual tasks, such as visual question answering, video temporal reasoning, etc., utilizing the advanced architectures and large-scale multimodal instruction tuning. Previous researchers have developed holistic benchmarks to measure the high-level visual capability of LMMs, but there is still a lack of pure and in-depth quantitative evaluation of the most fundamental categorization ability. According to the research on human cognitive process, categorization can be seen as including two parts: category learning and category use. Inspired by this, we propose a novel, challenging, and efficient benchmark based on composite blocks, called ComBo, which provides a disentangled evaluation framework and covers the entire categorization process from learning to use. By analyzing the results of multiple evaluation tasks, we find that although LMMs exhibit acceptable generalization ability in learning new categories, there are still gaps compared to humans in many ways, such as fine-grained perception of spatial relationship and abstract category understanding. Through the study of categorization, we can provide inspiration for the further development of LMMs in terms of interpretability and generalization., Comment: 39 pages, 28 figures, 4 tables. Accepted at The 35th British Machine Vision Conference (BMVC 2024). Project page at https://fubin29.github.io/Blocks-as-Probes/

Object detectors have shown outstanding performance on various public datasets. However, annotating a new dataset for a new task is usually unavoidable in real, since 1) a single existing dataset usually does not contain all object categories needed; 2) using multiple datasets usually suffers from annotation incompletion and heterogeneous features. We propose a novel problem as "Annotation-incomplete Multi-dataset Detection", and develop an end-to-end multi-task learning architecture which can accurately detect all the object categories with multiple partially annotated datasets. Specifically, we propose an attention feature extractor which helps to mine the relations among different datasets. Besides, a knowledge amalgamation training strategy is incorporated to accommodate heterogeneous features from different sources. Extensive experiments on different object detection datasets demonstrate the effectiveness of our methods and an improvement of 2.17%, 2.10% in mAP can be achieved on COCO and VOC respectively., Comment: 12 pages, 9 figures

3D pre-training is crucial to 3D perception tasks. However, limited by the difficulties in collecting clean 3D data, 3D pre-training consistently faced data scaling challenges. Inspired by semi-supervised learning leveraging limited labeled data and a large amount of unlabeled data, in this work, we propose a novel self-supervised pre-training framework utilizing the real 3D data and the pseudo-3D data lifted from images by a large depth estimation model. Another challenge lies in the efficiency. Previous methods such as Point-BERT and Point-MAE, employ k nearest neighbors to embed 3D tokens, requiring quadratic time complexity. To efficiently pre-train on such a large amount of data, we propose a linear-time-complexity token embedding strategy and a training-efficient 2D reconstruction target. Our method achieves state-of-the-art performance in 3D classification and few-shot learning while maintaining high pre-training and downstream fine-tuning efficiency., Comment: Under review. Pre-print

Much work on the cultural awareness of large language models (LLMs) focuses on the models' sensitivity to geo-cultural diversity. However, in addition to cross-cultural differences, there also exists common ground across cultures. For instance, a bridal veil in the United States plays a similar cultural-relevant role as a honggaitou in China. In this study, we introduce a benchmark dataset CUNIT for evaluating decoder-only LLMs in understanding the cultural unity of concepts. Specifically, CUNIT consists of 1,425 evaluation examples building upon 285 traditional cultural-specific concepts across 10 countries. Based on a systematic manual annotation of cultural-relevant features per concept, we calculate the cultural association between any pair of cross-cultural concepts. Built upon this dataset, we design a contrastive matching task to evaluate the LLMs' capability to identify highly associated cross-cultural concept pairs. We evaluate 3 strong LLMs, using 3 popular prompting strategies, under the settings of either giving all extracted concept features or no features at all on CUNIT Interestingly, we find that cultural associations across countries regarding clothing concepts largely differ from food. Our analysis shows that LLMs are still limited to capturing cross-cultural associations between concepts compared to humans. Moreover, geo-cultural proximity shows a weak influence on model performance in capturing cross-cultural associations., Comment: COLM 2024

Fully understanding performance is a growing challenge when building next-generation cloud systems. Often these systems build on next-generation hardware, and evaluation in realistic physical testbeds is out of reach. Even when physical testbeds are available, visibility into essential system aspects is a challenge in modern systems where system performance depends on often sub-$\mu s$ interactions between HW and SW components. Existing tools such as performance counters, logging, and distributed tracing provide aggregate or sampled information, but remain insufficient for understanding individual requests in-depth. In this paper, we explore a fundamentally different approach to enable in-depth understanding of cloud system behavior at the software and hardware level, with (almost) arbitrarily fine-grained visibility. Our proposal is to run cloud systems in detailed full-system simulations, configure the simulators to collect detailed events without affecting the system, and finally assemble these events into end-to-end system traces that can be analyzed by existing distributed tracing tools.

The high contrast and spatial resolution requirements for directly imaging exoplanets requires effective coordination of wavefront control, coronagraphy, observation techniques, and post-processing algorithms. However, even with this suite of tools, identifying and retrieving exoplanet signals embedded in resolved scattered light regions can be extremely challenging due to the increased noise from scattered light off the circumstellar disk and the potential misinterpretation of the true nature of the detected signal. This issue pertains not only to imaging terrestrial planets in habitable zones within zodiacal and exozodiacal emission but also to young planets embedded in circumstellar, transitional, and debris disks. This is particularly true for H{\alpha} detection of exoplanets in transitional disks. This work delves into recent H{\alpha} observations of three transitional disks systems with MagAO-X, an extreme adaptive optics system for the 6.5-meter Magellan Clay telescope. We employed angular differential imaging (ADI) and simultaneous spectral differential imaging (SSDI) in combination with KLIP, a PCA algorithm in post-processing, for optimal starlight suppression and quasi-static noise removal. We discuss the challenges in protoplanet identification with MagAO-X in environments rich with scattered and reflected light from disk structures and explore a potential solution for removing noise contributions from real astronomical objects with current observation and post-processing techniques.

Improving the performance and reducing the cost of cloud data systems is increasingly challenging. Data processing units (DPUs) are a promising solution, but utilizing them for data processing needs characterizing the new hardware and recognizing their capabilities and constraints. We hence propose DPDPU, a platform for holistically exploiting DPUs to optimize data processing tasks that are critical to performance and cost. It seeks to fill the semantic gap between DPUs and data processing systems and handle DPU heterogeneity with three engines dedicated to compute, networking, and storage. This paper describes our vision, DPDPU's key components, their associated utilization challenges, as well as the current progress and future plans.

The next generation of extreme adaptive optics (AO) must be calibrated exceptionally well to achieve the desired contrast for ground-based direct imaging exoplanet targets. Current wavefront sensing and control system responses deviate from lab calibration throughout the night due to non linearities in the wavefront sensor (WFS) and signal loss. One cause of these changes is the optical gain (OG) effect, which shows that the difference between actual and reconstructed wavefronts is sensitive to residual wavefront errors from partially corrected turbulence. This work details on-sky measurement of optical gain on MagAO-X, an extreme AO system on the Magellan Clay 6.5m. We ultimately plan on using a method of high-temporal frequency probes on our deformable mirror to track optical gain on the Pyramid WFS. The high-temporal frequency probes, used to create PSF copies at 10-22 lambda /D, are already routinely used by our system for coronagraph centering and post-observation calibration. This method is supported by the OG measurements from the modal response, measured simultaneously by sequenced pokes of each mode. When tracked with DIMM measurements, optical gain calibrations show a clear dependence on Strehl Ratio, and this relationship is discussed. This more accurate method of calibration is a crucial next step in enabling higher fidelity correction and post processing techniques for direct imaging ground based systems., Comment: 7 pages, 6 figures, proceedings of Adaptive Optics Systems IX at SPIE Astronomical Telescopes + Instrumentation 2024

MagAO-X is the extreme coronagraphic adaptive optics (AO) instrument for the 6.5-meter Magellan Clay telescope and is currently undergoing a comprehensive batch of upgrades. One innovation that the instrument features is a deformable mirror (DM) dedicated for non-common path aberration correction (NCPC) within the coronagraph arm. We recently upgraded the 97 actuator NCPC DM with a 1000 actuator Boston Micromachines Kilo-DM which serves to (1) correct non-common path aberrations which hamper performance at small inner-working angles, (2) facilitate focal-plane wavefront control algorithms (e.g., electric field conjugation) and (3) enable 10 kHz correction speeds (up from 2 kHz) to assist post-AO, real-time low-order wavefront control. We present details on the characterization and installation of this new DM on MagAO-X as part of our efforts to improve deep contrast performance for imaging circumstellar objects in reflected light. Pre-installation procedures included use of a Twyman-Green interferometer to build an interaction matrix for commanding the DM surface, in closed-loop, to a flat state for seamless integration into the instrument. With this new NCPC DM now installed, we report on-sky results from the MagAO-X observing run in March -- May 2024 for the Focus Diversity Phase Retrieval and implicit Electric Field Conjugation algorithms for quasistatic speckle removal and in-situ Strehl ratio optimization, respectively., Comment: 9 pages, 9 figures, proceedings of SPIE Astronomical Telescopes + Instrumentation 2024

MagAO-X is the coronagraphic extreme adaptive optics system for the 6.5 m Magellan Clay Telescope. We report the results of commissioning the first phase of MagAO-X. Components now available for routine observations include: the >2 kHz high-order control loop consisting of a 97 actuator woofer deformable mirror (DM), a 2040 actuator tweeter DM, and a modulated pyramid wavefront sensor (WFS); classical Lyot coronagraphs with integrated low-order (LO) WFS and control using a third 97-actuator non-common path correcting (NCPC) DM; broad band imaging in g, r, i, and z filters with two EMCCDs; simultaneous differential imaging in H-alpha; and integral field spectroscopy with the VIS-X module. Early science results include the discovery of an H-alpha jet, images of accreting protoplanets at H-alpha, images of young extrasolar giant planets in the optical, discovery of new white dwarf companions, resolved images of evolved stars, and high-contrast images of circumstellar disks in scattered light in g-band (500 nm). We have commenced an upgrade program, called "Phase II", to enable high-contrast observations at the smallest inner working angles possible. These upgrades include a new 952 actuator NCPC DM to enable coronagraphic wavefront control; phase induced amplitude apodization coronagraphs; new fast cameras for LOWFS and Lyot-LOWFS; and real-time computer upgrades. We will report the status of these upgrades and results of first on-sky testing in March-May 2024., Comment: Presented at SPIE Astronomical Telescopes

High-contrast imaging data analysis depends on removing residual starlight from the host star to reveal planets and disks. Most observers do this with principal components analysis (i.e. KLIP) using modes computed from the science images themselves. These modes may not be orthogonal to planet and disk signals, leading to over-subtraction. The wavefront sensor data recorded during the observation provide an independent signal with which to predict the instrument point-spread function (PSF). MagAO-X is an extreme adaptive optics (ExAO) system for the 6.5-meter Magellan Clay telescope and a technology pathfinder for ExAO with GMagAO-X on the upcoming Giant Magellan Telescope. MagAO-X is designed to save all sensor information, including kHz-speed wavefront measurements. Our software and compressed data formats were designed to record the millions of training samples required for machine learning with high throughput. The large volume of image and sensor data lets us learn a PSF model incorporating all the information available. This will eventually allow us to probe smaller star-planet separations at greater sensitivities, which will be needed for rocky planet imaging., Comment: 9 pages, 5 figures, proceedings of Adaptive Optics Systems IX at SPIE Astronomical Telescopes + Instrumentation 2024

Instance-incremental learning (IIL) focuses on learning continually with data of the same classes. Compared to class-incremental learning (CIL), the IIL is seldom explored because IIL suffers less from catastrophic forgetting (CF). However, besides retaining knowledge, in real-world deployment scenarios where the class space is always predefined, continual and cost-effective model promotion with the potential unavailability of previous data is a more essential demand. Therefore, we first define a new and more practical IIL setting as promoting the model's performance besides resisting CF with only new observations. Two issues have to be tackled in the new IIL setting: 1) the notorious catastrophic forgetting because of no access to old data, and 2) broadening the existing decision boundary to new observations because of concept drift. To tackle these problems, our key insight is to moderately broaden the decision boundary to fail cases while retain old boundary. Hence, we propose a novel decision boundary-aware distillation method with consolidating knowledge to teacher to ease the student learning new knowledge. We also establish the benchmarks on existing datasets Cifar-100 and ImageNet. Notably, extensive experiments demonstrate that the teacher model can be a better incremental learner than the student model, which overturns previous knowledge distillation-based methods treating student as the main role., Comment: 14 pages

Multilingual multimodal reasoning is a core component in achieving human-level intelligence. However, most existing benchmarks for multilingual multimodal reasoning struggle to differentiate between models of varying performance; even language models without visual capabilities can easily achieve high scores. This leaves a comprehensive evaluation of leading multilingual multimodal models largely unexplored. In this work, we introduce M4U, a novel and challenging benchmark for assessing the capability of multi-discipline multilingual multimodal understanding and reasoning. M4U contains 8,931 samples covering 64 disciplines across 16 subfields in Science, Engineering, and Healthcare in Chinese, English, and German. Using M4U, we conduct extensive evaluations of 21 leading Large Multimodal Models (LMMs) and Large Language Models (LLMs) with external tools. The evaluation results show that the state-of-the-art model, GPT-4o, achieves only 47.6% average accuracy on M4U. Additionally, we observe that the leading LMMs exhibit significant language preferences. Our in-depth analysis indicates that leading LMMs, including GPT-4o, suffer performance degradation when prompted with cross-lingual multimodal questions, such as images with key textual information in Chinese while the question is in German. We believe that M4U can serve as a crucial tool for systematically evaluating LMMs based on their multilingual multimodal reasoning capabilities and monitoring their development. The homepage, codes and data are public available., Comment: Work in progress

In this paper we give unified formulas for the numbers of representations of positive integers as sums of four generalized $m$-gonal numbers, and as restricted sums of four squares under a linear condition, respectively. These formulas are given as $\mathbb{Z}$-linear combinations of Hurwitz class numbers. As applications, we prove several Zhi-Wei Sun's conjectures. As by-products, we obtain formulas for expressing the Fourier coefficients of $\vartheta(\tau,z)^4$, $\eta(\tau)^{12}$, $\eta(\tau)^4$ and $\eta(\tau)^8\eta(2\tau)^8$ in terms of the Hurwitz class numbers, respectively. The proof is based on the theory of Jacobi forms., Comment: 12 pages, comments welcome!

Logical clocks are a fundamental tool to establish causal ordering of events in a distributed system. They have been applied in weakly consistent storage systems, causally ordered broadcast, distributed snapshots, deadlock detection, and distributed system debugging. However, prior logical clock constructs fail to work in an open network with Byzantine participants. In this work, we present Chrono, a novel logical clock system that targets such challenging environment. We first redefine causality properties among distributed processes under the Byzantine failure model. To enforce these properties, Chrono defines a new validator abstraction for building fault-tolerant logical clocks. Furthermore, our validator abstraction is customizable: Chrono includes multiple backend implementations for the abstraction, each with different security-performance trade-offs. We have applied Chrono to build two decentralized applications, a mutual exclusive service and a weakly consistent key-value store. Chrono adds only marginal overhead compared to systems that tolerate no Byzantine faults. It also out-performs state-of-the-art BFT total order protocols by significant margins.

We propose to embed the atomic magnetometer (AM) into an optical Mach-Zehnder interferometer (MZI). We analyze the effect of amplification of the Faraday rotation angle of the probe laser light, by properly postselecting the path-information state of the laser photons when passing through the MZI. The proposed scheme provides a postselected metrological protocol of probing weak magnetic fields, having a potential to further enhance the sensitivity of the nowadays state-of-the-art optical AM., Comment: 5 pages, 3 figures

In the field of emotion recognition and Human-Machine Interaction (HMI), personalised approaches have exhibited their efficacy in capturing individual-specific characteristics and enhancing affective prediction accuracy. However, personalisation techniques often face the challenge of limited data for target individuals. This paper presents our work on an enhanced personalisation strategy, that leverages data augmentation to develop tailored models for continuous valence and arousal prediction. Our proposed approach, Distance Weighting Augmentation (DWA), employs a weighting-based augmentation method that expands a target individual's dataset, leveraging distance metrics to identify similar samples at the segment-level. Experimental results on the MuSe-Personalisation 2023 Challenge dataset demonstrate that our method significantly improves the performance of features sets which have low baseline performance, on the test set. This improvement in poor-performing features comes without sacrificing performance on high-performing features. In particular, our method achieves a maximum combined testing CCC of 0.78, compared to the reported baseline score of 0.76 (reproduced at 0.72). It also achieved a peak arousal and valence scores of 0.81 and 0.76, compared to reproduced baseline scores of 0.76 and 0.67 respectively. Through this work, we make significant contributions to the advancement of personalised affective computing models, enhancing the practicality and adaptability of data-level personalisation in real world contexts.

Deep learning models, particularly those based on transformers, often employ numerous stacked structures, which possess identical architectures and perform similar functions. While effective, this stacking paradigm leads to a substantial increase in the number of parameters, posing challenges for practical applications. In today's landscape of increasingly large models, stacking depth can even reach dozens, further exacerbating this issue. To mitigate this problem, we introduce LORS (LOw-rank Residual Structure). LORS allows stacked modules to share the majority of parameters, requiring a much smaller number of unique ones per module to match or even surpass the performance of using entirely distinct ones, thereby significantly reducing parameter usage. We validate our method by applying it to the stacked decoders of a query-based object detector, and conduct extensive experiments on the widely used MS COCO dataset. Experimental results demonstrate the effectiveness of our method, as even with a 70\% reduction in the parameters of the decoder, our method still enables the model to achieve comparable or, Comment: 9 pages, 5 figures, 11 tables, CVPR2024 accepted

When physical testbeds are out of reach for evaluating a networked system, we frequently turn to simulation. In today's datacenter networks, bottlenecks are rarely at the network protocol level, but instead in end-host software or hardware components, thus current protocol-level simulations are inadequate means of evaluation. End-to-end simulations covering these components on the other hand, simply cannot achieve the required scale with feasible simulation performance and computational resources. In this paper, we address this with SplitSim, a simulation framework for end-to-end evaluation for large-scale network and distributed systems. To this end, SplitSim builds on prior work on modular end-to-end simulations and combines this with key elements to achieve scalability. First, mixed fidelity simulations judiciously reduce detail in simulation of parts of the system where this can be tolerated, while retaining the necessary detail elsewhere. SplitSim then parallelizes bottleneck simulators by decomposing them into multiple parallel but synchronized processes. Next, SplitSim provides a profiler to help users understand simulation performance and where the bottlenecks are, so users can adjust the configuration. Finally SplitSim provides abstractions to make it easy for users to build complex large-scale simulations. Our evaluation demonstrates SplitSim in multiple large-scale case studies., Comment: 12 pages, under submission to peer-reviewed conference

As ByteDance's business expands, the substantial infrastructure expenses associated with centralized Content Delivery Network (CDN) networks have rendered content distribution costs prohibitively high. In response, we embarked on exploring a peer-to-peer (P2P) network as a promising solution to alleviate the escalating costs of content distribution. However, the decentralized nature of P2P often introduces performance challenges, given the diversity and dispersion of peer devices. This study introduces Swarm, ByteDance's innovative hybrid system for video streaming. Swarm seamlessly integrates the robustness of a conventional CDN with the cost-efficiency of a decentralized P2P network. Its primary aim is to provide users with reliable streaming quality while minimizing traffic expenses. To achieve this, Swarm employs a centralized control plane comprised of a tracker cluster, overseeing a data plane with numerous edge residual resources. The tracker also takes on the responsibility of mapping clients to servers. Addressing the performance disparities among individual peer servers, Swarm utilizes our proprietary multipath parallel transmission method for communication between clients and peer servers. Operating stably for six years, Swarm now manages over a hundred thousand peer servers, serving nearly a hundred million users daily and saving the company hundreds of millions of RMB annually. Experimental results affirm that, while significantly cutting costs, Swarm performs on par with traditional CDNs.

In this paper, we investigate the intersection of large generative AI models and cloud-native computing architectures. Recent large models such as ChatGPT, while revolutionary in their capabilities, face challenges like escalating costs and demand for high-end GPUs. Drawing analogies between large-model-as-a-service (LMaaS) and cloud database-as-a-service (DBaaS), we describe an AI-native computing paradigm that harnesses the power of both cloud-native technologies (e.g., multi-tenancy and serverless computing) and advanced machine learning runtime (e.g., batched LoRA inference). These joint efforts aim to optimize costs-of-goods-sold (COGS) and improve resource accessibility. The journey of merging these two domains is just at the beginning and we hope to stimulate future research and development in this area.

The topology between Bloch states in reciprocal space has attracted tremendous attention in recent years. The quantum geometry of the band structure is composed of quantum metric as real part and berry curvature as imaginary part. While the Berry curvature, the Berry curvature dipole and Berry connection polarizability have been recently revealed by the first order anomalous hall, second order and third order nonlinear Hall effect respectively, the quantum metric induced second order nonlinear transverse and longitudinal response in topological antiferromagnetic material MnBi2Te4 was only very recently reported. Here we demonstrate the similar third order nonlinear transport properties in the topological antiferromagnetic CoNb3S6. We observed that the third order nonlinear longitudinal V3{\omega} xx increase significantly at the antiferromagnetic transition temperature TN ~ 29 K, which was probably induced by the quantum metric without time-reversal symmetry or inversion symmetry. Besides, temperature-dependent nonlinear behaviour was observed in the first order I-V curve below the Neel temperature TN, which was not reported in MnBi2Te4 and FeSn. Such nonlinear I-V behaviour hints for the possible existence of Charge Density Wave (CDW) state, which has been discovered in its sister material FeNb3S6. Simultaneously, two plateaus in the third order nonlinear longitudinal V3{\omega} xx~ I^{\omega} curve are observed, which is also speculated to be related with the possible CDW state. However, the genuine mechanism for the first order nonlinear I-V and its relation with the third order nonlinear transport call for more experimental investigations and theoretical interpretation. Our work provides a way to explore third harmonic nonlinear transport and interaction with magnetic order and CDW., Comment: 16pages,4figures

Two-dimensional single-crystal metals are highly sought after for next-generation technologies. Here, we report large-area (>10^4 {\mu}m2), single-crystal two-dimensional gold with thicknesses down to a single-nanometer level, employing an atomic-level-precision chemical etching approach. The ultrathin thickness and single-crystal quality endow two-dimensional gold with unique properties including significantly quantum-confinement-augmented optical nonlinearity, low sheet resistance, high transparency and excellent mechanical flexibility. By patterning the two-dimensional gold into nanoribbon arrays, extremely-confined near-infrared plasmonic resonances are further demonstrated with quality factors up to 5. The freestanding nature of two-dimensional gold allows its straightforward manipulation and transfer-printing for integration with other structures. The developed two-dimensional gold provides an emerging platform for fundamental studies in various disciplines and opens up new opportunities for applications in high-performance ultrathin optoelectronic, photonic and quantum devices.

The lack of centralized control, combined with highly dynamic adversarial behaviors, makes data durability a challenge in decentralized storage systems. In this work, we introduce a new storage system, Vault, that offers strong data durability guarantees in a fully decentralized, permission-less setting. Vault leverages the rateless property of erasure code to encode each data object into an infinite stream of encoding fragments. To ensure durability in the presence of dynamic Byzantine behaviors and targeted attacks, an infinite sequence of storage nodes are randomly selected to store encoding fragments. Encoding generation and candidate selection are fully decentralized: When necessary, Vault nodes use a gossip protocol and a publically verifiable selection proof to determine new fragments. Simulations and large-scale EC2 experiments demonstrate that Vault provides close-to-ideal mean-time-to-data-loss (MTTDL) with low storage redundancy, scales to more than 10,000 nodes, and attains performance comparable to IPFS

Logical clocks are a fundamental tool to establish causal ordering of events in a distributed system. They have been used as the building block in weakly consistent storage systems, causally ordered broadcast, distributed snapshots, deadlock detection, and distributed system debugging. However, prior logical clock constructs fail to work in a permissionless setting with Byzantine participants. In this work, we introduce Chrono, a novel logical clock system that targets an open and decentralized network. Chrono introduces a new logical clock construct, the Decaying Onion Bloom Clock (DOBC), that scales independently to the size of the network. To tolerate Byzantine behaviors, Chrono leverages non-uniform incrementally verifiable computation (IVC) to efficiently prove and verify the construction of DOBC clocks. We have applied Chrono to build two decentralized applications, a weakly consistent key-value store and an anti-censorship social network, demonstrating the power of scalable, verifiable causality in a decentralized network.

Efficiently aggregating trained neural networks from local clients into a global model on a server is a widely researched topic in federated learning. Recently, motivated by diminishing privacy concerns, mitigating potential attacks, and reducing communication overhead, one-shot federated learning (i.e., limiting client-server communication into a single round) has gained popularity among researchers. However, the one-shot aggregation performances are sensitively affected by the non-identical training data distribution, which exhibits high statistical heterogeneity in some real-world scenarios. To address this issue, we propose a novel one-shot aggregation method with layer-wise posterior aggregation, named FedLPA. FedLPA aggregates local models to obtain a more accurate global model without requiring extra auxiliary datasets or exposing any private label information, e.g., label distributions. To effectively capture the statistics maintained in the biased local datasets in the practical non-IID scenario, we efficiently infer the posteriors of each layer in each local model using layer-wise Laplace approximation and aggregate them to train the global parameters. Extensive experimental results demonstrate that FedLPA significantly improves learning performance over state-of-the-art methods across several metrics., Comment: 39pages, 4 figures

Query-based object detectors have made significant advancements since the publication of DETR. However, most existing methods still rely on multi-stage encoders and decoders, or a combination of both. Despite achieving high accuracy, the multi-stage paradigm (typically consisting of 6 stages) suffers from issues such as heavy computational burden, prompting us to reconsider its necessity. In this paper, we explore multiple techniques to enhance query-based detectors and, based on these findings, propose a novel model called GOLO (Global Once and Local Once), which follows a two-stage decoding paradigm. Compared to other mainstream query-based models with multi-stage decoders, our model employs fewer decoder stages while still achieving considerable performance. Experimental results on the COCO dataset demonstrate the effectiveness of our approach.

Despite recent advances in semantic segmentation, an inevitable challenge is the performance degradation caused by the domain shift in real applications. Current dominant approach to solve this problem is unsupervised domain adaptation (UDA). However, the absence of labeled target data in UDA is overly restrictive and limits performance. To overcome this limitation, a more practical scenario called semi-supervised domain adaptation (SSDA) has been proposed. Existing SSDA methods are derived from the UDA paradigm and primarily focus on leveraging the unlabeled target data and source data. In this paper, we highlight the significance of exploiting the intra-domain information between the labeled target data and unlabeled target data. Instead of solely using the scarce labeled target data for supervision, we propose a novel SSDA framework that incorporates both Inter and Intra Domain Mixing (IIDM), where inter-domain mixing mitigates the source-target domain gap and intra-domain mixing enriches the available target domain information, and the network can capture more domain-invariant features. We also explore different domain mixing strategies to better exploit the target domain information. Comprehensive experiments conducted on the GTA5 to Cityscapes and SYNTHIA to Cityscapes benchmarks demonstrate the effectiveness of IIDM, surpassing previous methods by a large margin., Comment: 7 pages, 4 figures

Large-scale well-annotated datasets are of great importance for training an effective object detector. However, obtaining accurate bounding box annotations is laborious and demanding. Unfortunately, the resultant noisy bounding boxes could cause corrupt supervision signals and thus diminish detection performance. Motivated by the observation that the real ground-truth is usually situated in the aggregation region of the proposals assigned to a noisy ground-truth, we propose DIStribution-aware CalibratiOn (DISCO) to model the spatial distribution of proposals for calibrating supervision signals. In DISCO, spatial distribution modeling is performed to statistically extract the potential locations of objects. Based on the modeled distribution, three distribution-aware techniques, i.e., distribution-aware proposal augmentation (DA-Aug), distribution-aware box refinement (DA-Ref), and distribution-aware confidence estimation (DA-Est), are developed to improve classification, localization, and interpretability, respectively. Extensive experiments on large-scale noisy image datasets (i.e., Pascal VOC and MS-COCO) demonstrate that DISCO can achieve state-of-the-art detection performance, especially at high noise levels. Code is available at https://github.com/Correr-Zhou/DISCO., Comment: Accepted by BMVC2024

In quantum precision metrology, the famous result of Heisenberg limit scaling as $1/N$ (with $N$ the number of probes) can be surpassed by considering nonlinear coupling measurement. In this work, we consider the most practice-relevant quadratic nonlinear coupling and show that the metrological precision can be enhanced from the $1/N^{\frac{3}{2}}$ super-Heisenberg scaling to $1/N^2$, by simply employing a pre- and post-selection (PPS) technique, but not using any expensive quantum resources such as quantum entangled state of probes., Comment: 6 pages, 4 figures

Two-dimensional (2D) material heterostructures have attracted considerable attention owing to their interesting and novel physical properties, which expand the possibilities for future optoelectronic, photovoltaic, and nanoelectronic applications. A portable, fast, and deterministic transfer technique is highly needed for the fabrication of heterostructures. Herein, we report a fast half wet poly(dimethylsiloxane) (PDMS) transfer process utilizing the change of adhesion energy with the help of micron-sized water droplets. Using this method, a vertical stacking of the WS2/Bi2Se3 heterostructure with a straddling band configuration is successfully assembled on a fluorophlogopite substrate. Thanks to the complementary band gaps and high efficiency of interfacial charge transfer, the photodetector based on the heterostructure exhibits a superior responsivity of 109.9 A/W for a visible incident light at 473 nm and 26.7 A/W for a 1064 nm near-infrared illumination. Such high photoresponsivity of the heterostructure demonstrates that our transfer method not only owns time efficiency but also ensures high quality of the heterointerface. Our study may open new pathways to the fast and massive fabrication of various vertical 2D heterostructures for applications in twistronics/valleytronics and other band engineering devices.

Photo-Thermoelectric (PTE) response is usually one of the main working mechanisms for photodetectors. However, as another fast and easier way to measure thermoelectric characteristics of materials, it can also reveal important physics such as electric-phonon coupling, electron-electron correlation, etc. Recently, the spin entropy related to magnetic order transition which contributes to thermoelectric power is attracting more and more attention. Here, we demonstrate the PTE response can be reshaped when Ca3Ru2O7 undergoes meta-magnetic phase (MMP) transition driven by both temperature and magnetic field. Firstly, a sign change is observed crossing TS = 48 K and the linear polarization angle dependent PTE current maximizes along a-axis above TS while maximizes along b-axis below TS, which indicates that the antiferromagnetic spin order contributes to such spatial anisotropy. Secondly, in the temperature range of around 40 ~ 50 K, the PTE current is found to be sharply suppressed when external magnetic field is applied in plane along a-axis but is only gradually suppressed when applied field is along b-axis which gives out two critical fields. We attribute such suppression of PTE current under magnetic field to the suppression of the spin entropy in the phase transition between the antiferromagnetic state and the MMP state and the H-T phase diagrams of Ca3Ru2O7 is redrawn accordingly. Compared to previously work which trying to understand the magnetic phase transition in Ca3Ru2O7, such as neutron scattering, specific heat, and other advanced transport measurements, our work provides a more convenient yet efficient method, which may also find applications in other correlated spin materials in general.

2D ferroelectric \{beta}-InSe/graphene heterostructure was fabricated by mechanical exfoliation, and the carrier dynamics crossing the heterostructure interface has been systematically investigated by Raman, photoluminescence and transient absorption measurements. Due to the efficient interfacial photo excited electron transfer and photogating effect from trapped holes, the heterostructure devices demonstrate superior performance with maximum responsivity of 2.12*10e4 A/W, detectivity of 1.73*10e14 Jones and fast response time (241 us) under {\lambda} = 532 nm laser illumination. Furthermore, the photo responses influenced by ferroelectric polarization field are investigated. Our work confirms ferroelectric \{beta}-InSe/graphene heterostructure as an outstanding material platform for sensitive optoelectronic application.