Search

A common strategy for Parameter-Efficient Fine-Tuning (PEFT) of pre-trained Vision Transformers (ViTs) involves adapting the model to downstream tasks by learning a low-rank adaptation matrix. This matrix is decomposed into a product of down-projection and up-projection matrices, with the bottleneck dimensionality being crucial for reducing the number of learnable parameters, as exemplified by prevalent methods like LoRA and Adapter. However, these low-rank strategies typically employ a fixed bottleneck dimensionality, which limits their flexibility in handling layer-wise variations. To address this limitation, we propose a novel PEFT approach inspired by Singular Value Decomposition (SVD) for representing the adaptation matrix. SVD decomposes a matrix into the product of a left unitary matrix, a diagonal matrix of scaling values, and a right unitary matrix. We utilize Householder transformations to construct orthogonal matrices that efficiently mimic the unitary matrices, requiring only a vector. The diagonal values are learned in a layer-wise manner, allowing them to flexibly capture the unique properties of each layer. This approach enables the generation of adaptation matrices with varying ranks across different layers, providing greater flexibility in adapting pre-trained models. Experiments on standard downstream vision tasks demonstrate that our method achieves promising fine-tuning performance.

One-dimensional van der Waals (1D vdW) materials, characterized by atomic chains bonded ionically or covalently in one direction and held together by van der Waals interactions in the perpendicular directions, have recently gained intensive attention due to their exceptional functions. In this work, we report the discovery of 1D ionic-bonded structures in NiSe nanowires. Utilizing aberration-corrected scanning transmission electron microscopy, we identified four distinct structural phases composed of two fundamental 1D building blocks: a triangle-shaped unit and a parallelogram-shaped unit. These phases can transform into one another through topotactic combinations of the structural units. Density functional theory calculations reveal that these structural units are bound by ionic bonds, unlike the van der Waals forces typically found in 1D vdW materials. The diverse arrangements of these building blocks may give rise to unique electronic structures and magnetic properties, paving the way for designing advanced materials with novel functionalities.

Let $\Gamma$ be a connected $7$-valent symmetric Cayley graph on a finite non-abelian simple group $G$. If $\Gamma$ is not normal, Li {\em et al.} [On 7-valent symmetric Cayley graphs of finite simple groups, J. Algebraic Combin. 56 (2022) 1097-1118] characterised the group pairs $(\mathrm{soc}(\mathrm{Aut}(\Gamma)/K),GK/K)$, where $K$ is a maximal intransitive normal subgroup of $\mathrm{Aut}(\Gamma)$. In this paper, we improve this result by proving that if $\Gamma$ is not normal, then $\mathrm{Aut}(\Gamma)$ contains an arc-transitive non-abelian simple normal subgroup $T$ such that $G

With the development of satellite communication technology, satellite-terrestrial integrated networks (STIN), which integrate satellite networks and ground networks, can realize seamless global coverage of communication services. Confronting the intricacies of network dynamics, the diversity of resource heterogeneity, and the unpredictability of user mobility, dynamic resource allocation within networks faces formidable challenges. Digital twin (DT), as a new technique, can reflect a physical network to a virtual network to monitor, analyze, and optimize the physical network. Nevertheless, in the process of constructing the DT model, the deployment location and resource allocation of DTs may adversely affect its performance. Therefore, we propose a STIN model, which alleviates the problem of insufficient single-layer deployment flexibility of the traditional edge network by deploying DTs in multi-layer nodes in a STIN. To address the challenge of deploying DTs in the network, we propose multi-layer DT deployment in a STIN to reduce system delay. Then we adopt a multi-agent reinforcement learning (MARL) scheme to explore the optimal strategy of the DT multi-layer deployment problem. The implemented scheme demonstrates a notable reduction in system delay, as evidenced by simulation outcomes.

The isomorphism problem for digraphs is a fundamental problem in graph theory. This problem for Cayley digraphs has been extensively investigated over the last half a century. In this paper, we consider this problem for $m$-Cayley digraphs which are generalization of Cayley digraphs. Let $m$ be a positive integer. A digraph admitting a group $G$ of automorphisms acting semiregularly on the vertices with exactly $m$ orbits is called an $m$-Cayley digraph of $G$. In particular, $1$-Cayley digraph is just the Cayley digraph. We first characterize the normalizer of $G$ in the full automorphism group of an $m$-Cayley digraph of a finite group $G$. This generalizes a similar result for Cayley digraph achieved by Godsil in 1981. Then we use this to study the isomorphisms of $m$-Cayley digraphs. The CI-property of a Cayley digraph (CI stands for `Cayley isomorphism') and the DCI-groups (whose Cayley digraphs are all CI-digraphs) are two key topics in the study of isomorphisms of Cayley digraphs. We generalize these concepts into $m$-Cayley digraphs by defining $m$CI- and $m$PCI-digraphs, and correspondingly, $m$DCI- and $m$PDCI-groups. Analogues to Babai's criterion for CI-digraphs are given for $m$CI- and $m$PCI-digraphs, respectively. With these we then classify finite $m$DCI-groups for each $m\geq 2$, and finite $m$PDCI-groups for each $m\geq 4$. Similar results are also obtained for $m$-Cayley graphs. Note that 1DCI-groups are just DCI-groups, and the classification of finite DCI-groups is a long-standing open problem that has been worked on a lot., Comment: 29

We present an implementation of alchemical free energy simulations at the quantum mechanical level by directly interpolating the electronic Hamiltonian. The method is compatible with any level of electronic structure theory and requires only one quantum calculation for each molecular dynamics step in contrast to multiple energy evaluations that would be needed when interpolating the ground-state energies. We demonstrate the correctness and applicability of the technique by computing alchemical free energy changes of gas-phase molecules, with both nuclear and electron creation/annihilation. We also show an initial application to first-principles pKa calculation for solvated molecules where we quantum mechanically annihilate a bonded proton.

Primordial black holes (PBHs) are hypothetical objects formed at the early epoch of the universe, which could be a type of dark matter (DM) candidate without the need for new particles. The abundance of PBH DM has been constrained strictly by many observations.In this work, with the radio observations of Fornax and Segue I, we constrain the abundance of PBH in dwarf spheroidal galaxies through the synchrotron self-Compton (SSC) effect of Hawking radiation electrons. By selecting optimal sources, we obtain the constraints on the fraction of PBH DM down to $\sim10^{-3}$ for Segue I and $\sim10^{-5}$ for Fornax at asteroidal mass. We also predict that, with 100 hours of future observation by the Square Kilometer Array, the SSC approach could place constraints comparable to the current strictest results for PBHs of $<5\times10^{15}\,{\rm g}$. Better projected constraints can be obtained by including the inverse Compton scattering on cosmic microwave background photons., Comment: 9 pages, 6 figures, accepted for publication in PRD

Mobile edge computing (MEC) is a promising computing paradigm that offers users proximity and instant computing services for various applications, and it has become an essential component of the Internet of Things (IoT). However, as compute-intensive services continue to emerge and the number of IoT devices explodes, MEC servers are confronted with resource limitations. In this work, we investigate a task-offloading framework for device-assisted edge computing, which allows MEC servers to assign certain tasks to auxiliary IoT devices (ADs) for processing. To facilitate efficient collaboration among task IoT devices (TDs), the MEC server, and ADs, we propose an incentive-driven pricing and task allocation scheme. Initially, the MEC server employs the Vickrey auction mechanism to recruit ADs. Subsequently, based on the Stackelberg game, we analyze the interactions between TDs and the MEC server. Finally, we establish the optimal service pricing and task allocation strategy, guided by the Stackelberg model and priority settings. Simulation results show that the proposed scheme dramatically improves the utility of the MEC server while safeguarding the interests of TDs and ADs, achieving a triple-win scenario., Comment: This paper is accepted by IEEE Globecom 2024

In iron-based superconductor Fe(Se,Te), a flat band-like feature near the Fermi level was observed around the Brillouin zone center in the superconducting state. It is under debate whether this is the evidence on the presence of the BCS-BEC crossover in the superconductor. High-resolution laser-based angle-resolved photoemission measurements are carried out on high quality single crystals of FeSe0.45Te0.55 superconductor to address the issue. By employing different polarization geometries, we have resolved and isolated the dyz band and the topological surface band, making it possible to study their superconducting behaviors separately. The dyz band alone does not form a flat band-like feature in the superconducting state and the measured dispersion can be well described by the BCS picture. We find that the flat band-like feature is formed from the combination of the dyz band and the topological surface state band in the superconducting state. These results reveal the origin of the flat band-like feature and rule out the presence of BCS-BEC crossover in Fe(Se,Te) superconductor.

In high temperature cuprate superconductors, it was found that in the overdoped region the superfluid density decreases with the increase of hole doping. One natural question is whether there exists normal fluid in the superconducting state in the overdoped region. In this paper, we have carried out high-resolution ultra-low temperature laser-based angle-resolved photoemission measurements on a heavily overdoped Bi2212 sample with a $T_{\mathrm{c}}$ of 48 K. We find that this heavily overdoped Bi2212 remains in the strong coupling regime with $2 \mathit{\Delta}_0 / k_{\mathrm{B}} T_{\mathrm{c}}=5.8$. The single-particle scattering rate is very small along the nodal direction ($\sim$5 meV) and increases as the momentum moves from the nodal to the antinodal regions. A hard superconducting gap opening is observed near the antinodal region with the spectral weight at the Fermi level fully suppressed to zero. The normal fluid is found to be negligibly small in the superconducting state of this heavily overdoped Bi2212. These results provide key information to understand the high $T_\mathrm{c}$ mechanism in the cuprate superconductors., Comment: 13 pages, 4 figures

We introduce the first version of GPU4PySCF, a module that provides GPU acceleration of methods in PySCF. As a core functionality, this provides a GPU implementation of two-electron repulsion integrals (ERIs) for contracted basis sets comprising up to g functions using Rys quadrature. As an illustration of how this can accelerate a quantum chemistry workflow, we describe how to use the ERIs efficiently in the integral-direct Hartree-Fock Fock build and nuclear gradient construction. Benchmark calculations show a significant speedup of two orders of magnitude with respect to the multi-threaded CPU Hartree-Fock code of PySCF, and performance comparable to other GPU-accelerated quantum chemical packages including GAMESS and QUICK on a single NVIDIA A100 GPU.

A novel thermodynamic phenomenon has been observed in the quantum Ba\~{n}ados-Teitelboim-Zanelli (qBTZ) black hole, utilizing generalized free energy and Kramer escape rate. This phenomenon also reveals the unique property of the quantum black hole. The stochastic thermal motion of various thermodynamic states within the black hole system induces phase transitions, under the influence of generalized free energy which obtained by extending Maxwell's construction. Through the analysis of Kramer escape rate, it is discovered that the qBTZ black hole thermodynamic system exhibits a bounce effect. Furthermore, the overall thermodynamic picture of the qBTZ black hole has been obtained under different quantum backreactions., Comment: 8 pages, 3 figures

A response to commenter Ke Lan's comment on our paper published in Nature Communications (2023)14:5782 by J. Yan et al

Temporal Video Grounding (TVG) aims to localize a moment from an untrimmed video given the language description. Since the annotation of TVG is labor-intensive, TVG under limited supervision has accepted attention in recent years. The great success of vision-language pre-training guides TVG to follow the traditional "pre-training + fine-tuning" paradigm, however, the pre-training process would suffer from a lack of temporal modeling and fine-grained alignment due to the difference of data nature between pre-train and test. Besides, the large gap between pretext and downstream tasks makes zero-shot testing impossible for the pre-trained model. To avoid the drawbacks of the traditional paradigm, we propose AutoTVG, a new vision-language pre-training paradigm for TVG that enables the model to learn semantic alignment and boundary regression from automatically annotated untrimmed videos. To be specific, AutoTVG consists of a novel Captioned Moment Generation (CMG) module to generate captioned moments from untrimmed videos, and TVGNet with a regression head to predict localization results. Experimental results on Charades-STA and ActivityNet Captions show that, regarding zero-shot temporal video grounding, AutoTVG achieves highly competitive performance with in-distribution methods under out-of-distribution testing, and is superior to existing pre-training frameworks with much less training data., Comment: Technique Report

While deep learning has achieved significant success in various domains, its application to logic circuit design has been limited due to complex constraints and strict feasibility requirement. However, a recent generative deep neural model, "Circuit Transformer", has shown promise in this area by enabling equivalence-preserving circuit transformation on a small scale. In this paper, we introduce a logic synthesis rewriting operator based on the Circuit Transformer model, named "ctrw" (Circuit Transformer Rewriting), which incorporates the following techniques: (1) a two-stage training scheme for the Circuit Transformer tailored for logic synthesis, with iterative improvement of optimality through self-improvement training; (2) integration of the Circuit Transformer with state-of-the-art rewriting techniques to address scalability issues, allowing for guided DAG-aware rewriting. Experimental results on the IWLS 2023 contest benchmark demonstrate the effectiveness of our proposed rewriting methods., Comment: In IWLS 2024

Large language models (LLMs) have demonstrated astonishing capabilities in natural language processing (NLP) tasks, sparking interest in their application to professional domains with higher specialized requirements. However, restricted access to closed-source LLMs via APIs and the difficulty in collecting massive high-quality datasets pose obstacles to the development of large language models in education fields of various courses. Given these challenges, we propose CourseGPT-zh, a course-oriented education LLM that supports customization and low-cost deployment. To address the comprehensiveness and diversity requirements of course-specific corpora, we design a high-quality question-answering corpus distillation framework incorporating prompt optimization, which effectively mines textbook knowledge and enhances its diversity. Moreover, considering the alignment of LLM responses with user needs, a novel method for discrete prompt optimization based on LLM-as-Judge is introduced. During optimization, this framework leverages the LLM's ability to reflect on and exploit error feedback and patterns, allowing for prompts that meet user needs and preferences while saving response length. Lastly, we obtain CourseGPT-zh based on the open-source LLM using parameter-efficient fine-tuning. Experimental results show that our discrete prompt optimization framework effectively improves the response quality of ChatGPT, and CourseGPT-zh exhibits strong professional capabilities in specialized knowledge question-answering, significantly outperforming comparable open-source models.

In this work, we introduce a differentiable implementation of the local natural orbital coupled cluster (LNOCC) method within the automatic differentiation framework of the PySCFAD package. The implementation is comprehensively tuned for enhanced performance, which enables the calculation of first-order static response properties on medium-sized molecular systems using coupled cluster theory with single, double, and perturbative triple excitations [CCSD(T)]. We evaluate the accuracy of our method by benchmarking it against the canonical CCSD(T) reference for nuclear gradients, dipole moments, and geometry optimizations. In addition, we demonstrate the possibility of property calculations for chemically interesting systems through the computation of bond orders and M\"ossbauer spectroscopy parameters for a [NiFe]-hydrogenase active site model, along with the simulation of infrared (IR) spectra via ab initio LNO-CC molecular dynamics for a protonated water hexamer.

Parameter-efficient fine-tuning for pre-trained Vision Transformers aims to adeptly tailor a model to downstream tasks by learning a minimal set of new adaptation parameters while preserving the frozen majority of pre-trained parameters. Striking a balance between retaining the generalizable representation capacity of the pre-trained model and acquiring task-specific features poses a key challenge. Currently, there is a lack of focus on guiding this delicate trade-off. In this study, we approach the problem from the perspective of Singular Value Decomposition (SVD) of pre-trained parameter matrices, providing insights into the tuning dynamics of existing methods. Building upon this understanding, we propose a Residual-based Low-Rank Rescaling (RLRR) fine-tuning strategy. This strategy not only enhances flexibility in parameter tuning but also ensures that new parameters do not deviate excessively from the pre-trained model through a residual design. Extensive experiments demonstrate that our method achieves competitive performance across various downstream image classification tasks, all while maintaining comparable new parameters. We believe this work takes a step forward in offering a unified perspective for interpreting existing methods and serves as motivation for the development of new approaches that move closer to effectively considering the crucial trade-off mentioned above. Our code is available at \href{https://github.com/zstarN70/RLRR.git}{https://github.com/zstarN70/RLRR.git}.

Language, a prominent human ability to express through sequential symbols, has been computationally mastered by recent advances of large language models (LLMs). By predicting the next word recurrently with huge neural models, LLMs have shown unprecedented capabilities in understanding and reasoning. Circuit, as the "language" of electronic design, specifies the functionality of an electronic device by cascade connections of logic gates. Then, can circuits also be mastered by a a sufficiently large "circuit model", which can conquer electronic design tasks by simply predicting the next logic gate? In this work, we take the first step to explore such possibilities. Two primary barriers impede the straightforward application of LLMs to circuits: their complex, non-sequential structure, and the intolerance of hallucination due to strict constraints (e.g., equivalence). For the first barrier, we encode a circuit as a memory-less, depth-first traversal trajectory, which allows Transformer-based neural models to better leverage its structural information, and predict the next gate on the trajectory as a circuit model. For the second barrier, we introduce an equivalence-preserving decoding process, which ensures that every token in the generated trajectory adheres to the specified equivalence constraints. Moreover, the circuit model can also be regarded as a stochastic policy to tackle optimization-oriented circuit design tasks. Experimentally, we trained a Transformer-based model of 88M parameters, named "Circuit Transformer", which demonstrates impressive performance in end-to-end logic synthesis. With Monte-Carlo tree search, Circuit Transformer significantly improves over resyn2 while retaining strict equivalence, showcasing the potential of generative AI in conquering electronic design challenges.

Reducing test inputs that trigger bugs is crucial for efficient debugging. Delta debugging is the most popular approach for this purpose. When test inputs need to conform to certain specifications, existing delta debugging practice encounters a validity problem: it blindly applies reduction rules, producing a large number of invalid test inputs that do not satisfy the required specifications. This overall diminishing effectiveness and efficiency becomes even more pronounced when the specifications extend beyond syntactical structures. Our key insight is that we should leverage input generators, which are aware of these specifications, to generate valid reduced inputs, rather than straightforwardly performing reduction on test inputs. In this paper, we propose a generator-based delta debugging method, namely GReduce, which derives validity-preserving reducers. Specifically, given a generator and its execution, demonstrating how the bug-inducing test input is generated, GReduce searches for other executions on the generator that yield reduced, valid test inputs. The evaluation results on five benchmarks show that GReduce significantly outperforms the state-of-the-art syntax-based reducer Perses: 28.5%, 34.6%, 75.6% in size of those from Perses with 17.5%, 0.6%, 65.4% time taken by Perses, and also outperforms the state-of-the-art choice-sequence-based reducer Hypothesis, demonstrating the effectiveness, efficiency, and versatility of GReduce.

Numerous applications at the photon-starved regime require a free-space coupling singlephoton detector with a large active area, low dark count rate (DCR), and superior time resolutions. Here,we developed a superconducting microstrip single-photon detector (SMSPD), with a large active area of 260 um in diameter, a DCR of ~5 kcps, and a low time jitter of ~171 ps, operated at near-infrared of 1550 nm. As a demonstration, we applied the detector to a single-pixel galvanometer scanning system and successfully reconstructed object information in depth and intensity using a time-correlated photon counting technology., Comment: 12 pages, 6 figures

The combination of mobile edge computing (MEC) and radio frequency-based wireless power transfer (WPT) presents a promising technique for providing sustainable energy supply and computing services at the network edge. This study considers a wireless-powered mobile edge computing system that includes a hybrid access point (HAP) equipped with a computing unit and multiple Internet of Things (IoT) devices. In particular, we propose a novel muti-user cooperation scheme to improve computation performance, where collaborative clusters are dynamically formed. Each collaborative cluster comprises a source device (SD) and an auxiliary device (AD), where the SD can partition the computation task into various segments for local processing, offloading to the HAP, and remote execution by the AD with the assistance of the HAP. Specifically, we aims to maximize the weighted sum computation rate (WSCR) of all the IoT devices in the network. This involves jointly optimizing collaboration, time and data allocation among multiple IoT devices and the HAP, while considering the energy causality property and the minimum data processing requirement of each device. Initially, an optimization algorithm based on the interior-point method is designed for time and data allocation. Subsequently, a priority-based iterative algorithm is developed to search for a near-optimal solution to the multi-user collaboration scheme. Finally, a deep learning-based approach is devised to further accelerate the algorithm's operation, building upon the initial two algorithms. Simulation results show that the performance of the proposed algorithms is comparable to that of the exhaustive search method, and the deep learning-based algorithm significantly reduces the execution time of the algorithm., Comment: Accepted to IEEE Open Journal of the Communications Society

With the development of the Internet of Things (IoT), certain IoT devices have the capability to not only accomplish their own tasks but also simultaneously assist other resource-constrained devices. Therefore, this paper considers a device-assisted mobile edge computing system that leverages auxiliary IoT devices to alleviate the computational burden on the edge computing server and enhance the overall system performance. In this study, computationally intensive tasks are decomposed into multiple partitions, and each task partition can be processed in parallel on an IoT device or the edge server. The objective of this research is to develop an efficient online algorithm that addresses the joint optimization of task partitioning and parallel scheduling under time-varying system states, posing challenges to conventional numerical optimization methods. To address these challenges, a framework called online task partitioning action and parallel scheduling policy generation (OTPPS) is proposed, which is based on deep reinforcement learning (DRL). Specifically, the framework leverages a deep neural network (DNN) to learn the optimal partitioning action for each task by mapping input states. Furthermore, it is demonstrated that the remaining parallel scheduling problem exhibits NP-hard complexity when considering a specific task partitioning action. To address this subproblem, a fair and delay-minimized task scheduling (FDMTS) algorithm is designed. Extensive evaluation results demonstrate that OTPPS achieves near-optimal average delay performance and consistently high fairness levels in various environmental states compared to other baseline schemes., Comment: Accepted to IEEE Internet of Things Journal

Low earth orbit (LEO) satellite communications can provide ubiquitous and reliable services, making it an essential part of the Internet of Everything network. Beam hopping (BH) is an emerging technology for effectively addressing the issue of low resource utilization caused by the non-uniform spatio-temporal distribution of traffic demands. However, how to allocate multi-dimensional resources in a timely and efficient way for the highly dynamic LEO satellite systems remains a challenge. This paper proposes a joint beam scheduling and power optimization beam hopping (JBSPO-BH) algorithm considering the differences in the geographic distribution of sink nodes. The JBSPO-BH algorithm decouples the original problem into two sub-problems. The beam scheduling problem is modelled as a potential game, and the Nash equilibrium (NE) point is obtained as the beam scheduling strategy. Moreover, the penalty function interior point method is applied to optimize the power allocation. Simulation results show that the JBSPO-BH algorithm has low time complexity and fast convergence and achieves better performance both in throughput and fairness. Compared with greedy-based BH, greedy-based BH with the power optimization, round-robin BH, Max-SINR BH and satellite resource allocation algorithm, the throughput of the proposed algorithm is improved by 44.99%, 20.79%, 156.06%, 15.39% and 8.17%, respectively.