Search

Federated learning can train models without directly providing local data to the server. However, the frequent updating of the local model brings the problem of large communication overhead. Recently, scholars have achieved the communication efficiency of federated learning mainly by model compression. But they ignore two problems: 1) network state of each client changes dynamically; 2) network state among clients is not the same. The clients with poor bandwidth update local model slowly, which leads to low efficiency. To address this challenge, we propose a communication-efficient federated learning algorithm with adaptive compression under dynamic bandwidth (called AdapComFL). Concretely, each client performs bandwidth awareness and bandwidth prediction. Then, each client adaptively compresses its local model via the improved sketch mechanism based on his predicted bandwidth. Further, the server aggregates sketched models with different sizes received. To verify the effectiveness of the proposed method, the experiments are based on real bandwidth data which are collected from the network topology we build, and benchmark datasets which are obtained from open repositories. We show the performance of AdapComFL algorithm, and compare it with existing algorithms. The experimental results show that our AdapComFL achieves more efficient communication as well as competitive accuracy compared to existing algorithms.

Quantum Federated Learning (QFL) enables collaborative training of a Quantum Machine Learning (QML) model among multiple clients possessing quantum computing capabilities, without the need to share their respective local data. However, the limited availability of quantum computing resources poses a challenge for each client to acquire quantum computing capabilities. This raises a natural question: Can quantum computing capabilities be deployed on the server instead? In this paper, we propose a QFL framework specifically designed for classical clients, referred to as CC-QFL, in response to this question. In each iteration, the collaborative training of the QML model is assisted by the shadow tomography technique, eliminating the need for quantum computing capabilities of clients. Specifically, the server constructs a classical representation of the QML model and transmits it to the clients. The clients encode their local data onto observables and use this classical representation to calculate local gradients. These local gradients are then utilized to update the parameters of the QML model. We evaluate the effectiveness of our framework through extensive numerical simulations using handwritten digit images from the MNIST dataset. Our framework provides valuable insights into QFL, particularly in scenarios where quantum computing resources are scarce.

Integration of physics principles with data-driven methods has attracted great attention in recent few years. In this study, a physics-informed dynamic mode decomposition (piDMD) method, where the mass conservation law is integrated with a purely data-driven DMD method, is developed for fast prediction of the spatiotemporal dynamics of solid volume fraction distribution in bubbling fluidized beds. Assessment of the prediction ability using both piDMD and DMD is performed using the CFD-DEM results as the benchmark: Both DMD and piDMD can predict the short-term behaviour of solid volume fraction reasonably well, but piDMD outperforms the DMD in both qualitative and quantitative comparisons; With respect to their long-term ability, the piDMD-based prediction of the instantaneous solid volume fraction distributions is qualitatively correct although the accuracy needs to be improved, and the predicted time-averaged radial and axial profiles are satisfactory; Whereas the DMD-based prediction of instantaneous snapshots and time-averaged results is completely nonphysical. Present study provides a fast and relatively accurate method for predicting the hydrodynamics of gas-solid flows.

With the increasingly strengthened data privacy act and the difficult data centralization, Federated Learning (FL) has become an effective solution to collaboratively train the model while preserving each client's privacy. FedAvg is a standard aggregation algorithm that makes the proportion of dataset size of each client as aggregation weight. However, it can't deal with non-independent and identically distributed (non-i.i.d) data well because of its fixed aggregation weights and the neglect of data distribution. In this paper, we propose an aggregation strategy that can effectively deal with non-i.i.d dataset, namely FedGraph, which can adjust the aggregation weights adaptively according to the training condition of local models in whole training process. The FedGraph takes three factors into account from coarse to fine: the proportion of each local dataset size, the topology factor of model graphs, and the model weights. We calculate the gravitational force between local models by transforming the local models into topology graphs. The FedGraph can explore the internal correlation between local models better through the weighted combination of the proportion each local dataset, topology structure, and model weights. The proposed FedGraph has been applied to the MICCAI Federated Tumor Segmentation Challenge 2021 (FeTS) datasets, and the validation results show that our method surpasses the previous state-of-the-art by 2.76 mean Dice Similarity Score. The source code will be available at Github.

In the inter-domain network, a route leak occurs when a routing announcement is propagated outside of its intended scope, which is a violation of the agreed routing policy. The route leaks can disrupt the internet traffic and cause large outages. The accurately detection of route leaks requires the share of AS business relationship information of ASes. However, the business relationship information between ASes is confidential due to economic issues. Thus, ASes are usually unwilling to revealing this information to the other ASes, especially their competitors. Recent advancements in federated learning make it possible to share data while maintaining privacy. Motivated by this, in this paper we study the route leak problem by considering the privacy of business relationships between ASes, and propose a method for route leak detection with privacy guarantee by using blockchain-based federated learning framework, in which ASes can train a global detection model without revealing their business relationships directly. Moreover, the proposed method provides a self-validation scheme by labeling AS triples with local routing policies, which mitigates route leaks' lack of ground truth. We evaluate the proposed method under a variety of datasets including unbalanced and balanced datasets. The different deployment strategies of the proposed method under different topologies are also examined. The results show that the proposed method has a better performance in detecting route leaks than a single AS detection regardless of whether using balanced or unbalanced datasets. In the analysis of the deployment, the results show that ASes with more peers have more possible route leaks and can contribute more on the detection of route leaks with the proposed method.

The CNN-based methods have achieved impressive results in medical image segmentation, but they failed to capture the long-range dependencies due to the inherent locality of the convolution operation. Transformer-based methods are recently popular in vision tasks because of their capacity for long-range dependencies and promising performance. However, it lacks in modeling local context. In this paper, taking medical image segmentation as an example, we present MISSFormer, an effective and powerful Medical Image Segmentation tranSFormer. MISSFormer is a hierarchical encoder-decoder network with two appealing designs: 1) A feed-forward network is redesigned with the proposed Enhanced Transformer Block, which enhances the long-range dependencies and supplements the local context, making the feature more discriminative. 2) We proposed Enhanced Transformer Context Bridge, different from previous methods of modeling only global information, the proposed context bridge with the enhanced transformer block extracts the long-range dependencies and local context of multi-scale features generated by our hierarchical transformer encoder. Driven by these two designs, the MISSFormer shows a solid capacity to capture more discriminative dependencies and context in medical image segmentation. The experiments on multi-organ and cardiac segmentation tasks demonstrate the superiority, effectiveness and robustness of our MISSFormer, the experimental results of MISSFormer trained from scratch even outperform state-of-the-art methods pre-trained on ImageNet. The core designs can be generalized to other visual segmentation tasks. The code has been released on Github: https://github.com/ZhifangDeng/MISSFormer, Comment: 10 pages, 6 figures

Bohmian mechanics is a non-relativistic quantum theory based on a particle approach. In this paper we study the Schr\"odinger equation with rapidly oscillating potential and the associated Bohmian trajectory. We prove that the corresponding Bohmian trajectory converges locally in measure, and the limit coincides with the Bohmian trajectory for the effective Schr\"{o}dinger equation on a finite time interval. This is beneficial for the efficient simulation of the Bohmian trajectories in oscillating potential fields.

The choice of activation function in deep networks has a significant effect on the training dynamics and task performance. At present, the most effective and widely-used activation function is ReLU. However, because of the non-zero mean, negative missing and unbounded output, ReLU is at a potential disadvantage during optimization. To this end, we introduce a novel activation function to manage to overcome the above three challenges. The proposed nonlinearity, namely "Soft-Root-Sign" (SRS), is smooth, non-monotonic, and bounded. Notably, the bounded property of SRS distinguishes itself from most state-of-the-art activation functions. In contrast to ReLU, SRS can adaptively adjust the output by a pair of independent trainable parameters to capture negative information and provide zero-mean property, which leading not only to better generalization performance, but also to faster learning speed. It also avoids and rectifies the output distribution to be scattered in the non-negative real number space, making it more compatible with batch normalization (BN) and less sensitive to initialization. In experiments, we evaluated SRS on deep networks applied to a variety of tasks, including image classification, machine translation and generative modelling. Our SRS matches or exceeds models with ReLU and other state-of-the-art nonlinearities, showing that the proposed activation function is generalized and can achieve high performance across tasks. Ablation study further verified the compatibility with BN and self-adaptability for different initialization.

The aim of this paper is to analyze the long-time dynamical behavior of the solution for a degenerate wave equation with time-dependent damping term $\partial_{tt}u + \beta(t)\partial_tu = \mathcal{L}u(x,t) + f(u)$ on a bounded domain $\Omega\subset\mathbb{R}^N$ with Dirichlet boundary conditions. Under some restrictions on $\beta(t)$ and critical growth restrictions on the nonlinear term $f$, we will prove the local and global well-posedness of the solution and derive the existence of a pullback attractor for the process associated with the degenerate damped hyperbolic problem.

Convolutional neural networks (CNNs) are inherently suffering from massively redundant computation (FLOPs) due to the dense connection pattern between feature maps and convolution kernels. Recent research has investigated the sparse relationship between channels, however, they ignored the spatial relationship within a channel. In this paper, we present a novel convolutional operator, namely comb convolution, to exploit the intra-channel sparse relationship among neurons. The proposed convolutional operator eliminates nearly 50% of connections by inserting uniform mappings into standard convolutions and removing about half of spatial connections in convolutional layer. Notably, our work is orthogonal and complementary to existing methods that reduce channel-wise redundancy. Thus, it has great potential to further increase efficiency through integrating the comb convolution to existing architectures. Experimental results demonstrate that by simply replacing standard convolutions with comb convolutions on state-of-the-art CNN architectures (e.g., VGGNets, Xception and SE-Net), we can achieve 50% FLOPs reduction while still maintaining the accuracy., Comment: 15 pages

The paper gives a detailed study of long-time dynamics generated by weakly damped wave equations in bounded 3D domains where the damping exponent depends explicitly on time and may change sign. It is shown that in the case when the non-linearity is superlinear, the considered equation remains dissipative if the weighted mean value of the dissipation rate remains positive and that the conditions of this type are not sufficient in the linear case. Two principally different cases are considered. In the case when this mean is uniform (which corresponds to deterministic dissipation rates), it is shown that the considered system possesses smooth uniform attractors as well as non-autonomous exponential attractors. In the case where the mean is not uniform (which corresponds to the random dissipation rate, for instance, when this dissipation rate is generated by the Bernoulli process), the tempered random attractor is constructed. In contrast to the usual situation, this random attractor is expected to have infinite Hausdorff and fractal dimension. The simplified model example which demonstrates infinite-dimensionality of the random attractor is also presented.

We study protection of a qubit that transfer through a decoherence noise by quantum control technique. In this work, we assume that the communication participants have some side information about the qubit. Our aim is to take fully advantage of given side information to design a quantum control scheme that makes the output state close to the input state as better as possible. Especially, in this paper, we consider the quantum composite control (QCC) structure, which has shown to capture a better balance between fidelity and success probability, i.e., two essential measures for protection. Comparison has been made between our scheme and previous ones. We show there's advantages in two aspects. On one hand, when compared with the scheme that has taking fully consideration of classical side information but not quantum one, our scheme achieves higher fidelity. Inversely, when the scheme with quantum side information but not classical one is considered, our scheme also has nontrivial improvement.

As the web expands in data volume and in geographical distribution, centralized search methods become inefficient, leading to increasing interest in cooperative information retrieval, e.g., federated text retrieval (FTR). Different from existing centralized information retrieval (IR) methods, in which search is done on a logically centralized document collection, FTR is composed of a number of peers, each of which is a complete search engine by itself. To process a query, FTR requires firstly the identification of promising peers that host the relevant documents and secondly the retrieval of the most relevant documents from the selected peers. Most of the existing methods only apply traditional IR techniques that treat each text collection as a single large document and utilize term matching to rank the collections. In this paper, we formalize the problem and identify the properties of FTR, and analyze the feasibility of extending LSI with clustering to adapt to FTR, based on which a novel approach called Cluster-based Distributed Latent Semantic Indexing (C-DLSI) is proposed. C-DLSI distinguishes the topics of a peer with clustering, captures the local LSI spaces within the clusters, and consider the relations among these LSI spaces, thus providing more precise characterization of the peer. Accordingly, novel descriptors of the peers and a compatible local text retrieval are proposed. The experimental results show that C-DLSI outperforms existing methods.

Representing meaning in the form of high dimensional vectors is a common and powerful tool in biologically inspired architectures. While the meaning of a set of concepts can be summarized by taking a (possibly weighted) sum of their associated vectors, this has generally been treated as a one-way operation. In this paper we show how a technique built to aid sparse vector decomposition allows in many cases the exact recovery of the inputs and weights to such a sum, allowing a single vector to represent an entire set of vectors from a dictionary. We characterize the number of vectors that can be recovered under various conditions, and explore several ways such a tool can be used for vector-based reasoning., Comment: In Biologically Inspired Cognitive Architectures 2018

In natural images, the scales (thickness) of object skeletons may dramatically vary among objects and object parts, making object skeleton detection a challenging problem. We present a new convolutional neural network (CNN) architecture by introducing a novel hierarchical feature integration mechanism, named Hi-Fi, to address the skeleton detection problem. The proposed CNN-based approach has a powerful multi-scale feature integration ability that intrinsically captures high-level semantics from deeper layers as well as low-level details from shallower layers. % By hierarchically integrating different CNN feature levels with bidirectional guidance, our approach (1) enables mutual refinement across features of different levels, and (2) possesses the strong ability to capture both rich object context and high-resolution details. Experimental results show that our method significantly outperforms the state-of-the-art methods in terms of effectively fusing features from very different scales, as evidenced by a considerable performance improvement on several benchmarks., Comment: IJCAI2018

The origin of transition metal vacancies and their effects on the properties of ThCr2Si2-type compounds have been less studied and poorly understood. Here we carefully investigate the structure, physical properties, and electronic structure for a series of lanthanum cobalt arsenides with nominal composition of LaCoxAs2 (1.6 < = x < = 2.1). It is revealed that the occupancy of Co can be tuned between 1.98(1) and 1.61(1). The structural analyses based on X-ray and neutron diffractions show the existence of Co vacancies results from charge balance due to the formation of bond between As-As. These Co vacancies affect the magnetic and electrical properties greatly, adjusting the Curie temperature from 205 to 47 K and increasing the resistivity by more than 100%. First principles calculations indicate that the Co vacancies weaken the spin polarization and reduce the density of states at the Fermi level, resulting in decreased Curie temperature and increased resistivity, respectively. Our results address the importance of transition metal vacancies in ThCr2Si2-type materials and offer a reliable route to tune the magnetism of ThCr2Si2-type structure., Comment: 10 pages, 4 figures