Search

With the urgent demand for generalized deep models, many pre-trained big models are proposed, such as BERT, ViT, GPT, etc. Inspired by the success of these models in single domains (like computer vision and natural language processing), the multi-modal pre-trained big models have also drawn more and more attention in recent years. In this work, we give a comprehensive survey of these models and hope this paper could provide new insights and helps fresh researchers to track the most cutting-edge works. Specifically, we firstly introduce the background of multi-modal pre-training by reviewing the conventional deep learning, pre-training works in natural language process, computer vision, and speech. Then, we introduce the task definition, key challenges, and advantages of multi-modal pre-training models (MM-PTMs), and discuss the MM-PTMs with a focus on data, objectives, network architectures, and knowledge enhanced pre-training. After that, we introduce the downstream tasks used for the validation of large-scale MM-PTMs, including generative, classification, and regression tasks. We also give visualization and analysis of the model parameters and results on representative downstream tasks. Finally, we point out possible research directions for this topic that may benefit future works. In addition, we maintain a continuously updated paper list for large-scale pre-trained multi-modal big models: https://github.com/wangxiao5791509/MultiModal_BigModels_Survey, Accepted by Machine Intelligence Research

Due to the binary spike signals making converting the traditional high-power multiply-accumulation (MAC) into a low-power accumulation (AC) available, the brain-inspired Spiking Neural Networks (SNNs) are gaining more and more attention. However, the binary spike propagation of the Full-Spike Neural Networks (FSNN) with limited time steps is prone to significant information loss. To improve performance, several state-of-the-art SNN models trained from scratch inevitably bring many non-spike operations. The non-spike operations cause additional computational consumption and may not be deployed on some neuromorphic hardware where only spike operation is allowed. To train a large-scale FSNN with high performance, this paper proposes a novel Dual-Stream Training (DST) method which adds a detachable Auxiliary Accumulation Pathway (AAP) to the full spiking residual networks. The accumulation in AAP could compensate for the information loss during the forward and backward of full spike propagation, and facilitate the training of the FSNN. In the test phase, the AAP could be removed and only the FSNN remained. This not only keeps the lower energy consumption but also makes our model easy to deploy. Moreover, for some cases where the non-spike operations are available, the APP could also be retained in test inference and improve feature discrimination by introducing a little non-spike consumption. Extensive experiments on ImageNet, DVS Gesture, and CIFAR10-DVS datasets demonstrate the effectiveness of DST.

The sparsity of Deep Neural Networks is well investigated to maximize the performance and reduce the size of overparameterized networks as possible. Existing methods focus on pruning parameters in the training process by using thresholds and metrics. Meanwhile, feature similarity between different layers has not been discussed sufficiently before, which could be rigorously proved to be highly correlated to the network sparsity in this paper. Inspired by interlayer feature similarity in overparameterized models, we investigate the intrinsic link between network sparsity and interlayer feature similarity. Specifically, we prove that reducing interlayer feature similarity based on Centered Kernel Alignment (CKA) improves the sparsity of the network by using information bottleneck theory. Applying such theory, we propose a plug-and-play CKA-based Sparsity Regularization for sparse network training, dubbed CKA-SR, which utilizes CKA to reduce feature similarity between layers and increase network sparsity. In other words, layers of our sparse network tend to have their own identity compared to each other. Experimentally, we plug the proposed CKA-SR into the training process of sparse network training methods and find that CKA-SR consistently improves the performance of several State-Of-The-Art sparse training methods, especially at extremely high sparsity. Code is included in the supplementary materials.

This study investigated the mechanism, prevention measures, and control methods for earthquake disasters typically occurring in mines with thick and hard rock strata. A mine stope with large faults and thick hard rock strata in Hebei Province was taken as the background study object. Then, theoretical analysis and numerical simulation methods were adopted in conjunction with field monitoring to explore how horizontal stress evolves in the thick and hard hanging roofs of such mines, potentially leading to mining earthquakes. Then, based on the obtained results, a mining design method was proposed to reduce the horizontal stress levels of earthquake mitigation. The results showed that, under the control of large faults, semiopen and semiclosed stopes with thick hard rock strata are formed, which cause influentially pressurized and depressurized zones during the evolution of the overburden movements and horizontal stress. It was determined that the stress concentrations mainly originated from the release and transfer of horizontal stress during the rock fractures and movements in the roof areas, which were calculated using a theoretical estimation model. The horizontal stress concentrations formed “counter torques” at both ends of the thick and hard strata, which prevented the support ending due to tensile failures. As a result, the limit spans were increased. This study proposed a mining strategy of using narrow working faces, strip mining processes, and reasonable mining speeds, which could effectively reduce horizontal stress concentrations and consequently prevent and control mining earthquakes. This study’s research results were successfully applied to the mining practices in working face 16103.

Out-of-distribution (OOD) detection is vital to safety-critical machine learning applications and has thus been extensively studied, with a plethora of methods developed in the literature. However, the field currently lacks a unified, strictly formulated, and comprehensive benchmark, which often results in unfair comparisons and inconclusive results. From the problem setting perspective, OOD detection is closely related to neighboring fields including anomaly detection (AD), open set recognition (OSR), and model uncertainty, since methods developed for one domain are often applicable to each other. To help the community to improve the evaluation and advance, we build a unified, well-structured codebase called OpenOOD, which implements over 30 methods developed in relevant fields and provides a comprehensive benchmark under the recently proposed generalized OOD detection framework. With a comprehensive comparison of these methods, we are gratified that the field has progressed significantly over the past few years, where both preprocessing methods and the orthogonal post-hoc methods show strong potential., Comment: Accepted by NeurIPS 2022 Datasets and Benchmarks Track. Codebase: https://github.com/Jingkang50/OpenOOD

In a water-cooled thermoelectric generator (TEG) system, there is an energy-consuming circulating water pump. In order to reduce pump energy consumption caused by backpressure and increase the output power of the entire system, combining numerical simulation and experimentation is adopted to explore the suitable coolant flow rate. Due to the limitations of the experimental conditions, numerical simulation is used to compute the temperature distribution and flow field inside the TEG. Base on the numerical results and the empirical formula, the circulating pump energy consumption is calculated. The maximum power output of the thermoelectric module (TEM) at the corresponding temperature difference is obtained by experiment. Finally, the maximum net output power of the module is revealed and the relationship between the coolant flow rate and average temperature of the hot end of the TEMs is proposed, which can serve as a theoretical basis for cooling water flow rate management of the TEG system.

Ensuring the safety and efficacy of Cyber-Physical Systems (CPSs) is challenging due to the large variability of their operating environment. Model checking has been proposed for validation of CPSs, but the models of the environment are either too specific to capture the variability of the environment, or too abstract to provide counter-examples interpretable by experts in the application domain. Domain-specific solutions to this problem require expertise in both formal methods and the application domain, which prevents effective application of model checking in CPSs validation. A domain-independent framework based on timed-automata is proposed for abstraction and refinement of environment models during model checking. The framework maintains an abstraction tree of environment models, which provides interpretable counter-examples while ensuring coverage of environment behaviors. With the framework, experts in the application domain can effectively use model checking without expertise in formal methods.

During creep aging (CA), through-thickness heterogeneity and in-plane anisotropy in creep and mechanical properties affect the shape forming and performance tailoring. The specimens are first machined from the surface (t1), subsurface (t2) and center (t3) layers along the thickness direction of the 7050 Al alloy plate. Then in the rolling plane of each layer, the specimens are taken along the angles of 0° (L0), 45° (L45) and 90° (L90) to the rolling direction. After CA, the creep strain shows the trends of t1 > t2 > t3 along the thickness direction and L45 > L90 > L0 for in-plane directions, while the yield strength (YS) shows t1 > t3 > t2 and L90 > L0 > L45, respectively. Before CA, the low-angle grain boundaries (LAGBs) increase along the thickness direction, leading to a reduced creep strain. After CA, through-thickness heterogeneous LAGBs contributes to the above trend of YS. After L45 and L90 loading, S, Brass and Copper textures are obviously weaker than that under L0 loading, and show the largest equivalent Schmid factor (ESF) for L45 loading, and that for L0 and L90 conditions are approaching; while the equivalent slip system number (ESSN) shows L90 > L0 > L45. Both ESF and ESSN determine the in-plane anisotropy by affecting dislocation slip, but ESF plays a dominant role.

Although time-dependent springback is minor compared with instantaneous springback, its deformation-history dependent effects may deteriorates the compactness of parts assembly and poses potential hazards to the safety and stability of the equipment systems especially under harsh service conditions. However, time-dependent springback still has not been paid enough attention, and how to accurately predict this phenomenon is still a challenge needed to be solved urgently. In this work, taking high-strength Ti-3Al-2.5V titanium tubular materials (HSTT) upon bending as a case, the experimental exploration of the physical mechanisms and the prediction modeling for time-dependent springback of the HSTT were conducted. First, the residual stress and dislocation evolution of HSTT during bending, instantaneous springback and time-dependent springback are comprehensively experimentally obtained, and the compression-tension uneven residual stress-driven creep deformation mechanism of time-dependent springback for HSTT was revealed. Then, considering stress attenuation, a Viscoelastic-Kelvin constitutive model for time-dependent springback was developed and combined with the stress invariant-based model (SIM) and Chord modulus model to accurately predict the whole process of bending, instantaneous springback and time-dependent springback of the HSTT. Finally, the verified model is employed to explore the evolution rules of time-dependent springback and residual stress of HSTT upon bending, and the deformation-history dependent effects of time-dependent springback for spatial multi-bending component are clarified. The findings and methods of this study could aid in achieving higher precision manufacturing of high performance tubular components.

Surface modification by generating a dense anionic layer on the surface of a substrate membrane is the main method for preparing monovalent selective anion exchange membranes. However, this method has a trade-off between the permselectivity and the stability of the surface layer. To overcome this problem, a simple preparative pathway is devised to covalently immobilize the polyethyleneimine (PEI) onto the surface of an anion exchange membrane (partly quaternized poly (phenylene oxide) (QPPO)). Firstly, a brominated poly (2,6-dimethyl-1, 4-phenylene oxide) (BPPO) based anion membrane was prepared by quaternization. Then, this membrane was modified by electro-deposition of a PEI solution. The composition and structure of the membrane were observed by infrared spectroscopy and scanning electron microscopy. The permselectivity was evaluated by electrodialysis in a Cl - /SO 4 2- system. The experimental results prove that the PEI modification method is an effective, simple, feasible and well-controlled strategy to fabricate mono-valent selective anion membranes.

Many studies have indicated that histone deacetylase (HDAC) inhibitors are promising agents for the treatment of cancer. With the aim to search for novel potent HDAC inhibitors, we designed and synthesized two series of hydroxamates and 2-aminobenzamides compounds as HDAC inhibitors and antitumor agents. Those compounds were investigated for their HDAC enzymatic inhibitory activities and in vitro anti-proliferation activities against diverse cancer cell line (A549, HepG2, MGC80-3 and HCT116). Most of the synthesized compounds displayed potent HDAC inhibitory activity and antiproliferative activity. In particular, Compound 12a, N-(2-aminophenyl)-4-[(4-fluorophenoxy)methyl]benzamide, was shown to have the most HDAC inhibitory activity (70.6% inhibition at 5 μM) and antitumor activity with IC50 value of as low as 3.84 μM against HepG2 human liver hepatocellular carcinoma cell line, more than 4.8–fold lower than CS055 and 5.9–fold lower than CI994. HDAC isoform selectivity assay indicated 12a is a potent HDAC2 inhibitor. Docking study of 12a suggested that it bound tightly to the binding pocket of HDAC2. Further investigation showed that 12a could inhibit the migration and colony formation of A549 cancer cells. Furthermore, 12a remarkably induced apoptosis and G2/M phase cell cycle arrest in A549 cancer cells. Those results indicated that compound 12a could be a promising candidate for treatment of cancer.

We have discovered that trifluoroacetic acid is an efficient catalyst to accelerate the Petasis three-component, one-pot process, reducing reaction times, and offering good to excellent yields. A series of azaarene-derived glycines have been successfully achieved via the TFA-accelerated Petasis reaction. To our best knowledge, the use of TFA to accelerate Petasis reaction has not been reported in the literature. By virtue of the operational simplicity, the present TFA-catalyzed Petasis reaction, capable of shortening reaction times, might find applications in other instances of the widely used Petasis reaction.

Deep learning provides a great potential for in-loop filtering to improve both coding efficiency and subjective quality in video coding. State-of-the-art work focuses on network structure design and employs a single powerful network to solve all problems. In contrast, this paper proposes a deep learning based systematic approach that includes an effective Convolutional Neural Network (CNN) structure, a hierarchical training strategy, and a video codec oriented switchable mechanism. First, we propose a novel CNN structure, i.e., Squeeze-and-Excitation Filtering CNN (SEFCNN), as an optional in-loop filter. To capture the non-linear interaction between channels, the SEFCNN is comprised of two subnets, i.e., Feature EXtracting (FEX) subnet and Feature ENhancing (FEN) subnet. Then, we develop a hierarchical model training strategy to adapt the two subnets to different coding scenarios. For high-rate videos with small artifacts, we train a single global model using the FEX for all types of frames, whereas for low-rate videos with large artifacts, different models are trained using both FEX and FEN for different types of frames. Finally, we propose an adaptive enhancing mechanism which is switchable between the CNN-based and the conventional methods. We selectively apply the CNN model to some frames or some regions in a frame. Experimental results show that the proposed scheme outperforms state-of-the-art work in coding efficiency, while the computational complexity is acceptable after GPU acceleration.

We study exclusive charmonium and bottomonium production in ultra-peripheral heavy-ion collisions and electron-ion collisions within the dipole picture. We employ heavy quarkonium light-front wavefunctions obtained within the basis light-front quantization framework, which has some features of the light-front holographic QCD. We focus on comparison with measurements of exclusive charmonium and bottomonium production in ultraperipheral $pp$, $p$Pb and PbPb collisions at LHC and find reasonable agreement with the cross sections. We also discuss the coherent production cross-section ratios of excited states to the ground state for charmonium and bottomonium, which exhibit insensitivity to the dipole model parameters. We show that electron-ion collisions provide an opportunity for a quantitative study of heavy quarkonium structure., 10 pages, 5 figures

We develop and test an ab initio time-dependent Basis Function (tBF) method to solve non-perturbative and time-dependent problems in quantum mechanics. For our test problem, we apply this method to the Coulomb excitation of the deuteron by an impinging heavy ion. In the tBF method applied to deuterium, we employ wave functions for its bound and excited states to calculate its transition probabilities and the r.m.s. radius during the scattering process. For comparison, corresponding results based on first-order perturbation theory are also provided. For the Coulomb excitation process in a weak and time-varying Coulomb field, where higher-order effects are negligible, we obtain good agreement of the results based on these two methods. The tBF method is then applied to the Coulomb excitation process with stronger external field. The higher-order effects, such as those appearing in the reorientation of the polarization of the deuteron system, are analyzed., Comment: 16 pages, 6 figures

Using the solid reaction method, the Y2O3 4mol%-doped BaZrO3 powder was prepared by BaCO3, ZrO2 and Y2O3 as raw materials, the effect of sintering temperature on the property and microstructure of Y2O3-doped BaZrO3 pellets was investigated, the crucible made of this Y2O3-doped BaZrO3 was utilized to melting the TiNi alloy by the vacuum induction method. The phase constitution of doped powder and the interface reaction between the crucible and the melt were studied by XRD and SEM. The results showed that this Y2O3-doped BaZrO3 consists of two phase (BaZrO3 and Ba2YZrO6), the pellets sintered at 1750°C for 6h reached the maximum relative density of 98.0%. Besides, there was no obvious interaction layer and element diffusion between the crucible and the melt, which may imply that the Y2O3-doped BaZrO3 refractory is a promising candidate for melting titanium alloys.

Various amount Y2O3(1–8 mol%) doped CaO powder was synthesized by the solid state reaction method, the pellete and crucible were fabricated by the cold isostatic pressing and were sintered at 1750°C for 4h. The microstructural characterization was revealed by X-ray diffraction(XRD) and scanning electron microscopy(SEM).The XRD results showed that when Y2O3 doped 2 mol%, metastable CaY2O4 phase existed in CaO grain boundary, when Y2O3 doped 3 mol%–8 mol%, in addition to the above structure, Y2O3 phase also be found in CaO grain boundary. Hydration experiment results showed when Y2O3 doped 0 mol%–4 mol%, CaO had excellent hydration resistance performance, Y2O3 doped 2 mol% had the best hydration resistance, its weight addition stored after 7 weeks (49 days) was only about 0.2 wt%. Melting experiment results showed that it was no reaction between crucible and alloy layer. Oxygen, calcium, titanium, nickle and yttrium element not diffusion between the CaO crucible and TiNi alloy, it was no oxygen content increase after melting.

本研究运用2个行为实验探讨了复杂情境下自我决策、为他人决策和预期他人决策在无意识思维方式和有意识思维方式下的决策表现差异。实验1发现复杂情境中无意识思维方式下,自我决策和为他人决策的决策表现显著优于预期他人决策,自我决策和为他人决策表现没有显著差异。实验2发现复杂情境中有意识思维时,在陌生人条件下,为他人决策表现分数显著高于自我决策与预期他人决策,自我决策和预期他人决策表现无显著差异;在朋友条件下,为他人决策和预期他人决策的决策表现显著优于自我决策,为他人决策和预期他人决策表现无显著差异。研究结果支持了本文提出的决策视角-心理距离作用假设（the Perspective-Distance Effect Hypothesis,PDEH）。