Search

The combination of the Metaverse and intelligent transportation systems (ITS) holds significant developmental promise, especially for visual perception tasks. However, the acquisition of high-quality scene data poses a challenging and expensive endeavor. Meanwhile, the visual disparity between the Metaverse and the physical world poses an impact on the practical applicability of the visual perception tasks. In this paper, a Metaverse Intelligent Traffic Visual Framework, MITVF, is developed to guide the implementation of visual perception tasks in the physical world. Firstly, a two-stage metadata optimization strategy is proposed that can efficiently provide diverse and high-quality scene data for traffic perception models. Specifically, an element reconfigurability strategy is proposed to flexibly combine dynamic and static traffic elements to enrich the data with a low cost. A diffusion model-based metadata optimization acceleration strategy is proposed to achieve efficient improvement of image resolution. Secondly, a Meta-Physical adaptive learning method is proposed, and further applied to visual perception tasks to compensate for the visual disparity between the Metaverse and the physical world. Experimental results show that MITVF achieves a $10\times $ acceleration in optimization speed, ensuring the image quality and reconstructing diverse. Further, MITVF is applied to the traffic object detection task to verify the effectiveness and validity. The performance of the model trained with 5k real data exceeded that of the model trained with 200k real data, with AP50 reaching 67.7%.

Despite neuromorphic computing (NC) technologies offer tremendous potential in executing computationally intensive tasks with high efficiency and low latency, most of existing methods are still difficult to achieve software-comparable accuracy. To address this challenge, we develop a multimodal local-global NC system (MLG-NCS) that can capture local characteristics and exchange global cross-modal information sufficiently. Specifically, a high-density memristor crossbar array is prepared to perform efficient parallel in-memory operations, serving as the fundamental component of the proposed MLG-NCS. To facilitate understanding of the proposed MLG-NCS design, the local feature representation module, the global cross-modal interaction module, and the output module are designed. The experimental results show that the proposed system has advantages in classification accuracy (ranked top three), time consumption (approximately ten times speed up), and latency (about 1.2–15.3 times faster), enabling good inter-related tradeoffs between latency, efficiency, and accuracy. This study is expected to promote the revolution and development of next-generation computing system, which takes a firm step toward artificial general intelligence (AGI).

Accuracy and speed of Lithium-ion battery state of charge (SOC) estimation determine the reliability and stability of electric vehicle (EV), as well as promoting the development of smart home energy management system. Existing SOC estimation approaches embedded in commercial EVs still suffer from limitations of low precision, effectiveness, and relatively low robustness. To address these issues, we propose an internal cascaded neuromorphic computing system (ICNCS) via memristor circuits for EV SOC estimation. Specifically, we use three circuit modules to facilitate the design of the proposed ICNCS. Firstly, a bidirectional gated recurrent unit (Bi-GRU) circuit module is designed, enabling adequate feature extraction from the time-contextual battery information. Secondly, an attention circuit module is proposed to distinguish the useful and unimportant information related to SOC estimation. Thirdly, the Kalman filter module is constructed to eliminate the random noise caused by data transmission and processing. Finally, a series of experiments and analysis demonstrate that the proposed ICNCS has good performances in terms of accuracy (the lowest achievable RMSE and MAE are 0.853 and 0.711, respectively), time efficiency (approximately $16\sim 20$ times), and robustness (anti-noise capacity), indicating an advancement in consumer electronics applications.

Traffic sign detection is of great significance to the development of the Intelligent Transportation System (ITS) as a database for environmental awareness. The main challenges of existing traffic sign detection method are inaccurate small object detection, difficult mobile deployment, and complex working environment. Based on these, a vehicle-mounted adaptive traffic sign detector (VATSD) for small-sized signs in multiple working conditions is proposed in this paper. First, the Backbone of the detector is optimized. A feature tight fusion structure is designed to constitute a new feature extraction module, DCSP, which improves the feature extraction capability and the detection accuracy of small objects with negligible additional parameters. Second, an image enhancement network IENet with an adaptive joint filtering strategy is proposed. The IENet enables the dynamic selection of filters and thus adaptively optimizes low-quality images under multiple conditions to improve the accuracy of subsequent detection tasks. The proposed method has experimented on three traffic sign datasets and the detection accuracy increased by up to 7.6% compared to the original. The proposed detector demonstrates superiority over other state-of-the-art (SOTA) methods in terms of small object detection accuracy, detection speed, and environmental adaptability. Further, we deployed VATSD to Jetson Xavier NX and achieved a detection speed of 21.6 FPS, meeting real-time requirements.

Video sentiment analysis can effectively establish the relationship between the emotion state and the multimodal information, while still suffer from intensive computation and low efficiency, due to the von Neumann computing architecture. Here, we present a brain-inspired hierarchical interactive in-memory computing (IMC) system, which can efficiently solve ‘von Neumann bottleneck’, enabling cross-modal interactions and semantic gap elimination. First, a 1T1M synapse array is fabricated using cost-effective, highly stable, flexible, and eco-friendly carbon materials, offering efficient analog multiply-accumulate operations. To illustrate the complexity of the proposed brain-inspired hierarchical interactive IMC system, three modules are proposed: 1) unimodal extraction module, 2) hierarchical interactive module, 3) output module. Furthermore, the proposed system is validated by applying it to video sentiment analysis. The experimental results demonstrate that the proposed system outperforms the existing state-of-the-art methods with high computational efficiency and good robustness. This work opens up a new way to achieve the deep integration of nanomaterials, deep learning, and modern electronics into IMC.

Mental health problems are an increasingly common social issue severely affecting health and well-being. Multimedia processing technologies via facial expression show appealing prospects in the consumer field for mental health monitoring, while still suffer from intensive computation and low energy efficiency. This paper proposes an energy-efficiency memristive sequencer network (EMSN) for human emotion classification, which offers an environmentally friendly approach for consumers with low cost and easily deployable hardware. Firstly, two-dimensional (2D) materials are employed to construct an eco-friendly memristor, the efficacy and reliability of which are confirmed through performance testing. Then, a sequencer block is proposed using memristive circuits. Notably, it is a core component of the EMSN, consisting of a bidirectional long short-term memory circuit, normalisation circuit module, and multi-layer perception module. After combining some necessary function modules, the EMSN can be achieved. Furthermore, the proposed EMSN is applied for human emotion classification. The experimental results demonstrate that the proposed EMSN has advantages in computational efficiency and classification accuracy compared to existing mainstream methods, indicating an advancement in consumer health monitoring.

Allosteric modulators are important regulation elements that bind the allosteric site beyond the active site, leading to the changes in dynamic and/or thermodynamic properties of the protein. Allosteric modulators have been a considerable interest as potential drugs with high selectivity and safety. However, current experimental methods have limitations to identify allosteric sites. Therefore, molecular dynamics simulation based on empirical force field becomes an important complement of experimental methods. Moreover, the precision and efficiency of current force fields need improvement. Deep learning and reweighting methods were used to train allosteric protein-specific precise force field (named APSF). Multiple allosteric proteins were used to evaluate the performance of APSF. The results indicate that APSFcan capture different types of allosteric pockets and sample multiple energy-minimum reference conformations of allosteric proteins. At the same time, the efficiency of conformation sampling for APSFis higher than that for ff14SB. These findings confirm that the newly developed force field APSFcan be effectively used to identify the allosteric pocket that can be further used to screen potential allosteric drugs based on these pockets.

We present a circuit design of the hierarchical attention network for multimodal affective computing, which can be used in mental health monitoring. Specifically, a kind of cost-effective memristor is fabricated using the albumen protein, and the corresponding testing performance is conducted to ensure its efficiency and stability. Then, considering the hierarchical mechanism inspired by the human limbic system, the nanoscale memristors arranged in a crossbar array configuration are further applied to construct a compact hierarchical attention network that can perform the multimodal affective computing. Furthermore, based on the wearable technology and flexible electronics technology, a mental health monitoring system with low privacy invasiveness, low energy consumption, and low fabrication cost can be designed. Based on the mapping relationship between the multimodal affective computing and mental health, the mental health state of the current user can be monitored. This study is expected to help achieving the deep integration of neuromorphic electronics and mental health monitoring system, further promoting the development of next-generation consumer healthcare technology in smart city.

To enable smart home applications, embedding home monitoring systems with different sensors can be a feasible remedy to capture the multimodal sensory information from daily life. However, the previously developed monitoring systems have high implementation cost, large power consumption, and complicated device configuration, which are not conducive to achieving carbon neutrality and carbon peak goals. Here, we demonstrate a multimodal neuromorphic sensory-processing system with memristor circuits for smart homes, offering a more environmentally friendly approach with low cost and easily deployable hardware. We used three components to facilitate the multimodal neuromorphic sensory-processing system design. First, a memristor crossbar array is fabricated using low-cost, reliable, and eco-friendly 2-D materials; enabling a parallel connection to reduce the system complexity and the deep-learning computational cost. Second, a neuromorphic multimodal sensory-processing module is designed for collecting and processing multiple sensory cues efficiently; to establish accurate depiction of the environment and realize timely response. Third, the biomimetic hierarchical learning module is realized for smart home monitoring and relative applications (e.g., indoor human behavior recognition). The developed multimodal neuromorphic sensory-processing system effectively combines the nanotechnology, intelligent sensing, and deep learning; indicating an advancement in smart home applications.

Neuromorphic computing is expected to realize fast and energy-efficient artificial neural networks and address the inherent limitations of von Neumann architectures in dedicated communication applications. To realize this vision, we identify the existing challenges in neuromorphic computing and provide a specific solution from the perspectives of device, circuit, and system. At the device level, we fabricate a metal-oxide-based memristor with high stability, low power, and good scalability, serving as the fundamental component of a neuromorphic computing system. At the circuit level, the basic circuit units and necessary peripheral circuits are designed to realize efficient vector-matrix multiplication and different functions, including nonlinear activation operation, subtraction operation, added operation, and so on. At the system level, a flexible neuromorphic computing system with a hardware-friendly training approach is proposed, which can perform effective communication with good trade-off between accuracy and time consumption. This study is expected to achieve the deep integration of nanotechnology, energy-efficient integrated circuits, and neuromorphic computing systems into communication applications.

Memristive technologies are attractive due to their non-volatility, high-density, low-power and compatibility with CMOS. For memristive devices, a model corresponding to practical behavioral characteristics is highly favorable for the realization of its neuromorphic system and applications. This paper presents a novel flexible memristor model with electronic resistive switching memory behavior. Firstly, the Ag-Au $/$ MoSe2-doped Se $/$ Au-Ag memristor is prepared using hydrothermal synthesis method and magnetron sputtering method, and its performance test is conducted on an electrochemical workstation. Then, the mathematical model and SPICE circuit model of the Ag-Au $/$ MoSe2-doped Se $/$ Au-Ag memristor are constructed. The model accuracy is verified by using the electrochemical data derived from the performance test. Furthermore, the proposed model is applied to the circuit implementation of spiking neural network with biological mechanism. Finally, computer simulations and analysis are carried out to verify the validity and effectiveness of the entire scheme.

Globally, vinegar is a widely used acidic condiment with unique flavors and rich nutritional value. Aroma can be used as an important factor to measure the quality of vinegar and affect consumers’ choices and preferences. Volatile components provide different aroma characteristics of vinegar and have an important impact on its sensory quality. In this study, the volatile components in solid-state fermented vinegar and liquid-state fermented vinegar were investigated by headspace solid-phase microextraction followed by gas chromatography—mass spectrometry. Principal component analysis (PCA) and hierarchical cluster analysis (HCA) for evaluating and distinguishing between solid-state fermented vinegar and liquid-state fermented vinegar based on their composition of volatile components were carried out. The results showed that a total of 38 different volatile compounds were identified, including 5 alcohols, 17 ethers, 3 acids, 4 heterocyclic compounds, 4 aldehydes, and 5 other volatiles. PCA and HCA have proven to be effective methods for identifying volatile compounds as well as evaluating and comparing of solid-state fermented vinegars and liquid-state fermented vinegars. Multivariate statistical analysis revealed substantial variation between solid-state and liquid-state fermented vinegars.

AbstractVolatile compounds present in green plants exert a certain influence on the atmospheric environment, air quality and human physical and mental health. To provide valuable information for further studies and the improvement of the human environment, a comparative investigation of the volatile compounds and bioactive compounds present in typical coniferous and broad-leaved tree species was conducted. Headspace solid-phase microextraction and gas chromatography-mass spectrometry combined with multivariate statistical analysis was employed for this purpose. By comparing the components of mass spectra with the reference database as well as by calculating RIs with the values reported in the literature, a total of 46 components were successfully identified in the leaves, which differed among leaf samples. The volatiles was assigned to five groups according to the functional group of the compound. The volatiles were mainly terpenes. Through principal component and cluster analyses, the relationship among the tree species was presented.

Intrinsically disordered proteins (IDPs) are proteins without a fixed three-dimensional (3D) structure under physiological conditions and are associated with Parkinson’s disease, Alzheimer’s disease, cancer, cardiovascular disease, amyloidosis, diabetes, and other diseases. Experimental methods can hardly capture the ensemble of diverse conformations for IDPs. Molecular dynamics (MD) simulations can sample continuous conformations that might provide a valuable complement to experimental data. However, the accuracy of MD simulations depends on the quality of force field. In particular, the evolutionary conservation and coevolution of IDPs introduce that current force fields could not precisely reproduce the conformation of IDPs. In order to improve the performance of force field, deep learning and reweighting methods were used to automatically generate personal force field parameters for intrinsically disordered and ordered proteins. At first, the deep learning method predicted more accuracy φ/ψ dihedral of residue than the previous method. Then, reweighting optimized the personal force field parameters for each residue. Finally, typical representative systems such as IDPs, structure protein, and fast-folding protein were used to evaluate this force field. The results indicate that two personal force field parameters (named PPFF1 and PPFF1_af2) could better reproduce the experimental observables than ff03CMAP force field. In summary, this strategy will provide feasibility for the development of precise personal force fields.