Search

We are now witnessing a dramatic growth of heterogeneous data, consisting of a complex set of cross-media content, such as texts, images, videos, audio, graphics, spatio-temporal data, and multivariate time series. The inception of modern techniques from computer science have offered very robust and hi-tech solutions for data and information analysis, collection, storage, and organization, as well as product and service delivery to customers. Recently, technological advancements, particularly in the form of big data, have resulted in the storage of enormous amounts of potentially valuable data in a wide variety of formats. This situation is creating new challenges for the development of effective algorithms and frameworks to meet the strong requirements of big data representation and analysis, knowledge understanding, and discovery.

In recent years, with the rapid development of 3D technology, view-based methods have shown excellent performance in both 3D model classification and retrieval tasks. In view-based methods, how to aggregate multi-view features is a key issue. There are two commonly used solutions in the existing methods: 1) Use pooling strategy to merge multi-view features, but it ignores the context information contained in the continuous view sequence. 2) Leverage grouping strategy or long short term memory networks (LSTM) to select representative views of the 3D model, however, it easily neglects the semantic information of individual views. In this paper, we propose a novel Semantic and Context information Fusion Network (SCFN) to compensate for these drawbacks. First, we render views from multiple perspectives of the 3D model and extract the raw feature of the individual view by 2D convolutional neural networks (CNN). Then we design the channel attention mechanism (CAM) to exploit the view-wise semantic information. By modeling the correlation among view feature channels, we can assign higher weights to useful feature attributes, while suppressing the useless. Next, we propose a context information fusion module (CFM) to fuse multiple view features to obtain a compact 3D representation. Extensive experiments are conducted on three popular datasets, i.e., ModelNet10, ModelNet40, and ShapeNetCore55, which can demonstrate the superiority of the proposed method comparing to the state-of-the-arts on both 3D classification and retrieval tasks.

The application of digital products in smart city results in ever-increasing 3D model data and how to obtain the relevant 3D model becomes a crucial issue. In this paper, we propose the Multi-View Tree Structure (MVTS) learning for 3D model retrieval and recognition. MVTS contains three key consecutive modules. Firstly, the visual feature learning module extracts the visual features of multiple views. Then, we design a score matrix to estimate the value of contextual information between view pairs. Based on the score matrix, a maximum spanning tree is constructed to further explore the contextual information within multiple views. Then, we utilize the bidirectional Tree-LSTM to encode the contextual information among views and the spatial information of tree structure and optimize the tree parameters. After that, the tree attention strategy is adopted to explore the importance of each view. Comparing to existing methods, our proposed method explores the spatial information of 3D model without the requirement of specific camera settings, which is more suitable for real applications. Moreover, our method jointly realizes the feature learning, view-wise contextual information and tree spatial information encoding and view importance estimating, which enhances the discrimination of the 3D model representation. Extensive experimental results on Modelnet40 and ShapeNetCore55 demonstrate the superiority of our method.

The rapid development of 3D technique has led to the dramatic increase in 3D data. The scalable and effective 3D object retrieval and classification algorithms become mandatory for large-scale 3D object management. One critical problem of view-based 3D object retrieval and classification is how to exploit the relevance and discrimination among multiple views. In this paper, we propose a multi-view hierarchical fusion network (MVHFN) for these two tasks. This method mainly contains two key modules. First, the module of visual feature learning applies the 2D CNNs to extract the visual feature of multiple views rendered around the specific 3D object. Then, the multi-view hierarchical fusion module we proposed is employed to fuse the multiple view features into a compact descriptor. This module can not only fully exploit the relevance among multiple views by intra-cluster multi-view fusion mechanism, but also discover the content discrimination by inter-cluster multi-view fusion mechanism. Experimental results on two public datasets, i.e., ModelNet40 and ShapeNetCore55, show that our proposed MVHFN outperforms the current state-of-the-art methods in both the 3D object retrieval and classification tasks.

3D model retrieval is becoming a hot research topic due to its wide applications such as computer-aided design, digital entertainment, and virtual reality. For this challenging task, feature learning and similarity measure are two critical problems. However, existing approaches usually learn discriminative visual features and develop a complex graph matching strategy to measure the similarity independently. In this paper, we propose an unsupervised method which can embed similarity measure into the feature space. The proposed method utilizes both similarity and dissimilarity information to better leverage the unsupervised problem and estimates the labels which are further used for metric learning. With the learned metric, we project the original features to more discriminative feature space and efficiently measure the similarity among models under the new feature space. We conduct extensive evaluations of three popular and challenging datasets. The experimental results demonstrate the superiority and effectiveness of the proposed method, competing against the state of the arts.

3D CPS (Cyber Physical System) data has been widely generated and utilized for multiple applications, e.g. autonomous driving, unmanned aerial vehicle and so on. For large-scale 3D CPS data analysis, 3D object retrieval plays a significant role for urban perception. In this paper, we propose an end-to-end domain adaptation framework for cross-domain 3D objects retrieval (C3DOR-Net), which learns a joint embedding space for 3D objects from different domains in an end-to-end manner. Specifically, we focus on the unsupervised case when 3D objects in the target domain are unlabeled. To better encode a 3D object, the proposed method learns multi-view visual features in a data-driven manner for 3D object representation. Then, the domain adaptation strategy is implemented to benefit both domain alignment and final classification. Specifically, an center-based discriminative feature learning method enables the domain invariant features with better intra-class compactness and inter-class separability. C3DOR-Net can achieve remarkable retrieval performances by maximizing the inter-class divergence and minimizing the intra-class divergence. We evaluate our method on two cross-domain protocols: 1) CAD-to-CAD object retrieval on two popular 3D datasets (NTU and PSB) in three designed cross-domain scenarios; 2) SHREC’19 monocular image based 3D object retrieval. Experimental results demonstrate that our method can significantly boost the cross-domain retrieval performances.

Attention mechanism has been extensively used in video captioning tasks, which enables further development of deeper visual understanding. However, most existing video captioning methods apply the attention mechanism on the frame level, which only model the temporal structure and generated words, but ignore the region-level spatial information that provides accurate visual features corresponding to the semantic content. In this paper, we propose a fine-grained spatial-temporal attention model (FSTA), and the spatial information of objects appearing in the video will be our main concern. In the proposed FSTA, we achieve the spatial-hard attention at a fine-grained region level of objects through the mask pooling module and compute the temporal soft attention by using a two-layer LSTM network with attention mechanism to generate sentences. We test the proposed model on two benchmark datasets, namely, MSVD and MSR-VTT. The results indicate that our proposed FSTA model can achieve competitive performance against the state of the arts on both datasets.

We present an approach to jointly detect mitotic events spatially and temporally in time-lapse phase contrast microscopy images. In particular, we combine a convolutional neural network (CNN) and a long short-term memory (LSTM) network to detect mitotic events in patch sequences. The CNN-LSTM network can be trained end-to-end to simultaneously learn convolutional features within each frame and temporal dynamics between frames, without hand-crafted visual or temporal feature design. Owing to the LSTM layer, this approach is able to detect mitotic events in patch sequences of variable length, as well as making use of longer context information among frames in the sequences. To the best of our knowledge, this is the first work to detect mitosis using deep learning in both spatial and temporal domains. Experiments have shown that the CNN-LSTM network can be trained efficiently, and we evaluate this design by applying the network to original raw microscopy image sequences to locate mitotic events both spatially and temporally. The data with which we validate the proposed method include C3H10 mesenchymal and C2C12 myoblastic stem cell populations. Our approach achieved the F score of 98.72% on the C2C12 data set, and the F score of 96.5% on the C3H10 data set. The results on both data sets outperform the traditional graph modelbased approaches by a large margin, both in terms of detection accuracy and frame localization accuracy. Furthermore, we have developed a framework to aid humans in annotating mitosis with high efficiency and accuracy in raw phase contrast microscopy images based on the joint detection results using the proposed method. Under this framework, expert level annotations can be obtained in raw phase contrast microscopy image sequences, and the annotations have shown to further improve the training performance of the CNN-LSTM network.

The successful launch of the first spaceborne CO2 IPDA LIDAR onboard Daqi-1 (DQ-1) in April 2022 marks a milestone in advancing scientific research. However, a notable discrepancy in the physical definitions of ${{\mathrm{X}}_{\mathrm{C}{{\mathrm{O}}_2}}}$ products between the IPDA LIDAR and TCCON presents a challenge for directly using TCCON for verifying and evaluating the performance of DQ-1’s ${{\mathrm{X}}_{\mathrm{C}{{\mathrm{O}}_2}}}$ products. To address this, we propose a method based on statistical hypothesis testing to globally validate the ${{\mathrm{X}}_{\mathrm{C}{{\mathrm{O}}_2}}}$ products derived from the spaceborne LIDAR. Our validation method does not compare DQ-1’s observations with the TCCON observations. We only utilize the useful information from TCCON to simulate the probability distribution of the true ${{\mathrm{X}}_{\mathrm{C}{{\mathrm{O}}_2}}}$ value in the LIDAR definition, and then hypothesis testing is adopted for deriving the systematic error. Our method improves the accuracy of computing the systematic error of DQ-1 by over 50% compared to the traditional approach. Up to now, we have produced the first four months (June 2022–September 2022) of DQ-1’s ${{\mathrm{X}}_{\mathrm{C}{{\mathrm{O}}_2}}}$ products. In this context, we present preliminary validation results based on the four months of data. Based on our method, we have validated that the accuracy of DQ-1’s ${{\mathrm{X}}_{\mathrm{C}{{\mathrm{O}}_2}}}$ products is 0.1 ± 1 ppm. This approach not only sets the stage for future official global validations of DQ-1’s ${{\mathrm{X}}_{\mathrm{C}{{\mathrm{O}}_2}}}$ products but also holds promise for application in upcoming similar missions such as MERLIN and DQ-2.

Intrusion detection systems (IDS) identify network intrusions by detecting abnormal traffic data, thereby ensuring network security. However, intrusion detection data can vary with changes in the network and attack environment, resulting in poor performance and portability of intrusion detection algorithms. Therefore, an intrusion detection method based on PSO-GA hyperparameter optimized ResNet-BiGRU is proposed. The two-layer bidirectional gated recurrent unit (BiGRU) is connected to the fully connected layer of the residual neural network (ResNet). Firstly, ResNet is used to extract parallel local features, and BiGRU is used to extract long-distance-dependent features from the parallel local features, and the attention mechanism is added after the BIGRU to utilize correlation between the features to assign weights to the extracted features, so as to more comprehensively capture the important features of network intrusion and improve the detection performance. At the same time, the parameters of the basic particle swarm optimization (PSO) are dynamically optimized and combined with the genetic algorithm (GA) to perform a mutation operation when the iterative process falls into a local optimal solution, adding a random perturbation to the current velocity and position of the particles, so that the particles are able to explore new regions in the space in order to jump out of the local optimal solution, and ultimately achieve automatic optimization of the hyperparameters of the ResNet-BiGRU model to achieve a model with better generalization performance. Finally, the proposed method is validated by using the variant NSL-KDD dataset, which achieves an accuracy of 98.46% and average precision, average recall and average False Alarm Rate(FAR) of 91.84%, 95.99% and 0.31%, and achieved high accuracy on three datasets KDD99, UNSW-NB15, and CIC-IDS 2017. The method is proved to have a strong intrusion detection capability by comparison experiments with other algorithms.

In the field of autonomous driving, monocular 3D object detection is focused on the task of representing 3D scenes using a single camera image and conducting 3D object detection. While Bird’s-Eye View (BEV) method effectively decreases the computational burden associated with 3D scene representation, its limitation in considering height information can lead to a less accurate depiction of complex 3D structures. This study introduces an innovative monocular 3D object detection framework called MonoMPV. This framework represents a complete 3D scene by mapping spatial objects onto Multi-Projection Views (MPV) without the need for voxelization, thus simplifying the process. Notably, MPV systems consist of Feature Cross-Attention (FCA) and Projection Cross-Attention (PCA) components. FCA aims to enhance image features to MPV level, while PCA enables direct information interaction among the views within MPV. Furthermore, Triplet Loss for Top Feature (TLTF) was employed in conjunction with FCA and PCA to distinguish effectively between top-plane and background features. By engaging in this practice, it is possible to develop more complex 3D structural models and establish precise optimization objectives through TLTF. Consequently, this approach enhances the effective utilization of data by the model. Experimental results on the nuScenes dataset illustrate that this approach surpasses existing monocular 3D object detection methods. To implement algorithms on on-board edge computing devices, the monocular 3D object detection task has been executed on the edge device Jetson Orin NX, ensuring high precision.

The advent of the sixth generation of mobile communications (6G) ushers in an era of heightened demand for advanced network intelligence to tackle the challenges of an expanding network landscape and increasing service demands. Deep Learning (DL), as a crucial technique for instilling intelligence into 6G, has demonstrated powerful and promising development. This paper provides a comprehensive overview of the pivotal role of DL in 6G, exploring the myriad opportunities and challenges that arise. Firstly, we present a detailed vision for DL in 6G, emphasizing areas such as adaptive resource allocation, intelligent network management, robust signal processing, ubiquitous edge intelligence, and endogenous security. Secondly, this paper reviews how DL models leverage their unique learning capabilities to solve complex service demands in 6G. The models discussed include Convolutional Neural Networks (CNN), Generative Adversarial Networks (GAN), Graph Neural Networks (GNN), Deep Reinforcement Learning (DRL), Transformer, Federated Learning (FL), and Meta Learning. Additionally, we examine the specific challenges each DL model faces within the 6G context. Moreover, we delve into the rapidly evolving field of Artificial Intelligence Generated Content (AIGC), examining its development and impact within the 6G framework. Finally, this paper culminates in a detailed discussion of ten critical open problems in integrating DL with 6G, setting the stage for future research and development in this field.

This paper presents a clustering-based data collection protocol for wireless sensor networks. Constructive interference-based flooding presents a novel way to achieve global synchronization, yet its communication mechanism requires that each message’s transmission involves a flooding process across the whole network, which suffers from scalability problems, especially for a densely deployed network. Additionally, an increase in the number of transmitting nodes compromises concurrent transmission reliability. This work aims to tackle these problems from a clustering perspective. Firstly, we improved the time synchronization method proposed by constructive inference-based flooding to make it applicable to our protocol. Then, we designed a heuristic algorithm from the balance and connectivity viewpoints to carry out network clustering simultaneously during the data collection process. The prototype was implemented on Tmote Sky sensor nodes. Extensive simulations using the Contiki operating system and experiments on a deployed testbed have verified that the protocol can achieve almost 100% data collection with lower energy consumption.

According to the latest report by the WHO, cardiovascular disease claims approximately 17.9 million lives annually, making it one of the leading causes of mortality. Hence, early screening and detection of cardiovascular diseases are important for their prevention. Heart sound signals contain a wealth of information on cardiac function and health status. Researchers have recently utilized deep learning methods to detect abnormal features in heart sound signals, thereby facilitating disease diagnosis. Currently, existing heart sound datasets suffer from imbalanced data proportions, complex feature types, and low discriminative power between systolic and diastolic murmurs, resulting in the suboptimal performance of deep learning algorithms in detection. Therefore, we propose a heart sound abnormality detection algorithm based on the Swin Transformer architecture. Firstly, we enhance the ability to extract local texture features of heart sound signals by introducing a convolutional embedding module into the positional encoding layer of the backbone network. Second, we augmented the model’s capability to extract the frequency-domain features of heart-sound signals by incorporating a discrete convolutional mapping structure. This structure utilizes discrete cosine transformation in conjunction with convolutional projection to acquire feature matrices, thereby improving classification accuracy. Finally, we employed a Focal Loss function to prioritize abnormal heart-sound samples, enhancing the generalization ability of the model and evaluating the proposed algorithm using the PhysionNet/CinC 2016 public dataset. The results demonstrated an Accuracy of 93.4%, a Specificity of 90.4% and a Sensitivity of 95.7%.

In causal learning, discovering the causal graph of the underlying generative mechanism from observed data is crucial. However, real-world data for causal discovery is scarce and expensive, leading researchers to rely on synthetic datasets, which may not accurately reflect real-world performance. To address this, we propose a novel method for evaluating causal discovery algorithms without needing real causal graphs. Specifically, our method employs deep learning evaluation strategies and ensemble learning techniques to robustly assess the performance of causal discovery methods. To elaborate, our approach emulates deep learning validation strategies by dividing the data into training and testing sets. We perform causal discovery on the training set and subsequently use the testing set to conduct Markov blanket tests on the node set and causal direction determination on the edge set. Moreover, we employ multiple ensemble strategies to ensure a comprehensive evaluation of the algorithms. Furthermore, experiments on both synthetic and real datasets demonstrate our method’s effectiveness in accurately and comprehensively validating causal discovery algorithms. Our results show that our proposed method can reflect the performance of causal discovery methods in practice with reasonable error.

The radio imaging method (RIM) is a non-destructive detection technique widely used in geophysical surveys and monitoring. However, it currently faces challenges in accurately detecting geological anomalies. To enhance detection accuracy, we propose a novel approach that integrates forward modeling and the deep learning UNet semantic segmentation architecture, termed the FM-UNet method. We first construct a forward model that reflects the geological characteristics of the target exploration area and validate it using field RIM survey data. Next, we use the RIM data generated from the forward model for inversion and construct the training dataset. Finally, we apply the deep learning UNet architecture to accurately predict the locations of geological anomalies within the exploration region. Our results demonstrate that the FM-UNet method effectively reduces the false geological anomaly (FGA) areas and significantly improves detection accuracy compared to traditional methods. Additionally, we compared the performance of four UNet architectures, with the Transformer UNet achieving the best results: the average ratio of FGA decreased by 37.37%, and the mean intersection over union (IoU) and average F1 score increased by 24.26% and 23.78%, respectively, confirming the feasibility of our approach.

Automatic seizure detection plays a crucial role in epilepsy diagnosis and treatment. Traditional machine learning based automatic seizure detection requires additional feature engineering and finding the optimal hand-crafted features is a challenging issue. Therefore, a novel end-to-end deep learning model that combines attention mechanism, temporal convolutional network(TCN), and bidirectional gated recurrent unit (BiGRU) is proposed for seizure detection in this work. Our model only requires filtering of the raw electroencephalogram(EEG) signals to remove artifacts, without the need for time-consuming feature extraction. Post-processing procedures including moving-average filtering, thresholding, and the collar technique are then applied to enhance the model’s performance. Experiments were conducted on the CHB-MIT dataset and the SH-SDU dataset. In patient-specific experiments, our model achieved average accuracies of 98.77% on the CHB-MIT dataset and 93.61% on the SH-SDU dataset. In cross-patient experiments, average accuracies of 93.78% and 91.37% were obtained, respectively. The total time required for the model to process 1-hour EEG signals is only 5.33s. These outstanding results indicate that our model achieves high accuracy and real-time performance in seizure detection tasks and could provide reference for clinical seizure diagnosis.

Coastal change detection holds significant importance in the management of marine resources, coastal city change analysis, coastal land planning, and utilization. Deep learning-based remote sensing semantic change detection has evolved into a crucial method for identifying alterations in coastal areas. However, commonly used land cover type annotations lack specific multiclass semantic change detection type annotations unique to coastal areas. Additionally, existing datasets for coastal change detection lack rich spectral details. Therefore, this study has created a finely annotated coastal high spatial resolution multiclass semantic change detection dataset, namely CHRM-SCD, which includes 5 land cover types and 20 semantic change types. This is the first high-resolution semantic change benchmark dataset for coastal areas based on Gaofen-2 imagery. Based on this dataset, a multiclass semantic change detection network based on multiband spatial-spectral attention mechanism has been proposed in this study to achieve multiclass semantic change detection in coastal areas. It achieves 89.20% overall accuracy, 81.48% mean intersection over union, and 50.26% separated kappa coefficient, showing improvements of 7.28%, 11.58%, and 21.39% over the BiSRNet method, respectively. The stability of this research method is also demonstrated on the semantic change detection dataset. The dataset developed in this study is applicable for tasks related to detecting changes in coastal areas. The proposed method demonstrates practical effectiveness in the field of multispectral high-resolution remote sensing for coastal change detection.

In recent years, synthetic aperture radar automatic target recognition (SAR ATR) has been widely researched for its ability to achieve high-performance target classification through supervised training, facilitating battlefield reconnaissance, and intelligence generation. When dealing with unknown class data for ATR model training, significant time and effort are typically required for manual interpretation and labeling. However, when unknown class data shares the same domain as the known labeled data in the library, leveraging their shared deep semantic knowledge can enable automatic classification and labeling of the unknown class data. In this article, we investigate the novel category discovery (NCD) problem in SAR images, using labeled data to guide the clustering process of new class data. Specifically, we utilize the Unified Objective function training framework to address the training imbalance between labeled and unlabeled data in the NCD process, incorporating various improvements on this foundation. Through a binary segmentation-based strategy, we effectively mitigate the interference of “noise pairs” with significant semantic differences on the model's self-supervised pretraining. In addition, we introduce multicrop consistency loss and equal distance loss to impose constraints on training by leveraging intraclass and interclass relationships in the latent space, thereby obtaining representations with higher interclass separability. Our method achieves state-of-the-art clustering performance in multiple scenarios. Extensive experimental results on the MSTAR benchmark dataset demonstrate the effectiveness of the proposed methods.

Precise control of strength is of significant importance in upper limb functional rehabilitation. Understanding the neuro-muscular response in strength regulation can help optimize the rehabilitation prescriptions and facilitate the relative training process for recovery control. This study aimed to investigate the inherent characteristics of neural-muscular activity during dynamic hand strength adjustment. Four dynamic grip force tracking modes were set by manipulating different magnitude and speed of force variations, and thirteen healthy young individuals took participation in the experiment. Electroencephalography were recorded in the contralateral sensorimotor cortex area, as well as the electromyography from the first dorsal interosseous muscle were collected synchronously. The metrics of the Event-related desynchronization, the electromyography stability index, and the force variation, were used to represent the corresponding cortical neural responses, muscle contraction activities, and the level of strength regulation, respectively; and further neuro-muscular coupling between the sensorimotor cortex and the first dorsal interosseous muscle was investigated by transfer entropy analysis. The results indicated a strong relationship that the increase of force regulation demand would result in a force variation increase as well as a stability reduction in muscle motor unit output. Meanwhile, the intensity of neural response increased in both the $\alpha $ and $\beta $ frequency bands. As the force regulation demand increased, the strength of bidirectional transfer entropy showed a clear shift from $\beta $ to the $\gamma $ frequency band, which facilitate rapid integration of dynamic strength compensation to adapt to motor task changes.

To investigate the biomechanical mechanisms underlying the pathogenesis and explore the effects of kinesiology taping (KT) on neuromuscular control in HV patients. The study population consisted of 16 young controls (YC group) and 15 patients with hallux valgus (HV group). All subjects underwent a natural velocity gait assessment. Additionally, 11 patients from the HV group received KT intervention over a period of one month, consisting of 15 sessions administered every other day. After the one-month intervention, these patients underwent a gait assessment and were included in the HV-KT group. The electromyography (EMG) and joint motion were evaluated using non-negative matrix factorization (NNMF) to compare the difference in muscle and kinematic synergy among the three groups. The center of plantar pressure (COP) and ground reaction force (GRF) were measured by the force platform. The number of synergies did not differ within the three groups, but the structure of muscle synergies and kinematic synergies differed in the HV group. The KT intervention (HV-KT group) altered the structure of synergies. The correlation between kinematic synergies and muscular synergies was lower in the HV group than in the YC group, whereas the correlation between the two increased after the KT intervention in the HV group. During gait, the HV group tended to activate more muscles around foot joints to maintain body stability. The visual analogue scale (VAS) scores, hallux valgus angle (HVA), and COP were significantly decreased after the intervention ( ${P}\lt 0.05$ ). HV patients exhibited altered kinematic and muscular synergies structures as well as muscle activation. Also, it weakened the balance and athletic ability of HV patients. KT intervention improved neuromuscular control to provide a better gait performance.

Breast cancer is the second deadliest cancer (after lung cancer) globally among women, with high incidence and mortality rates. Its early diagnosis is pivotal for improving the cure rate. With the continuous development and maturity of deep learning technologies, traditional classification models have been widely applied for automated classification of pathological images. However, several challenges still persist. For instance, traditional classification models typically perform well in processing images with clear distinctions between target objects and backgrounds, but struggle to accurately classify pathological images due to the lack of clear distinctions between tumor lesion areas and background areas. In the light of this, we propose a two-stage breast tumor pathological classification model based on weakly supervised target localization, named ST-Double-Net. In the proposed model, precise lesion localization and classification are achieved in two stages. In the first stage, a set of global feature maps is obtained by utilizing the Swin Transformer. These feature maps are then input into a newly designed heatmap cropping (HMC) module, which forces the model to focus on discriminative features of lesion areas through heatmap-guided cropping, without requiring bounding boxes or relevant annotation information. This gradual refinement of target localization facilitates the extraction of useful global features, from coarse to fine. The images with discriminative features generated in the first stage serve as inputs for the second stage, where another Swin Transformer extracts local features from the magnified lesion region images. Finally, the global and local features extracted in the first and second stage, respectively, are fused to emphasize subtle differences in the images, thereby enhancing the model’s classification ability. The proposed ST-Double-Net model is evaluated on the BreaKHis and BACH public datasets, demonstrating superior performance compared to state-of-the-art models.

As the connectivity between Electronic Control Units (ECUs) and the external environment intensifies, the security and safeguarding of In-Vehicle Networks (IVNs) have become pressing issues. The Controller Area Network (CAN) bus, the most commonly employed vehicular network protocol, is particularly vulnerable due to its inherent lack of security measures, making it susceptible to various attacks. In this paper, we introduce a novel intrusion detection model for CAN networks, named IVN-ViT. The proposed IVN-ViT model utilizes an enhanced Vision Transformer architecture (Edge-ViT), incorporating self-supervised learning with Cutout data augmentation, Particle Swarm Optimization (PSO) for feature selection, and a fine-tuning strategy that merges Parameter Instance Discrimination (PID) with supervised loss. This not only enhances the model’s convergence speed and predictive accuracy but also meets the lightweight requirements of CAN networks. Experimental results on the “HCRL-car hacking” dataset and the “X-CANIDS” dataset demonstrate that our model achieves over 99% accuracy across all types of attacks. In a simulated vehicular environment, the detection latency for each message averages approximately 1.04ms, fully meeting the first-time requirements of CAN networks. The experiments validate that our proposed method significantly improves model accuracy while ensuring efficient operation within the limited computational resources available in vehicle systems.

As a country with a rich cultural and historical heritage, there are many grottoes in China. Unfortunately, many grottoes suffer from fracture seepage due to their age, causing weathering damage that requires prompt attention, detection, and repair to preserve them for future generations. In order to preserve the historical and cultural heritage of Beishan, Chongqing, China, this study illustrates how we assess the fracture seepage system of sandstone grottoes within the Penglaizhen Formation using multiple geophysical survey techniques. They are core data, conventional logging, borehole televiewer, and ground penetrating radar (GPR). Combining these methods, borehole televiewer and core data can improve the accuracy of conventional logging for stratigraphic delineation, as well as aid GPR in finding deep stratigraphic and seepage fractures. This study provides an effective routine to find sandstone grotto fractures and achieves positive results.

The fault characteristics of large-capacity wind farms affect the sensitivity of transmission-line protection, resulting in protection devices refusing to operate or mal-operate. A phase-to-ground fault-locating method with single-end transient information for wind farm transmission lines was proposed in this study. The proposed method employs a time-domain transient quantity integration equation matrix with the transmission line equivalent resistance, inductance, and fault path resistance to solve the fault location. The effectiveness of the scheme was evaluated using Real-Time Lab (RTLAB). The impact of factors such as the fault resistance, location, and distributed capacitance is considered. Under different fault locations, the distance measurement error of the proposed algorithm is within 3%, and there is no rejection or mal-operation. The results demonstrate that the proposed algorithm is characterized by high speed and reliability.

Multitarget tracking technology is the core topic in the field of intelligent driving. Multi-target complex manoeuvring, measurement outliers and unknown environmental prior parameters strongly affect the tracking accuracy of the target state. To address the accurate tracking of multitarget under the above complex working conditions, we propose a new multitarget tracking algorithm, named the Multiconstrained Generalized Probabilistic Data Association Filtering (MCGPDAF) algorithm. In this algorithm, we use the target position and heading information to construct constraint parameters to calculate the association probability between each effective measurement combination and the target track. This algorithm can effectively suppress the measurement association anomalies and aprior information errors, as well as enable the robust association of single-sensor multitarget measurements and accurate tracking of target states under complex working conditions. On this basis, a multitarget tracking method based on composite perception fusion is further constructed, and the correlation sequential track association algorithm and covariance cross fusion algorithm are used to enable the track association and the estimation and fusion of target states among multiple sensors, which further enhances the tracking accuracy of the multitarget state. The simulation and real vehicle experiment results reveal that, compared to current advanced algorithms, the RMSE and MAPE of the MCGPDAF algorithm for multitarget tracking are enhanced by an average of 23.97% and 24.35%, respectively. Additionally, the MOTA and MOTP of the MCGPDAF algorithm improve by an average of 14.68% and 15.71%. Moreover, compared to single-sensor multitarget tracking, the RMSE and MAPE of composite perception fusion results based on the MCGPDAF algorithm are further enhanced by 26.43% and 27.15% on average, which reflects the practicality of the tracking method showcased in this research.

With the popularization of the Internet, it is very important to effectively identify abnormal behaviors in network traffic. This study focuses on the construction of an internet traffic monitoring model, aiming to improve the accurate recognition rate of abnormal behavior and reduce information loss during small block segmentation. To this end, a internet traffic monitoring algorithm based on the improved Transformer is optimized. This model adopts a block segmentation algorithm that preserves important information during the segmentation process, thereby enhancing the segmentation quality and accuracy of the model. By effectively interacting with multiple receptive field information, the model reduces information loss and improves accuracy and efficiency. After experimental verification, the model performed well on CICIDS sample data, with an F1 value of 93% for normal internet traffic. The F1 value of internet attack traffic was 91%. Compared with the original Transformer model, it increased by 5% and 2.4%, respectively. On the NSLKDD sample, the improved algorithm proposed in the study had an area under the curve value of 0.90, which outperformed other models. This proves that it has significant advantages in the dual classification task of internet traffic anomaly monitoring. This study provides an effective deep learning algorithm for internet traffic anomaly monitoring, which is expected to provide strong support for network security assurance in practical application scenarios.

The combination of multispectral image (MSI) and synthetic aperture radar (SAR) data has made certain progress in coastal wetland classification. How to realize the interactive fusion between the two data and make full use of their fusion characteristics becomes challenging. However, the existing joint classification methods neglect interaction information between features and underutilize fusion features. Therefore, this article proposes a dual-branch feature interaction network (DFI-Net) that joins MSI and SAR data for coastal wetland classification. The dual-branch independent structure of 3DCNN processing MSI and 2DCNN processing SAR is designed, which can effectively capture spectral–spatial features and polarization features. In addition, we develop two novel modules. The feature interaction fusion block is designed to enhance the complementarity between the features of the two kinds of data. This block employs a cross-agent attention mechanism to realize effective interaction between MSI and SAR features and adaptive fusion of contextual information from the two branches. Finally, a plug-and-play module channel–spatial transformer encode (CSTE) is proposed to improve the utilization rate of interactive fusion data. The CSTE utilizes two parallel transformers to deeply mine information in interactive fusion data and explore channel–spatial features across all dimensions to the maximum extent possible. The classification experiment is conducted on the Yellow River Delta coastal wetland dataset. The experimental results show that the overall accuracy of DFI-Net reaches 97.03%, which outperforms the performance of other competitive approaches. The effectiveness of DFI-Net provides a reference method for combining MSI and SAR for coastal wetland classification.

A fast recovery from disruptions is of vital importance for the reliability and sustainability of urban zone transit systems. It’s a complex task to coordinate multiple transit departments to mitigate disruption. There are many ways to response but it’s not always obvious how to combine them in an optimized manner. This study presents a new attempt to tackle this problem in a comprehensive and hierarchical way. At phase (i), a network-scale strategy selection optimization model is formulated as a joint routing and resource allocation (nJRRA) problem. This model produces solutions for efficient allocation of network resources to facilitate inter-department coordination. By constraining the problem further into an $\epsilon $ -constrained nJRRA problem, classic solution algorithms can be applied to solve the quadratically constrained quadratic program (QCQP). On top of this “basic model”, we propose adding a decision to delay the resource allocation decisions up to a maximum initiation time when the incident duration is stochastic. To test the models, a quasi-dynamic evaluation program with a given incident duration distribution is constructed using discretized time steps and discrete distributions. Five different demand patterns and four different disruption duration distributions (20 combinations) are tested on a small transit network. The results show that the two models outperform benchmark strategies such as using only line level adjustment or only bus bridging. They also highlight conditions when delaying the decision is preferred.

The fitting efficiency of lines or arcs fitting methods for various structural designs cannot meet the design requirements, and the fitting accuracy also needs to be improved. Therefore, this paper proposes a novel algorithm that uses a smaller number of elliptical arc segments to accurately fit the curve, while point-to-point continuity and a tangent relationship between all connected line segments of the curve can be maintained (G1 continuity). This algorithm uses elliptical arcs for curve fitting, and the advantage of using this conventional quadratic curve is that it does not require a higher-order equation to achieve that all connected line segments are tangent to each other, and its feature data is easily obtainable. In this paper, the novel elliptic arc fitting algorithm is used for the sine function curve, butterfly curve, parabola function curve, spiral line function curve and NURBS curve. The results show that this proposed method could be used to fit the curves with a smaller number of elliptical arc segments while satisfying the accuracy requirements. This algorithm with a simple solution method requires a significant reduction in the number of fitting segments while achieving the same fitting accuracy as other algorithms. It provides new ideas for the field of geometric modeling, tool path generation for numerical control (NC) machining and robot path planning.

Predicting the price of soybean meal futures has always been a hot topic in the financial market. It is influenced by many factors and has complex nonlinear characteristics, which makes it difficult to predict accurately. This paper proposes a prediction model for soybean meal futures price based on CNN-ILSTM-SA, which consists of CNN (Convolutional Neural Network), ILSTM (Improved Long Short-Term Memory), and SA (Self-Attention mechanism). CNN can well perform feature extraction on the input time series data. The ILSTM model presented in this paper removes the output gate of LSTM (Long Short-Term Memory) and improves the forget gate and input gate, thus reducing ILSTM’s complexity, shortening training time, and improving learning ability. SA can get the correlation between different input terms and predicted values, and assign different weight coefficients according to the correlation level to improve prediction accuracy. To verify the CNN-ILSTM-SA’s effectiveness, we compare it with eighteen baseline models, and the experimental results show that it is optimal. The MAE (Mean Absolute Error) of CNN-ILSTM-SA is 39.0441, the smallest of all models. As the data of the soybean meal futures price fluctuates in the range of 3500-4500 in experiments, the proportion of the MAE value of 39.0441 is 0.87%-1.15%, which is at an extremely low level; $\mathrm {R}^{2}$ (R-square) is 0.97487, which is closest to 1; The training time is 157.64s.

Recently, the neural network (NN)-based in-loop filters are actively explored and being attempted to integrate in the versatile video coding (VVC). Transformer shows better global feature processing capability than convolutional neural network (CNN), which is complimentary to CNN that has advantage of local feature extraction. Since the in-loop filter operates in the coding tree unit (CTU) with fixed and relatively small size, it is very suitable for introducing Transformer into the in-loop filter in VVC. In this paper, we propose an NN-based in-loop filter in VVC using Resblock and Transformer, named RTNN, to suppress complicated compression artifacts in VVC. The proposed RTNN is an efficient filter, which combines Resblock and Transformer to extract deep features and capture local and non-local correlation between features. Moreover, a novel attention module is designed in RTNN to better refine features by introducing auxiliary information. Besides, a multi-stage training strategy is used to consider QP distance in training and maximize the learning ability of RTNN. The proposed RTNN filter is embedded into the VVC Test Model (VTM)-11.0_Neural Network-based Video Coding (NNVC)-3.0. Experimental results show that the proposed RTNN achieves average {8.49%, 22.98%, 24.07%} and {9.23%, 22.47%, 22.31%} BD-rate reductions over VTM-11.0_NNVC-3.0 for {Y, Cb, Cr} channels in AI and RA configurations, respectively, as well as yields significant performance with acceptable complexity.

Snow on the Tibetan Plateau (TP) is not only a freshwater resource for the major rivers in Asia but also plays a significant role in adjusting temperature by reflecting solar radiation. Fractional snow cover mapping with fine spatial and temporal resolution is of great significance for clarifying snow resources and accurately managing snow water resources. However, due to the complex TP topography, the existing fractional snow cover (FSC) retrieval methods are affected by a variety of disturbance factors, resulting in a decrease in accuracy. In this study, a deep learning-based FSC retrieval method, FSC U-shape net, is proposed to improve snow cover mapping accuracy. The input images of FengYun-4A advanced geosynchronous radiation imager and geographic data are extracted using the proposed spatial-channel feature extraction module to characterize the shallow features with texture information into high-dimensional feature images. Additionally, an attention mechanism is introduced to improve the feature differences between different FSC degrees. Finally, the correlation of the decoded features in the channel direction is mined using the channel refinement module to obtain the final FSC results. In this study, a backpropagation artificial neural network, random forest, ResNet_FSC, and UNet are trained, compared, and validated against the MOA10A1 FSC product. The results show that the proposed method effectively mitigates the problems of unclear texture of snow edges, poor robustness, and underestimation in some areas, which exist in other models. Additionally, the proposed method has higher accuracy, with an R2 and explained variance score reaching 0.7182 and 0.7332, respectively. Compared to the MOD10A1 snow product, the proposed method exhibits higher detection accuracy in mountainous areas with high snow cover and significantly improves the low snow cover detection rate.

Data center interconnections are cost-sensitive. We propose and demonstrate a cost-effective C-band intensity modulation direct detection (IM/DD) transmitter for extended range (ER) inter-datacenter interconnects utilizing only a 2-bit digital-to-analog converter (DAC). Tomlinson-Harashima precoding (THP) is used in transmitter DSP to compensate for dispersion-induced impairments. Due to the uniform distribution of signal levels after THP encoding, a high-resolution DAC is required to generate electrical signals. The hardware cost of the transmitter can be significantly reduced by compressing the physical number of bits (PNOB) of the DAC to 2 bits. We quantize the signal to 4 levels and utilize up-sampling and noise shaping to greatly reduce the in-band quantization noise. The in-band signal to noise ratio (SNR) can be increased to 23 dB. We experimentally demonstrate a cost-effective 40-Gbaud probabilistic shaping PAM-4 C-band IM/DD transmission over 50-km utilizing only a 2-bit DAC in the transmitter without any optical dispersion compensation. The performance of the proposed scheme is proximate to the common case of using an 8-bit DAC. It has a BER gap of about 2 × 10−3 only when the received optical power (ROP) is higher than −6 dBm.

Self-powered photodetectors which operate without external power sources hold immense promise in future photodetection systems. To achieve high-performance self-powered optoelectronic devices, efficient electron-hole pair separation is critical to generate high photocurrents. In this work, we successfully synthesized semiconducting graphene nanoribbons (GNRs) with a direct bandgap of 1.80 eV and employed them to construct a high-performance GNR/Al2O3/IGZO heterostructure photodetector. The built-in electric field in the heterojunctions enables this photodetector to exhibit remarkable performance, showing a responsivity of up to 68 mA/W, a detectivity of 8.34 × 1010 Jones, and rapid response times of 21/20 ms at zero bias. Furthermore, the photodetector features a wide spectral detection range of 405 to 1550 nm. These results highlight the promising potential of GNR/IGZO p-n heterojunction-based self-powered photodetectors in optoelectronic applications.

The compatibility of the advanced BSIM-CMG to the low frequency noise (LFN) simulation in amorphous IZO TFTs is evaluated over subthreshold and linear regions. Two kinds of devices with SiO2-SiNx and Al2O3 gate insulators are studied. In these devices, the 1/f noise is confirmed as the main component of LFN. Then the dominated origin of the 1/f noise is explained by the $\Delta \text{N}$ model in devices with SiO2-SiNx layers, and by the $\Delta \text{N}$ - $\Delta \mu $ model in devices with Al2O3 layers, respectively. Based on these models, the interficial traps density and the Hooge’s parameters are further calculated, and then applied to the extraction of noise parameters (NOIAeff, NOIB and NOIC) in BSIM-CMG. Compared to the measured data, the simulated results indicate that the noise can be well simulated by the improved BSIM-CMG both in the subthreshold and linear regions of IZO TFTs. It provides a comprehensive evaluation on the suitability of the BSIM-CMG for 1/f noise modelling in amorphous metal oxide TFTs.

Active learning is a strategy that involves the deliberate selection of training samples with significant discriminative features, to formulate a more efficient learning paradigm. While active learning has seen success in image classification, its application in object detection remains complex due to the multiple candidate object instances in an image and the interference of background regions on these instances. To tackle the challenge of sample selection in active learning for large object recognition datasets, this paper introduces a active learning method based on local feature saliency and posterior probability. Initially, the method selects active learning samples with discriminative saliency by clustering the multi-scale feature maps of the processed samples by self-attention weights. Subsequently, it calculates the posterior entropy based on the posterior probability of the current model’s candidate instances. This is combined with the entropy encoding of feature points corresponding to instances in the cluster to further select uncertain samples in the object set. On the COCO dataset, our method demonstrates superior overall performance in the learning curve compared to existing object detection baselines. Moreover, under the same training configuration, the final detection accuracy shows a notable improvement of at least 2.36%, thus confirming the effectiveness of our method in enhancing learning and detection performance.

Achieving automatic target recognition in synthetic aperture radar (SAR) imagery is a long-standing difficulty because of the limited training samples and its sensitivity to imaging condition. Active target recognition methods can offer an innovative perspective to improve recognition accuracy compared to their passive counterparts. Although prevailing in the optical imagery area, the active target recognition in SAR image processing remains underexplored. This article proposes an active SAR target recognition framework based on deep reinforcement learning for the first time, where we design a simple view-matching task and model it as a Markov decision process. The proximal policy optimization algorithm is used to help the agent learn how to alter the observing azimuth to seek more discriminative target images for the classifier. Furthermore, the single-view feature extractor is trained with the contrastive learning method to help distinguish the target images under different azimuths, allowing the agent to successfully learn the active data collection policy in the training environment and transfer it to the test environment. Lastly, the effectiveness and advancement of the proposed framework are verified on the SAMPLE dataset. When the training samples for the classifier are very scarce, it could bring around 10% more gain in target recognition rate compared to existing active target recognition frameworks.

The functional architecture undergoes alterations during the preclinical phase of Alzheimer’s disease. Consequently, the primary research focus has shifted towards identifying Alzheimer’s disease and its early stages by constructing a functional connectivity network based on resting-state fMRI data. Recent investigations show that as Alzheimer’s Disease (AD) progresses, modular tissue and connections in the core brain areas of AD patients diminish. Sparse learning methods are powerful tools for understanding Functional Brain Networks (FBNs) with Regions of Interest (ROIs) and a connectivity matrix measuring functional coherence between them. However, these tools often focus exclusively on functional connectivity measures, neglecting the brain network’s modularity. Modularity orchestrates dynamic activities within the FBN to execute intricate cognitive tasks. To provide a comprehensive delineation of the FBN, we propose a local similarity-constrained low-rank sparse representation (LSLRSR) method that encodes modularity information under a manifold-regularized network learning framework and further formulate it as a low-rank sparse graph learning problem, which can be solved by an efficient optimization algorithm. Specifically, for each modularity structure, the Schatten p-norm regularizer reduces the reconstruction error and provides a better approximation of the low-rank constraint. Furthermore, we adopt a manifold-regularized local similarity prior to infer the intricate relationship between subnetwork similarity and modularity, guiding the modeling of FBN. Additionally, the proximal average method approximates the joint solution’s proximal map, and the resulting nonconvex optimization problems are solved using the alternating direction multiplier method (ADMM). Compared to state-of-the-art methods for constructing FBNs, our algorithm generates a more modular FBN. This lays the groundwork for further research into alterations in brain network modularity resulting from diseases.

Synthetic aperture radar automatic target recognition (SAR ATR) has been widely studied in recent years. Most ATR models are designed based on the traditional closed-set assumption. This type of ATR model can only identify target categories existing in the training set, and it will result in missed detection or misclassification of unseen target categories encountered in battlefield reconnaissance, posing a potential threat. Therefore, it is of great significance to design a model that can simultaneously achieve known class classification and unknown class judgment. In addition, researchers usually use the obtained unknown class data for model relearning to enable it to recognize new categories. However, before this process, it is necessary to manually interpret and annotate the obtained unknown class data, which undoubtedly requires a large time cost and is difficult to meet the timeliness requirements. To solve these problems, we propose a framework that integrates the open-set recognition module and the novel class discovery module. By introducing the K-contrast loss, the open-set recognition module can accurately distinguish unknown class data, classify known class data, and then transfer the known class knowledge through deep clustering for clustering annotation of unknown class data. Extensive experimental results on the MSTAR benchmark dataset demonstrate the effectiveness of the proposed methods.

This paper proposes a physical charge-based analytical MOSFET threshold voltage model that explicitly incorporates interface-trapped charges which have been identified as playing a dominant role in defining threshold voltage trends in deep cryogenic temperatures. The model retains standard threshold voltage definition by various charges across the MOSFET capacitor while being analytical in its form, therefore, suitable for cryogenic CMOS VLSI design. Consequently, a model covering each and all above characteristics is proposed for the first time. Excellent fit between the model and measurement data from 180-nm bulk foundry devices is shown from room temperature to 4 K.

Most of the existing superpixel segmentation-based synthetic aperture radar (SAR) target detection algorithms cannot keep the independence of small targets under complex background, especially when the size of the targets varies greatly. Focusing on the problem, we propose a novel local fuzzy iterative superpixel segmentation method using edge information (LFI-EI) for SAR image target detection. First, we use an edge detector to extract the edge information in the SAR image. Second, a new superpixel initialization method combining the edge information is proposed and a dissimilarity measurement is defined. Third, instead of hard iterative clustering, we introduce the fuzzy theory to construct local fuzzy iterative clustering to obtain the superpixel segmentation result. Finally, a small regions integration step using edge information is implemented to enhance the superpixel segmentation result. By comparing the superpixel segmentation results of three competitors on simulated SAR images and real SAR images, it can be found that the proposed LFI-EI method can achieve satisfying superpixel segmentation performance, and the problem that small targets cannot maintain their independence when the size of targets varies greatly in the SAR image can be solved to some extent. Meanwhile, based on the LFI-EI superpixel segmentation method, the LFI-EI detector is proposed for target detection. The detector uses the contrast features between superpixels to transform the detection problem into a classification problem, and ultimately obtains the target detection result. The target detection experiment on the superpixel segmentation results proves that LFI-EI detector has good detection performance, and further verifies the superiority of the proposed LFI-EI superpixel segmentation method.

The El Niño-Southern Oscillation (ENSO) is a global significant signal which exerts substantial climatic and socioeconomic impacts worldwide. However, the long-term prediction of ENSO persists as a challenge because of its diversity, irregularity, and asymmetry. Here, we develop a spatiotemporal fusion transformer network (STFTN), which designed a parallel encoder structure to effectively extract the spatiotemporal information from sea surface temperature anomaly and Niño3.4 index simultaneously, thereby enhancing the precision of Niño3.4 index forecasts. STFTN leverages the attention mechanism within its parallel encoder structure to extract global characteristics and establish remote dependencies on targets. With this structure, STFTN displays better prediction accuracy in different lead months. Furthermore, the activation map used in STFTN visualizes the contribution of the predictors to the output which helps to comprehend the factors contributing to ENSO events. The results highlight the potential of our model of ENSO forecasts and comprehension.

In remote sensing image classification task, it is often apply a model trained on one dataset (source domain) to another dataset (target domain). However, due to the presence of domain shift between these domains where data are not independent and identically distributed, the performance of the model typically deteriorates. Domain adaptation aims to improve the generalization performance of the model in the target domain. In response to the challenges of intricate backgrounds, domain shift, and potentially unlabeled target domain in remote sensing images, this article proposes a network specifically designed for unsupervised domain adaptation (UDA) classification of remote sensing images, named PPLM-net. The network consists of a domain adversarial training (DAT) module, a partial patch local masking (PPLM) module and a teacher–student network module. The DAT module enables the network to extract domain-invariant features. The PPLM module compels the model to focus on the global information of target domain remote sensing images with intricate backgrounds, learning contextual content to improve model performance. The teacher network generates pseudolabels for complete unlabeled target domain images. The student network trained with PPLM target domain classification loss to generate robust and discriminative features. We construct a dataset dedicated to the UDA scene classification task of remote sensing images named RSDA. We collect images from four publicly available datasets spanning seven common categories, containing over 10 000 images. Compared with the current state-of-the-art UDA model, PPLM-net achieves the best results in 12 domain adaptation classification tasks on RSDA. The average accuracy reaches 99.115%.

As Advanced Driver Assistance Systems (ADAS) become increasingly intelligent, facial expression recognition (FER) has become a significant requirement for the purpose of monitoring a driver’s emotional state as well as fatigue level. An automobile system with FER is very useful in improving transportation safety by recognizing the driver’s state to provide timely alerts and potentially reduce the likelihood of accidents. While deep neural networks (DNN) based systems have achieved high accuracy in FER in recent years based on data collected under good laboratory environments, recognizing real-world facial expressions remains challenging due to variations in lighting and head pose especially prevalent in the driving scenario. In this paper, we propose an attention-based multi-modal and multi-view fusion FER model that can accurately recognize facial expressions regardless of lighting conditions or head poses, using image data of various modalities including RGB, Near-infrared (NIR), and Depth Maps from different viewpoints. The model is developed on a novel facial expression dataset we collected that includes multiple modalities captured from multiple viewpoints, with varying lighting conditions and head poses. Our multi-modal and multi-view fusion approach shows superior performance compared with models that use data from a single modality/view. The model achieves an accuracy of over 95% when recognizing drivers’ facial expressions in real-world scenarios, even in poor lighting conditions and different head poses.

Federated learning is a distributed learning approach which can protect clients’ privacy. The success of federated learning depends on the participation of high quality clients. However, many works neglect how to incentivize high quality clients to contribute to the model training. Furthermore, in existing works focusing on incentive mechanisms, the communication between the server and clients is unexplored. In this paper, we propose an auction-based incentive mechanism considering communication path finding to incentivize high quality clients’ participation. Specifically, we conduct the multi-criteria path finding to improve the communication between the server and clients. Then we take the communication parameters into the auction between the server and clients to select high quality clients for model training. When clients only have their own information, we design a distributed algorithm for clients to choose their optimal bids. Finally, numerical simulation results have been implemented to evaluate the efficiency of our proposed method.

In this study, an autonomous robotic polishing system leveraging sensor signal processing and control technology is developed. It utilizes a primary-secondary configuration with machine vision and force sensors for intelligent defect detection. The system processes force sensor data to obtain an accurate contact force assessment and performs precise polishing using proportional integral derivative and impedance-based control strategies. It supports remote teaching operator control and autonomous operation of intelligent polishing. Experimental results show that the maximum error between the measured and true values is 0.36 mm. In arc detection, the maximum measurement error between the measured and true values is 1.02mm, and the average error is 0.33mm. This level of accuracy is considered sufficient for the specific applications targeted in this study, such as precision manufacturing, where high accuracy is essential but variability in measurements can be adapted to within a certain range. In actual polishing experiments, the roughness values of the designed industrial robot on pipes and wood were $0.491~\mu $ m and $0.487~\mu $ m, respectively. Most of the polishing of defective locations is completed. Overall, the designed automatic polishing industrial robot is effective and can be applied to polishing operation in industrial production.

Current road accident survey methodologies often overlook the low-brightness and occlusion conditions, and this study specifically fuses UAV-borne and terrestrial LiDAR data, to facilitate rapid and high-precision accident scene monitoring for these scenarios. Firstly, UAV-borne LiDAR is utilized to acquire point cloud data from the perspective of wide-view. Meanwhile, both the side overlap rate and flight altitude of UAV mapping are considered, and the impacts of them on the accident scene modeling accuracy are explored in low-brightness scenario. Then, the terrestrial LiDAR scanning is adopted to model the accident scene under occlusion condition. Secondly, the Gaussian filtering and voxelization downsampling are conducted to improve the quality of point cloud data and the modeling speed. Thirdly, for enhancing the feature fusion, the Normal Distributions Transform (NDT) algorithm is adopted to register the UAV-borne and terrestrial LiDAR data. Finally, the DJI M300RTK UAV is selected, and two traffic accident scenes of low-brightness and occlusion conditions are conducted. A 50% side overlap rate and a 15m altitude of UAV mapping are recommended, to construct an efficient and accurate accident scene investigation. In addition, compared with the manual taping, the total station measurement and photography method, the monitoring accuracy of the proposed method is optimal with an RMSE of 0.0491. This shows that it is promising to adopt the proposed method to investigate the low-brightness and occlusion traffic accident scenes. Moreover, the limitations and future research directions of the proposed method are presented.

With the development of the metro system, an increasing number of new metro lines are being constructed. However, the lack of historical data on the new metro lines leads to the challenge of predicting the origin-destination distribution of the whole network. Based on the advantage of Automatic Fare Collection data, this study proposes a novel method to predict origin-destination distribution over metro networks with expansions. Firstly, the disaggregate model is built to predict the origin-destination distributions of the whole network with consideration of passengers’ destination choice behavior. Then, the aggregate model which can capture the historical passenger distribution patterns is developed to correct the origin-destination distribution in existing metro lines. Finally, this study combines the results of the disaggregate model and aggregate model to obtain the whole network origin-destination distribution. This study applies the method to the Guangzhou Metro network, and the results show the superiority of the proposed model. The results can help to improve the operation and management of urban rail transit.

With the development of the brain-computer interface, it is possible to assess the image quality based on an electroencephalogram (EEG) signal, which is essential to construct a quality assessment system that accords with the characteristics of the human visual system. However, as the complexity of image degradation levels rises, the analysis of brain features becomes progressively more challenging, leading the neural mechanisms still unclear. In this paper, an image quality assessment experiment with three levels of degradation to explore the neural mechanisms of subjects when confronted with images degraded at multiple levels. Subsequently, we proposed a dynamic model combined with Topological Data Analysis (TDA) and Grey Theory to extract the feature of brain response to different distortion-level images, which is called grey-topological data analysis (Grey-TDA) in this paper. The results indicate that the proposed method is effective in identifying brain responses under stimuli of varying image qualities, which contributes to the research of image quality assessment from the perspective of brain cognition.