Search

Representative large models and their related applications, such as Bidirectional encoder representations from transformers (BERT), Generative pretrained transformer (GPT), Segment anything model (SAM), ChatGPT, DALL-E, Wenxin, and Pangu, have made astounding strides and exerted considerable influence across various fields domestically and abroad. They constantly attract the attention and follow-up of diverse societal sectors, including enterprises, universities, and research institutions. Large model applications have been successfully applied in scenarios such as biology, medicine, law, and social governance. Designing, modifying, and constructing domain-specific large models are crucial for truly harnessing their application value. Therefore, this paper provides inspiration for the application of large models in materials science. First, it provides an overview of large models, introducing their basic concepts, developmental process, technical classification, and features. Second, from the perspectives of the general domain and specific large models, this paper summarizes the applications of large models and analyzes the application scenarios and functions of various types of large models. Subsequently, considering the specific needs and current state of research in the field of materials science, this paper reviews the application of large language models, large visual models, and large multimodal models. It integrates engineering strategies and zero-shot knowledge transfer learning from specific tasks in natural language processing and computer vision and referencing typical application cases, clarifying current research paradigms and limiting factors for applying large models to materials science. To verify the effectiveness and potential of the visual large model, basal experiments of image segmentation and key structure extraction are performed on the microscopic image data of four types of materials using improved SAM, including Ni-superalloy, superalloy, polycrystalline pure iron grain, and Inconel 939. The experimental results reveal that the zero-shot segmentation capability of SAM has enormous potential for accurate and efficient representation of material microstructures. With the help of tailored prompt engineering, precise masks of the precipitated phase, grain boundaries, and cracks can be outputted without any label. Finally, this paper proposes future research opportunities for technologies and methods related to large models in materials science. This paper assesses the feasibility and technical challenges for the development and tuning of unimodal to comprehensive multimodal large models. With continuous innovations and collaborations, the horizon for large models in materials science seems boundlessly promising. The integration of these models can produce a new era of advanced research, leading to advancements that were previously considered unattainable. The symbiosis between materials science and large models can pave the way for unforeseen discoveries, enriching our scientific prowess.

In some complex systems, because of the influence of internal and external factors, periodic changes occur among runtime stages, with each stage exhibiting distinct dynamics. When we employ data-driven parameterized methods to model and predict such systems, a unified model restricts the learning of the dynamics and transitions of multiple stages. To address the aforementioned challenges, inspired by the ordinary differential equations network (ODENet), this paper proposes a novel predictive simulation framework, referred to as the deterministic finite automaton ordinary differential equation net (DFA-ODENet). This framework is a continuous-time deep learning framework designed to model periodic multistage systems using irregularly–sampled historical system trajectories. The model includes two principal predictions for forecasting system dynamics and stage transition. In terms of learning the dynamics of the system, the model comprises several ODENets, whose number is determined from the number of stages of the modeled system. Each ODENet individually learns the continuous-time nonlinear dynamics within its respective stage. For learning the stage transitions, a stage transition predictor is employed to learn the duration of each stage from observational data. These stage transition predictors are prelabeled based on the prior knowledge of the system. During prediction, the stage transition predictor serves as a switcher for selecting the appropriate ODENet to predict the system outputs. Moreover, the framework incorporates a specific encoder–decoder structure, where the encoder solves the initial state based on historical system inputs and outputs, while the decoder predicts future system outputs using the inputs of the prediction window based on the solved initial state. To evaluate the feasibility and effectiveness of the proposed approach, the encoder–decoder framework is employed in a cooling system of a real data center to simulate specific dynamic variables during operation. After providing multivariate operational data, including server power and environmental temperature, the model successfully simulates the system behavior in the expected operational patterns and predicts the open-loop output variables, such as power consumption and inlet air temperature. Notably, when the prediction horizon extends beyond 30 min, the mean absolute percentage error of the predicted energy consumption remains

To reduce the communication and computation overhead of neural networks, a novel pinning event-triggered scheme (PETS) is developed in this paper, which enables pinning synchronization of uncertain coupled memristive neural networks (CMNNs) under limited resources. Time-varying delays, uncertainties, and mismatched parameters are all considered, which makes the system more interpretable. In addition, from the low energy cost point of view, an algorithm for pinned node selection is designed to further investigate the newly event-triggered function under limited communication resources. Meanwhile, based on the PETS and following the Lyapunov functional method, sufficient conditions for the pinning exponential stability of the proposed coupled error system are formulated, and the analysis of the self-triggered method shows that our method can efficiently avoid Zeno behavior under the newly determined triggered conditions, which contribute to better PETS performance. Extensive experiments demonstrate that the PETS significantly outperforms the existing schemes in terms of solution quality.

Abstract The predictive performance of supervised learning algorithms depends on the quality of labels. In a typical label collection process, multiple annotators provide subjective noisy estimates of the “truth” under the influence of their varying skill‐levels and biases. Blindly treating these noisy labels as the ground truth limits the accuracy of learning algorithms in the presence of strong disagreement. This problem is critical for applications in domains where the annotation cost is high. Such problems are extremely serious in the domain of agricultural imaging in leaf disease classification. Due to the limitation of acquisition methods, noisy examples are often not only the normal mislabelled ones, but also the samples which contain more than one instance. To cope with the combination of the two noises, label smoothing is blended in the point of increasing entropy for uncertain labels, with Taylor cross entropy loss, which is proved to be efficient to solve the problem of artificial noisy labels on public datasets. And the proposed method is called smooth‐Taylor cross entropy loss, which can deal with the real‐world noises in vision leaf disease dataset. Extensive experimental results on cassava leaf disease dataset demonstrate that our proposed approach significantly outperforms the state‐of‐the‐art counterparts.

Abstract Background Cryo-electron tomography is an important and powerful technique to explore the structure, abundance, and location of ultrastructure in a near-native state. It contains detailed information of all macromolecular complexes in a sample cell. However, due to the compact and crowded status, the missing edge effect, and low signal to noise ratio (SNR), it is extremely challenging to recover such information with existing image processing methods. Cryo-electron tomogram simulation is an effective solution to test and optimize the performance of the above image processing methods. The simulated images could be regarded as the labeled data which covers a wide range of macromolecular complexes and ultrastructure. To approximate the crowded cellular environment, it is very important to pack these heterogeneous structures as tightly as possible. Besides, simulating non-deformable and deformable components under a unified framework also need to be achieved. Result In this paper, we proposed a unified framework for simulating crowded cryo-electron tomogram images including non-deformable macromolecular complexes and deformable ultrastructures. A macromolecule was approximated using multiple balls with fixed relative positions to reduce the vacuum volume. A ultrastructure, such as membrane and filament, was approximated using multiple balls with flexible relative positions so that this structure could deform under force field. In the experiment, 400 macromolecules of 20 representative types were packed into simulated cytoplasm by our framework, and numerical verification proved that our method has a smaller volume and higher compression ratio than the baseline single-ball model. We also packed filaments, membranes and macromolecules together, to obtain a simulated cryo-electron tomogram image with deformable structures. The simulated results are closer to the real Cryo-ET, making the analysis more difficult. The DOG particle picking method and the image segmentation method are tested on our simulation data, and the experimental results show that these methods still have much room for improvement. Conclusion The proposed multi-ball model can achieve more crowded packaging results and contains richer elements with different properties to obtain more realistic cryo-electron tomogram simulation. This enables users to simulate cryo-electron tomogram images with non-deformable macromolecular complexes and deformable ultrastructures under a unified framework. To illustrate the advantages of our framework in improving the compression ratio, we calculated the volume of simulated macromolecular under our multi-ball method and traditional single-ball method. We also performed the packing experiment of filaments and membranes to demonstrate the simulation ability of deformable structures. Our method can be used to do a benchmark by generating large labeled cryo-ET dataset and evaluating existing image processing methods. Since the content of the simulated cryo-ET is more complex and crowded compared with previous ones, it will pose a greater challenge to existing image processing methods.

Interactive segmentation is a technique for picking objects of interest in images according to users’ input interactions. Some recent works take the users’ interactive input to guide the deep neural network training, where the users’ click information is utilized as weak-supervised information. However, limited by the learning capability of the model, this structure does not accurately represent the user’s interaction intention. In this work, we propose a multi-click interactive segmentation solution for employing human intention to refine the segmentation results. We propose a coarse segmentation network to extract semantic information and generate rough results. Then, we designed a feature-aware attention module according to the symmetry of user intention and image semantic information. Finally, we establish a refinement module to combine the feature-aware results with coarse masks to generate precise intentional segmentation. Furthermore, the feature-aware module is trained as a plug-and-play tool, which can be embedded into most deep image segmentation models for exploiting users’ click information in the training process. We conduct experiments on five common datasets (SBD, GrabCut, DAVIS, Berkeley, MS COCO) and the results prove our attention module can improve the performance of image segmentation networks.

Since the bacterial foraging optimization algorithm (BFO) was proposed, many variants about it have been designed in order to improve the performance and applied in different fields. Even so, people are constantly probing new methods designed to enhance the performance of BFO, so as to form new variants with superior performance. As a new variant of original BFO, bacterial foraging optimization using strategies of progressive exploitation approximating local optimum and adaptive raid (BFO-DX) was proposed. On the one hand, the strategy of progressive exploration approximating local optimum (PELO) was introduced into BFO to enhance its ability of exploitation in a local space, which enables the algorithm to find the global optima better possibly. On the other hand, the strategy of the adaptive raid for the leader (ARL) was adopted to boost the speed of convergence by strengthening its exploration capacity. The numerical experiments indicates that the BFO-DX possesses better ability of finding global optima, better stability and other acceptable terms such as iteration and running time compared with classical genetic algorithm (GA), particle swarm algorithm (PSO), and conventional bacterial foraging optimization (BFO).

Real-time and accurate vehicle tracking by Cameras and Surveillance can provide strong support for the acquisition and application of important traffic parameters, which is the basis of the traffic condition evaluation and the reasonable traffic command and dispatch. To deal with difficult problems of vehicle tracking research in a complex environments, such as occlusion, sudden illumination change, similar target interference and real-time tracking, measures are taken as follows. Firstly, the existing color local entropy particle filter tracking method is improved. The symmetry of information entropy is used to overcome the tracking failure caused by large-area occlusion. Secondly, the SIFT feature tracking method is improved to enhance real-time performance and robustness. Thirdly, two tracking methods were combined according to their characteristics, aiming at effectively improving the quasi-determination and real-time performance of vehicle tracking. Fourthly, Kalman filter was used to predict the motion state of vehicles. According to the SIFT characteristics and license plate information of vehicles, the exact position of the lost target vehicles is quickly located. It has been verified by experiments that our method has effectively improved the accuracy and real-time performance of vehicle tracking in complex situations.

The information acquisition and automatic processing technology based on visual surveillance sensors in intelligent transportation system (ITS) has become an important application field of computer vision technology. The first step of a visual traffic surveillance system usually needs to correctly detect objects from videos and classify them into different categories. In this paper, the improved spatiotemporal sample consistency algorithm (STSC) is proposed, to enhance the robustness of background subtraction in complex scenes. To address this challenge of classifying acquired from visual traffic surveillance sensors in a particular area in China, improved spatiotemporal sample consistency algorithm is proposed, which consists of two main stages. In the first stage, the robustness of moving object detection is further provided by the method we proposed based spatiotemporal sample consistency; in the second stage, we propose the target classification method based prior knowledge, in addition correcting in tracking progress. The experiments on the CDnet 2014, MIO-TCD, and BIT-Vehicle show that the method we proposed successfully overcomes the adverse effects in the complex environment with different shooting angle and resolution taken by single fixed cameras, besides effectively reduces the false alarm rate of classification.

Personalized education aims to provide cooperative and exploratory courses for students by using computer and network technology to construct a more effective cooperative learning mode, thus improving students’ cooperation ability and lifelong learning ability. Based on students’ interests, this paper proposes an effective student grouping strategy and group-oriented course recommendation method, comprehensively considering characteristics of students and courses both from a statistical dimension and a semantic dimension. First, this paper combines term frequency–inverse document frequency and Word2Vec to preferably extract student characteristics. Then, an improved K-means algorithm is used to divide students into different interest-based study groups. Finally, the group-oriented course recommendation method recommends appropriate and quality courses according to the similarity and expert score. Based on real data provided by junior high school students, a series of experiments are conducted to recommend proper social practical courses, which verified the feasibility and effectiveness of the proposed strategy.

Gesture recognition plays an important role in the applications of human-centered smart technologies. This paper designs a gesture recognition approach in a special human-computer interaction environment. Generally, in an aseptic operation room, doctors cannot access the patient's image data directly, and they have to let the nurse operate the system under their instructions. After the nurses finish their task, they also need to re-dress the surgical gown, disinfect their hands, then they can go back to the operating room. If the doctor could access the image directly by gestures during the operation, they would be able to save a lot of time and work more efficiently. Although some template matching algorithms, such as dynamic time warping (DTW), have been widely used in gesture recognition, those traditional methods are not accurate enough, and they are unable to identify continuous repetitive gestures. This paper proposes an improved DTW method based on posteriori processing, which achieved some key operations, including parameter regulations, invalid area determination, and static gesture processing. This method will address the problem of continuous repetitive gesture recognition, with higher accuracy. The experimental results show that the method can improve the accuracy of recognition by over 40%, and it can also perform real-time dynamic gesture recognition tasks effectively.

On the premise of guaranteeing the accuracy of the solution, to reduce the time spent for optimization by an existing algorithm named adaptive shrinking grid search chaos wolf optimization algorithm (ASGS-CWOA), an adaptive distribution size (ADS) wolf pack optimization algorithm using the strategy of jumping for raid (for short: WDX-WPOA) was proposed. First, the strategy of jumping for raid is proposed to improve the performance in raiding process by airlifting directly half of wolves to be around the leader wolf that can accelerate the convergence of the existing wolf pack algorithm; Moreover, an ADS was used to distribute the half of wolves to specific locations according to its rules, so as to enhance the probability of finding the optical value. Under the same condition, the results of numerical experiments demonstrate that WDX-WPOA possesses preferable optimization ability including best global search accuracy, better robustness, and fast convergence speed compared with classical genetic algorithm, particle swarm optimization algorithm, LWPS and CWOA; especially, WDX-WPOA has better performance in time than ASGS-CWOA with the prerequisite of the same solution quality.

This paper focuses on the time series prediction problem for underflow concentration of deep cone thickener. It is commonly used in the industrial sedimentation process. In this paper, we introduce a dual attention neural network method to model both spatial and temporal features of the data collected from multiple sensors in the thickener to predict underflow concentration. The concentration is the key factor for future mining process. This model includes encoder and decoder. Their function is to capture spatial and temporal importance separately from input data, and output more accurate prediction. We also consider the domain knowledge in modeling process. Several supplementary constructed features are examined to enhance the final prediction accuracy in addition to the raw data from sensors. To test the feasibility and efficiency of this method, we select an industrial case based on Industrial Internet of Things (IIoT). This Tailings Thickener is from FLSmidth with multiple sensors. The comparative results support this method has favorable prediction accuracy, which is more than 10% lower than other time series prediction models in some common error indices. We also try to interpret our method with additional ablation experiments for different features and attention mechanisms. By employing mean absolute error index to evaluate the models, experimental result reports that enhanced features and dual-attention modules reduce error of fitting ~5% and ~11%, respectively.

Recently, many related algorithms have been proposed to find an efficient wireless sensor network with good sustainability, a stable connection, and a high covering rate. To further improve the coverage rate of movable wireless sensor networks under the condition of guaranteed connectivity, this paper proposes an adaptive, discrete space oriented wolf pack optimization algorithm for a movable wireless sensor network (DSO-WPOA). Firstly, a strategy of adaptive expansion based on a minimum overlapping full-coverage model is designed to achieve minimum overlap and no-gap coverage for the monitoring area. Moreover, the adaptive shrinking grid search wolf pack optimization algorithm (ASGS-CWOA) is improved to optimize the movable wireless sensor network, which is a discrete space oriented problem. This improvement includes the usage of a target−node probability matrix and the design of an adaptive step size method, both of which work together to enhance the convergence speed and global optimization ability of the algorithm. Theoretical research and experimental results indicate that compared with the coverage algorithm based on particle swarm optimization (PSO-WSN) and classical virtual force algorithm, the newly proposed algorithm possesses the best coverage rate, better stability, acceptable performance in terms of time, advantages in energy savings, and no gaps.

With the rapid development of mobile networks and smart terminals, mobile crowdsourcing has aroused the interest of relevant scholars and industries. In this paper, we propose a new solution to the problem of user selection in mobile crowdsourcing system. The existing user selection schemes mainly include: (1) find a subset of users to maximize crowdsourcing quality under a given budget constraint; (2) find a subset of users to minimize cost while meeting minimum crowdsourcing quality requirement. However, these solutions have deficiencies in selecting users to maximize the quality of service of the task and minimize costs. Inspired by the marginalism principle in economics, we wish to select a new user only when the marginal gain of the newly joined user is higher than the cost of payment and the marginal cost associated with integration. We modeled the scheme as a marginalism problem of mobile crowdsourcing user selection (MCUS-marginalism). We rigorously prove the MCUS-marginalism problem to be NP-hard, and propose a greedy random adaptive procedure with annealing randomness (GRASP-AR) to achieve maximize the gain and minimize the cost of the task. The effectiveness and efficiency of our proposed approaches are clearly verified by a large scale of experimental evaluations on both real-world and synthetic data sets.

Multiple-phase fluids’ simulation and 3D visualization comprise an important cooperative visualization subject between fluid dynamics and computer animation. Interactions between different fluids have been widely studied in both physics and computer graphics. To further the study in both areas, cooperative research has been carried out; hence, a more authentic fluid simulation method is required. The key to a better multiphase fluid simulation result is surface extraction. Previous works usually have problems in extracting surfaces with unnatural fluctuations or detail missing. Gaps between different phases also hinder the reality of simulation. In this paper, we propose a unified surface extraction approach integrated with a modified density model for the particle-based multiphase fluid simulation. We refine the original asymmetric smoothing kernel used in the color field and address a binary tree scheme for surface extraction. Besides, we employ a multiphase fluid framework with modified density to eliminate density deviation between different fluids. With the methods mentioned above, our approach can effectively reconstruct the fluid surface for particle-based multiphase fluid simulation. It can also resolve the issue of overlaps and gaps between different fluids, which has widely existed in former methods for a long time. The experiments carried out in this paper show that our approach is able to have an ideal fluid surface condition and have good interaction effects.

The prediction of urban traffic congestion has emerged as one of the most pivotal research topics of intelligent transportation systems (ITSs). Currently, different neural networks have been put forward in the field of traffic congestion prediction and have been put to extensive use. Traditional neural network training takes a long time in addition to easily falling into the local optimal and overfitting. Accordingly, this inhibits the large-scale application of traffic prediction. On the basis of the theory of the extreme learning machine (ELM), the current paper puts forward a symmetric-ELM-cluster (S-ELM-Cluster) fast learning methodology. In this suggested methodology, the complex learning issue of large-scale data is transformed into different issues on small- and medium-scale data sets. Additionally, this methodology makes use of the extreme learning machine algorithm for the purpose of training the subprediction model on each different section of road, followed by establishing a congestion prediction model cluster for all the roads in the city. Together, this methodology fully exploits the benefits associated with the ELM algorithm in terms of accuracy over smaller subsets, high training speed, fewer parameters, and easy parallel acceleration for the realization of high-accuracy and high-efficiency large-scale traffic congestion data learning.

Multimodal emotion recognition has become one of the new research fields of human-machine interaction. This paper focuses on feature extraction and data fusion in audio-visual emotion recognition, aiming at improving recognition effect and saving storage space. A semi-serial fusion symmetric method is proposed to fuse the audio and visual patterns of emotional recognition, and a method of Symmetric S-ELM-LUPI is adopted (Symmetric Sparse Extreme Learning Machine-Learning Using Privileged Information). The method inherits the generalized high speed of the Extreme Learning Machine, and combines this with the acceleration in the recognition process by the Learning Using Privileged Information and the memory saving of the Sparse Extreme Learning Machine. It is a learning method, which improves the traditional learning methods of examples and targets only. It introduces the role of a teacher in providing additional information to enhance the recognition (test) without complicating the learning process. The proposed method is tested on publicly available datasets and yields promising results. This method regards one pattern as the standard information source, while the other pattern as the privileged information source. Each mode can be treated as privileged information for another mode. The results show that this method is appropriate for multi-modal emotion recognition. For hundreds of samples, the execution time is less than one percent seconds. The sparsity of the proposed method has the advantage of storing memory economy. Compared with other machine learning methods, this method is more accurate and stable.

Image registration is an important process in image processing which is used to improve the performance of computer vision related tasks. In this paper, a novel self-registration method, namely symmetric face normalization (SFN) algorithm, is proposed. There are three contributions in this paper. Firstly, a self-normalization algorithm for face images is proposed, which normalizes a face image to be reflection symmetric horizontally. It has the advantage that no face model needs to be built, which is always severely time-consuming. Moreover, it can be considered as a pre-processing procedure which greatly decreases the parameters needed to be adjusted. Secondly, an iterative algorithm is designed to solve the self-normalization algorithm. Finally, SFN is applied to the between-image alignment problem, which results in the symmetric face alignment (SFA) algorithm. Experiments performed on face databases show that the accuracy of SFN is higher than 0.95 when the translation on the x-axis is lower than 15 pixels, or the rotation angle is lower than 18°. Moreover, the proposed SFA outperforms the state-of-the-art between-image alignment algorithm in efficiency (about four times) without loss of accuracy.

Quantitative analysis through image processing is a key step to gain information regarding the microstructure of materials. In this paper, we develop a deep learning-based method to address the task of image segmentation for microscopic images using an Al–La alloy. Our work makes three key contributions. (1) We train a deep convolutional neural network based on DeepLab to achieve image segmentation and have significant results. (2) We adopt a local processing method based on symmetric overlap-tile strategy which makes it possible to analyze the microscopic images with high resolution. Additionally, it achieves seamless segmentation. (3) We apply symmetric rectification to enhance the accuracy of results with 3D information. Experimental results showed that our method outperforms existing segmentation methods.

In order to simulate fluid-solid boundary interaction for non-Newtonian Smoothed Particle Hydrodynamics (SPH) fluids, we present a steady and realistic fluid-solid boundary handling method using symmetrical interaction forces. Firstly, we use the improved SPH method to model the non-Newtonian fluid. Secondly, the density of boundary particle is created into the calculation of fluid-solid interaction forces. Besides, we apply friction conditions to constrain the fluid particles at the boundary. Finally, we apply the predictive-corrective scheme to correct the density deviation and improve boundary computing efficiency. The experiment confirms the feasibility for the interaction between non-Newtonian fluid and solid objects with this method. At the same time, it reflects the viscous characteristics and ensures the physical properties of non-Newtonian fluid. In addition, compared to existing methods, this method is more stable and easier to implement.

We present a symmetric particle simulation scheme for diffuse fluids based on the Lagrangian Smoothed Particle Hydrodynamics (SPH) model. In our method, the generation of diffuse particles is determined by the entropy of fluid particles, and it is calculated by the velocity difference and kinetic energy. Diffuse particles are generated near the qualified diffuse particle emitters whose diffuse material generation rate is greater than zero. Our method fits the laws of physics better, as it abandons the common practice of adding diffuse materials at the crest empirically. The coupling between diffuse materials and fluid is a post-processing step achieved by the velocity field, which enables the avoiding of the time-consuming process of cross finding neighbors. The influence weights of the fluid particles are assigned based on the degree of coupling. Therefore, it improved the accuracy of the diffuse particle position and made the simulation results more realistic. The approach is appropriate for large scale diffuse fluid, as it can be easily integrated in existing SPH simulation methods and the computational overhead is negligible.

In this paper, a symmetry particle method, the smoothed particle hydrodynamics (SPH) method, is extended to deal with non‐Newtonian fluids. First, the viscous liquid is modeled by a non‐Newtonian fluid flow and the variable viscosity under shear stress is determined by the Carreau‐Yasuda model. Then a pressure correction method is proposed, by correcting density error with individual stiffness parameters for each particle, to ensure the incompressibility of fluid. Finally, an implicit method is used to improve efficiency and stability. It is found that the nonNewtonian behavior can be well displayed in all cases, and the proposed SPH algorithm is stable and efficient.