Search

Potholes and other road surface damages pose significant risks to vehicles and traffic safety. The current methods of in situ visual inspection for potholes or cracks are inefficient, costly, and hazardous. Therefore, there is a pressing need to develop automated systems for assessing road surface conditions, aiming to efficiently and accurately reconstruct, recognize, and locate potholes. In recent years, various methods utilizing (a) computer vision, (b) three-dimensional (3D) point clouds, or (c) smartphone data have been employed to map road surface quality conditions. Machine learning and deep learning techniques have increasingly enhanced the performance of these methods. This review aims to provide a comprehensive overview of cutting-edge computer vision and machine learning algorithms for pothole detection. It covers topics such as sensing systems for acquiring two-dimensional (2D) and 3D road data, classical algorithms based on 2D image processing, segmentation-based algorithms using 3D point cloud modeling, machine learning, deep learning algorithms, and hybrid approaches. The review highlights that hybrid methods combining traditional image processing and advanced machine learning techniques offer the highest accuracy in pothole detection. Machine learning approaches, particularly deep learning, demonstrate superior adaptability and detection rates, while traditional 2D and 3D methods provide valuable baseline techniques. By reviewing and evaluating existing vision-based methods, this paper clarifies the current landscape of pothole detection technologies and identifies opportunities for future research and development. Additionally, insights provided by this review can inform the design and implementation of more robust and effective systems for automated road surface condition assessment, thereby contributing to enhanced roadway safety and infrastructure management. [ABSTRACT FROM AUTHOR]

In this paper, we proposed Mix-VIO, a monocular and binocular visual-inertial odometry, to address the issue where conventional visual front-end tracking often fails under dynamic lighting and image blur conditions. Mix-VIO adopts a hybrid tracking approach, combining traditional handcrafted tracking techniques with Deep Neural Network (DNN)-based feature extraction and matching pipelines. The system employs deep learning methods for rapid feature point detection, while integrating traditional optical flow methods and deep learning-based sparse feature matching methods to enhance front-end tracking performance under rapid camera motion and environmental illumination changes. In the back-end, we utilize sliding window and bundle adjustment (BA) techniques for local map optimization and pose estimation. We conduct extensive experimental validations of the hybrid feature extraction and matching methods, demonstrating the system's capability to maintain optimal tracking results under illumination changes and image blur. [ABSTRACT FROM AUTHOR]

Forest resources are one of the indispensable resources of the earth, which are the basis for the survival and development of human society. With the swift advancements in computer vision and artificial intelligence technology, the utilization of deep learning for smoke detection has achieved remarkable results. However, the existing deep learning models have poor performance in forest scenes and are difficult to deploy because of numerous parameters. Hence, we introduce an optimized forest fire smoke monitoring system for embedded edge devices based on a lightweight deep learning model. The model makes full use of the multi-scale variable attention mechanism of Transformer architecture to strengthen the ability of image feature extraction. Considering the needs of application scenarios, we propose an improved lightweight network model LCNet for feature extraction, which can reduce the parameters and enhance detecting ability. In order to improve running speed, a simple semi-supervised label knowledge distillation scheme is used to enhance the overall detection capability. Finally, we design and implement a forest fire smoke detection system on an embedded device, including the Jetson NX hardware platform, high-definition camera, and detection software system. The lightweight model is transplanted to the embedded edge device to achieve rapid forest fire smoke detection. Also, an asynchronous processing framework is designed to make the system highly available and robust. The improved model reduces three-fourths of the parameters and increases speed by 3.4 times with similar accuracy to the original model. This demonstrates that our system meets the precision demand and detects smoke in time. [ABSTRACT FROM AUTHOR]

Machine vision, an interdisciplinary field that aims to replicate human visual perception in computers, has experienced rapid progress and significant contributions. This paper traces the origins of machine vision, from early image processing algorithms to its convergence with computer science, mathematics, and robotics, resulting in a distinct branch of artificial intelligence. The integration of machine learning techniques, particularly deep learning, has driven its growth and adoption in everyday devices. This study focuses on the objectives of computer vision systems: replicating human visual capabilities including recognition, comprehension, and interpretation. Notably, image classification, object detection, and image segmentation are crucial tasks requiring robust mathematical foundations. Despite the advancements, challenges persist, such as clarifying terminology related to artificial intelligence, machine learning, and deep learning. Precise definitions and interpretations are vital for establishing a solid research foundation. The evolution of machine vision reflects an ambitious journey to emulate human visual perception. Interdisciplinary collaboration and the integration of deep learning techniques have propelled remarkable advancements in emulating human behavior and perception. Through this research, the field of machine vision continues to shape the future of computer systems and artificial intelligence applications. [ABSTRACT FROM AUTHOR]

Existing macroeconomic forecasting methods primarily focus on the characteristics of economic data, but they overlook the energy-related features concealed behind these economic characteristics, which may lead to inaccurate GDP predictions. Therefore, this paper meticulously analyzes the relationship between energy big data and economic data indicators, explores the coupling feature mining of energy big data and economic data, and constructs features coupling economic and energy data. Targeting the nonlinear variation coupling features in China's quarterly GDP data and using the long short-term memory (LSTM) neural network model based on deep learning, we employ wavelet analysis technology (WA) to decompose selected macroeconomic variables and construct a prediction model combining LSTM and WA, which is further compared with multiple benchmark models. The research findings show that, in terms of quarterly GDP data prediction, the combined deep learning model and wavelet analysis significantly outperform other methods. When processing structurally complex, nonlinear, and multi-variable data, the LSTM and WA combined prediction model demonstrate better generalization capabilities, with its prediction accuracy generally surpassing other benchmark models. [ABSTRACT FROM AUTHOR]

With the high demand and sub-nanometer design for integrated circuits, surface defect complexity and frequency for semiconductor wafers have increased; subsequently emphasizing the need for highly accurate fault detection and root-cause analysis systems as manual defect diagnosis is more time-intensive, and expensive. As such, machine learning and deep learning methods have been integrated to automated inspection systems for wafer map defect recognition and classification to enhance performance, overall yield, and cost-efficiency. Concurrent with algorithm and hardware advances, in particular the onset of neural networks like the convolutional neural network, the literature for wafer map defect detection exploded with new developments to address the limitations of data preprocessing, feature representation and extraction, and model learning strategies. This article aims to provide a comprehensive review on the advancement of machine learning and deep learning applications for wafer map defect recognition and classification. The defect recognition and classification methods are introduced and analyzed for discussion on their respective advantages, limitations, and scalability. The future challenges and trends of wafer map detection research are also presented. [ABSTRACT FROM AUTHOR]

This study proposes a model for transient stability assessment, which is a convolutional neural network model combined with a saliency map (S–CNN model). The convolutional neural network model is trained on dynamic data acquired through the data measurement devices of a power system. Applying the saliency map to the acquired dynamic data visually highlights the critical aspects of transient stability assessment. This reduces data training time by eliminating unnecessary aspects during the convolutional neural network model training, thus improving training efficiency. As a result, the proposed model can achieve high performance in transient stability assessment. The dynamic data are acquired by configuring benchmark models, IEEE 39 and 118 bus systems, through MATLAB/Simulink and performing time-domain simulations. Based on the acquired dynamic data, the performance of the proposed model is verified through a confusion matrix. Furthermore, an analysis of the effects of noise interference on the performance is conducted. [ABSTRACT FROM AUTHOR]

Recently, deep learning models have been widely applied to modulation recognition, and they have become a hot topic due to their excellent end-to-end learning capabilities. However, current methods are mostly based on uni-modal inputs, which suffer from incomplete information and local optimization. To complement the advantages of different modalities, we focus on the multi-modal fusion method. Therefore, we introduce an iterative dual-scale attentional fusion (iDAF) method to integrate multimodal data. Firstly, two feature maps with different receptive field sizes are constructed using local and global embedding layers. Secondly, the feature inputs are iterated into the iterative dual-channel attention module (iDCAM), where the two branches capture the details of high-level features and the global weights of each modal channel, respectively. The iDAF not only extracts the recognition characteristics of each of the specific domains, but also complements the strengths of different modalities to obtain a fruitful view. Our iDAF achieves a recognition accuracy of 93.5% at 10 dB and 0.6232 at full signal-to-noise ratio (SNR). The comparative experiments and ablation studies effectively demonstrate the effectiveness and superiority of the iDAF. [ABSTRACT FROM AUTHOR]

Neonatal jaundice is a prevalent condition among newborns, with potentially severe complications that can result in permanent brain damage if left untreated during its early stages. The existing approaches for jaundice detection involve invasive procedures such as blood sample collection, which can inflict pain and distress on the patient, and may give rise to additional complications. Alternatively, a non-invasive method using image-processing techniques and implementing kNN, Random Forest, and XGBoost machine learning algorithms as a classifier can be employed to diagnose jaundice, necessitating a comprehensive database of infant images to achieve a diagnosis with high accuracy. This data article presents the NJN collection, a repository of newborn images encompassing diverse birthweights and skin tones, spanning an age range of 2 to 8 days. The dataset is accompanied by an Excel sheet file in CSV format containing the RGB and YCrCb channel values, as well as the status of each sample. The dataset and associated resources are openly accessible at Zenodo website. Moreover, the Python code for data testing utilizing various AI techniques is provided. Consequently, this article offers an unparalleled resource for AI researchers, enabling them to train their AI systems and develop algorithms that can assist neonatal intensive care unit (NICU) healthcare specialists in monitoring neonates while facilitating the fast, real-time, non-invasive, and accurate diagnosis of jaundice. [ABSTRACT FROM AUTHOR]

Small target detection has been widely used in applications that are relevant to everyday life and have many real-time requirements, such as road patrols and security surveillance. Although object detection methods based on deep learning have achieved great success in recent years, they are not effective in small target detection. In order to solve the problem of low recognition rate caused by factors such as low resolution of UAV viewpoint images and little valid information, this paper proposes an improved algorithm based on the YOLOv5s model, called YOLOv5s-pp. First, to better suppress interference from complex backgrounds and negative samples in images, we add a CA attention module, which can better focus on task-specific important channels while weakening the influence of irrelevant channels. Secondly, we improve the forward propagation and generalisation of the network using the Meta-ACON activation function, which adaptively learns to adjust the degree of linearity or nonlinearity of the activation function based on the input data. Again, the SPD Conv module is incorporated into the network model to address the problems of reduced learning efficiency and loss of fine-grained information due to cross-layer convolution in the model. Finally, the detection head is improved by using smaller, smaller-target detection heads to reduce missed detections. We evaluated the algorithm on the VisDrone2019-DET and UAVDT datasets and compared it with other state-of-the-art algorithms. Compared to YOLOv5s, mAP@.5 improved by 7.4% and 6.5% on the VisDrone2019-DET and UAVDT datasets, respectively, and compared to YOLOv8s, mAP@.5 improved by 0.8% and 2.1%, respectively. For improving the performance of the UAV-side small target detection algorithm, it will help to enhance the reliability and safety of UAVs in critical missions such as military reconnaissance, road patrol and security surveillance. [ABSTRACT FROM AUTHOR]

Monkeypox disease is caused by a virus that causes lesions on the skin and has been observed on the African continent in the past years. The fatal consequences caused by virus infections after the COVID pandemic have caused fear and panic among the public. As a result of COVID reaching the pandemic dimension, the development and implementation of rapid detection methods have become important. In this context, our study aims to detect monkeypox disease in case of a possible pandemic through skin lesions with deep-learning methods in a fast and safe way. Deep-learning methods were supported with transfer learning tools and hyperparameter optimization was provided. In the CNN structure, a hybrid function learning model was developed by customizing the transfer learning model together with hyperparameters. Implemented on the custom model MobileNetV3-s, EfficientNetV2, ResNET50, Vgg19, DenseNet121, and Xception models. In our study, AUC, accuracy, recall, loss, and F1-score metrics were used for evaluation and comparison. The optimized hybrid MobileNetV3-s model achieved the best score, with an average F1-score of 0.98, AUC of 0.99, accuracy of 0.96, and recall of 0.97. In this study, convolutional neural networks were used in conjunction with optimization of hyperparameters and a customized hybrid function transfer learning model to achieve striking results when a custom CNN model was developed. The custom CNN model design we have proposed is proof of how successfully and quickly the deep learning methods can achieve results in classification and discrimination. [ABSTRACT FROM AUTHOR]

Recording the trajectory of table tennis balls in real-time enables the analysis of the opponent's attacking characteristics and weaknesses. The current analysis of the ball paths mainly relied on human viewing, which lacked certain theoretical data support. In order to solve the problem of the lack of objective data analysis in the research of table tennis competition, a target detection algorithm-based table tennis trajectory extraction network was proposed to record the trajectory of the table tennis movement in video. The network improved the feature reuse rate in order to achieve a lightweight network and enhance the detection accuracy. The core of the network was the "feature store & return" module, which could store the output of the current network layer and pass the features to the input of the network layer at the next moment to achieve efficient reuse of the features. In this module, the Transformer model was used to secondarily process the features, build the global association information, and enhance the feature richness of the feature map. According to the designed experiments, the detection accuracy of the network was 96.8% for table tennis and 89.1% for target localization. Moreover, the parameter size of the model was only 7.68 MB, and the detection frame rate could reach 634.19 FPS using the hardware for the tests. In summary, the network designed in this paper has the characteristics of both lightweight and high precision in table tennis detection, and the performance of the proposed model significantly outperforms that of the existing models. [ABSTRACT FROM AUTHOR]

Logo detection is one of the crucial branches in computer vision due to various real-world applications, such as automatic logo detection and recognition, intelligent transportation, and trademark infringement detection. Compared with traditional handcrafted-feature-based methods, deep learning-based convolutional neural networks (CNNs) can learn both low-level and high-level image features. Recent decades have witnessed the great feature representation capabilities of deep CNNs and their variants, which have been very good at discovering intricate structures in high-dimensional data and are thereby applicable to many domains including logo detection. However, logo detection remains challenging, as existing detection methods cannot solve well the problems of a multiscale and large aspect ratios. In this paper, we tackle these challenges by developing a novel long-range dependence involutional network (LDI-Net). Specifically, we designed a strategy that combines a new operator and a self-attention mechanism via rethinking the intrinsic principle of convolution called long-range dependence involution (LD involution) to alleviate the detection difficulties caused by large aspect ratios. We also introduce a multilevel representation neural architecture search (MRNAS) to detect multiscale logo objects by constructing a novel multipath topology. In addition, we implemented an adaptive RoI pooling module (ARM) to improve detection efficiency by addressing the problem of logo deformation. Comprehensive experiments on four benchmark logo datasets demonstrate the effectiveness and efficiency of the proposed approach. [ABSTRACT FROM AUTHOR]

For the building energy consumption models with complex scale sensitivity, it is difficult to achieve ideal prediction effect with single-granularity prediction model. Therefore, this paper proposed a multigranularity MgHa-LSTM model based on convolutional recurrent neural network, including a multigranularity feature extraction module and a long-term dependency capture module. Multigranularity feature extraction included granularity segmentation, feedback mechanism, and parallel convolutional modules, which can capture short-term scale sensitivity dependencies. Long-term dependency capture consists of a hybrid attention mechanism and long-short term memory layers, which are able to capture long-term dependencies. For building energy consumption patterns with different scale sensitivity, MgHa-LSTM, MLP, CNN, LSTM, and MsC-LSTM models were constructed on the IHEPC building energy consumption dataset used in this paper for comparative experiments. The experimental results showed that on the IHEPC dataset, the MSE of the building energy consumption prediction model is 0.2821 based on the MgHa-LSTM model proposed in this paper, which is equivalent to 93.72% of the MsC-LSTM model with the smallest MSE among other deep learning prediction models. Compared with other deep learning prediction models, the prediction results of the MgHa-LSTM building energy consumption prediction model are more accurate. [ABSTRACT FROM AUTHOR]

In a newborn baby, the particular complication is jaundice. Jaundice is due to many reasons. An increase in the bilirubin level denotes the presence of jaundice. Neonatal hyperbilirubinemia is most common in the first postnatal week. Early prediction and classification of jaundice are essential to reduce the mortality rate. Diagnosis of the disease using computer vision techniques helps to achieve accuracy in prediction and classification. Here, I studied and extracted the information from medical data mining and achieved the goal. It involves examining, analyzing, extracting, selecting, and classifying the features. Finally, it provides treatment suggestions. It enables the physician to perform early prediction and provide appropriate jaundice treatment. Computer vision and machine learning techniques make the process easy and simple. The optimized method of classification increases accuracy. The review provides a detailed description of the diagnosis of neonatal jaundice using the skin region. The entire survey is performed based on their processing steps. Examining a huge number of medical images can reduce the risk. However, the medical images incorporate a huge amount of information about the bilirubin level on the baby's skin, and the increase in the number of images makes their accurate assessment a challenging task. This paper reviews various promising methods developed in computer vision to detect and classify neonatal jaundice. It presents a comprehensive analysis of various approaches. The analysis focuses on the advantages and disadvantages of various methods. [ABSTRACT FROM AUTHOR]

With the emergence of the information age, computers have entered the homes of ordinary people and have become essential daily appliances for people. The integration of people and computers has become more popular and in-depth. Based on this situation, how to make computers and humans communicate intelligently and make human-computer interaction more convenient and practical is the main issue of scientific research and discussion on computers. Language recognition is widely used in industry, the economy, commerce, tourism, style, and sports. The BP neural network based on deep learning has supercomputing functions and is very suitable for speech recognition, which promotes the promotion and application of speech recognition technology in many fields. Different types of applications and diverse scenarios enable users to experience different personal experiences. This paper examines the use of application software for ideological and political theory courses in colleges and universities and the application of application software for ideological and political theory courses in colleges and universities in the management of ideological and political theory courses. Analyzed the use of apps in the management of ideological and political theory courses in colleges and universities, analyzed the possibilities and challenges that this brought to the management of ideological and political theory courses in colleges and universities based on the app, and put forward the necessary countermeasures. Including the development and design of ideological and political theory applications, combining deep learning and CTC algorithms to build acoustic models, using server-client interaction, designing an ideological and political theory course app based on speech recognition and deep learning, and forming an offline speech recognition system software platform, the app provides a certain reference for improving the skills of teaching managers. [ABSTRACT FROM AUTHOR]

It has been recognized that texture patterns with abundant high-frequency components, such as grass and water, produce visual masking effects, and the distortion in textures is hard to be perceived by human eyes than structure regions. However, modern video codecs in a rate-distortion optimized manner usually consume a lot of bits to encode textures, leading to the insufficiency in perceptual coding performance. Nowadays, with the rapid development of deep learning, learning based texture synthesis methods have been proposed to replace the coding process of prediction residuals to reduce the rate cost. In this paper, we present a deep texture synthesizer named edge-based texture synthesis framework (ETSF). At encoder side, the framework detects texture regions by semantic and fidelity classification criteria, and the detected regions are quantized coarsely by the hybrid coding framework. In texture characterization, ETSF extracts low-level edge features representing pixel intensity variation. Feature processing tools are developed to remove the spatiotemporal redundancy of edges. The processed edge information is compressed and transmitted. To effectively recover textures, we design an edge-based texture synthesis generative adversarial network (ETSGAN) at the decoder of ETSF, which can incorporate edge information into convolutional layers and generate realistic textures. Experimental results on a collected texture dataset show that the proposed ETSF can achieve an average of -12.8%, -14.2% and -9.6% MOS BD-rate under lowdelay_B, lowdelay_P and random_access configurations of VVC coding, respectively. [ABSTRACT FROM AUTHOR]

Smoke plumes are the first things seen from space when wildfires occur. Thus, fire smoke detection is important for early fire detection. Deep Learning (DL) models have been used to detect fire smoke in satellite imagery for fire detection. However, previous DL-based research only considered lower spatial resolution sensors (e.g., Moderate-Resolution Imaging Spectroradiometer (MODIS)) and only used the visible (i.e., red, green, blue (RGB)) bands. To contribute towards solutions for early fire smoke detection, we constructed a six-band imagery dataset from Landsat 5 Thematic Mapper (TM) and Landsat 8 Operational Land Imager (OLI) with a 30-metre spatial resolution. The dataset consists of 1836 images in three classes, namely "Smoke", "Clear", and "Other_aerosol". To prepare for potential on-board-of-small-satellite detection, we designed a lightweight Convolutional Neural Network (CNN) model named "Variant Input Bands for Smoke Detection (VIB_SD)", which achieved competitive accuracy with the state-of-the-art model SAFA, with less than 2% of its number of parameters. We further investigated the impact of using additional Infra-Red (IR) bands on the accuracy of fire smoke detection with VIB_SD by training it with five different band combinations. The results demonstrated that adding the Near-Infra-Red (NIR) band improved prediction accuracy compared with only using the visible bands. Adding both Short-Wave Infra-Red (SWIR) bands can further improve the model performance compared with adding only one SWIR band. The case study showed that the model trained with multispectral bands could effectively detect fire smoke mixed with cloud over small geographic extents. [ABSTRACT FROM AUTHOR]

This article proposes a four-dimensional brain mapping method, which can represent the continuous process of a person’s fatigue state in the form of image frames in a space-time range. This work couples 3-D-convolutional neural networks and long-short-term memory networks to form a cognitive detection model of brain fatigue dynamics, which can simulate the continuous process of a person’s brain fatigue dynamics and accurately identify different cognitive fatigue states. Our approach can be applied to any type of brain fatigue detection. [ABSTRACT FROM AUTHOR]

The detection of conductive particle images is an important part of the chip on glass (COG) process and can be used to ensure the performance of electrical connections. The segmentation of conductive particles is essential but a difficult task, since the scale and edge of the conductive particles on the chip and the imaging effect are different. In recent years, methods based on deep learning have become the representative method of image segmentation. However, the currently existing methods cannot fully consider the characteristics of conductive particles and have high model complexity. In this article, a multi-frequency feature learning-based convolutional neural network (CNN) is proposed. The entire network structure consists of a basic U-Net module and multi-frequency module (MFM), which are used to enhance multi-frequency feature fusion of conductive particles and accelerate network training. At the same time, for the feature of particle shape, an active contour without edge (ACWE) loss function is designed to extract the fine contour feature of particles. Experimental results on three datasets show the superiority of the proposed method over the major existing mainstream methods with respect to the three performance indicators: recall, precision, and Intersection-over-Union (IoU). [ABSTRACT FROM AUTHOR]

Histology images are inherently symmetric under rotation, where each orientation is equally as likely to appear. However, this rotational symmetry is not widely utilised as prior knowledge in modern Convolutional Neural Networks (CNNs), resulting in data hungry models that learn independent features at each orientation. Allowing CNNs to be rotation-equivariant removes the necessity to learn this set of transformations from the data and instead frees up model capacity, allowing more discriminative features to be learned. This reduction in the number of required parameters also reduces the risk of overfitting. In this paper, we propose Dense Steerable Filter CNNs (DSF-CNNs) that use group convolutions with multiple rotated copies of each filter in a densely connected framework. Each filter is defined as a linear combination of steerable basis filters, enabling exact rotation and decreasing the number of trainable parameters compared to standard filters. We also provide the first in-depth comparison of different rotation-equivariant CNNs for histology image analysis and demonstrate the advantage of encoding rotational symmetry into modern architectures. We show that DSF-CNNs achieve state-of-the-art performance, with significantly fewer parameters, when applied to three different tasks in the area of computational pathology: breast tumour classification, colon gland segmentation and multi-tissue nuclear segmentation. [ABSTRACT FROM AUTHOR]

The collateral score is an important biomarker in decision making for endovascular treatment (EVT) of patients with ischemic stroke. The existing collateral grading systems are based on visual inspection and prone to subjective interpretation and interobserver variation. The purpose of our work is the development of an automatic collateral scoring method. In this work, we present a method that is inspired by human collateral scoring. Firstly, we define an anatomical region by atlas-based registration and extract vessel structures using a deep convolutional neural network. From this, high-level features based on the ratios of vessel length and volume of the occluded and the contralateral side are defined. Multi-class classification models are used to map the feature space to a four-grade collateral score and a quantitative score. The dataset used for training, validation and testing is from a registry of images acquired in clinical routine at multiple medical centers. The model performance is tested on 269 subjects, achieving an accuracy of 0.8. The dichotomized collateral score accuracy is 0.9. The error is comparable to the interobserver variation, the results are comparable to the performance of two radiologists with 10 to 30 years of experience. [ABSTRACT FROM AUTHOR]

There is an increasing demand for computing the relevant structures, equilibria, and long-timescale kinetics of biomolecular processes, such as protein-drug binding, from highthroughput molecular dynamics simulations. Current methods employ transformation of simulated coordinates into structural features, dimension reduction, clustering the dimension-reduced data, and estimation of a Markov state model or related model of the interconversion rates between molecular structures. This handcrafted approach demands a substantial amount of modeling expertise, as poor decisions at any step will lead to large modeling errors. Here we employ the variational approach for Markov processes (VAMP) to develop a deep learning framework for molecular kinetics using neural networks, dubbed VAMPnets. A VAMPnet encodes the entire mapping from molecular coordinates to Markov states, thus combining the whole data processing pipeline in a single end-to-end framework. Our method performs equally or better than state-of-the-art Markov modeling methods and provides easily interpretable few-state kinetic models. [ABSTRACT FROM AUTHOR]

The rise of deep learning technology has broadly promoted the practical application of artificial intelligence in production and daily life. In computer vision, many human-centered applications, such as video surveillance, human-computer interaction, digital entertainment, etc., rely heavily on accurate and efficient human pose estimation techniques. Inspired by the remarkable achievements in learning-based 2D human pose estimation, numerous research studies are devoted to the topic of 3D human pose estimation via deep learning methods. Against this backdrop, this paper provides an extensive literature survey of recent literature about deep learning methods for 3D human pose estimation to display the development process of these research studies, track the latest research trends, and analyze the characteristics of devised types of methods. The literature is reviewed, along with the general pipeline of 3D human pose estimation, which consists of human body modeling, learning-based pose estimation, and regularization for refinement. Different from existing reviews of the same topic, this paper focus on deep learning-based methods. The learning-based pose estimation is discussed from two categories: single-person and multi-person. Each one is further categorized by data type to the image-based methods and the video-based methods. Moreover, due to the significance of data for learning-based methods, this paper surveys the 3D human pose estimation methods according to the taxonomy of supervision form. At last, this paper also enlists the current and widely used datasets and compares performances of reviewed methods. Based on this literature survey, it can be concluded that each branch of 3D human pose estimation starts with fully-supervised methods, and there is still much room for multi-person pose estimation based on other supervision methods from both image and video. Besides the significant development of 3D human pose estimation via deep learning, the inherent ambiguity and occlusion problems remain challenging issues that need to be better addressed. [ABSTRACT FROM AUTHOR]

Cryo-electron microscopy (cryo-EM) is becoming the imaging method of choice for determining protein structures. Many atomic structures have been resolved based on an exponentially growing number of published three-dimensional (3D) high resolution cryo-EM density maps. However, the resolution value claimed for the reconstructed 3D density map has been the topic of scientific debate for many years. The Fourier Shell Correlation (FSC) is the currently accepted cryo-EM resolution measure, but it can be subjective, manipulated, and has its own limitations. In this study, we first propose supervised deep learning methods to extract representative 3D features at high, medium and low resolutions from simulated protein density maps and build classification models that objectively validate resolutions of experimental 3D cryo-EM maps. Specifically, we build classification models based on dense artificial neural network (DNN) and 3D convolutional neural network (3D CNN) architectures. The trained models can classify a given 3D cryo-EM density map into one of three resolution levels: high, medium, low. The preliminary DNN and 3D CNN models achieved 92.73% accuracy and 99.75% accuracy on simulated test maps, respectively. Applying the DNN and 3D CNN models to thirty experimental cryo-EM maps achieved an agreement of 60.0% and 56.7%, respectively, with the author published resolution value of the density maps. We further augment these previous techniques and present preliminary results of a 3D U-Net model for local resolution classification. The model was trained to perform voxel-wise classification of 3D cryo-EM density maps into one of ten resolution classes, instead of a single global resolution value. The U-Net model achieved 88.3% and 94.7% accuracy when evaluated on experimental maps with local resolutions determined by MonoRes and ResMap methods, respectively. Our results suggest deep learning can potentially improve the resolution evaluation process of experimental cryo-EM maps. [ABSTRACT FROM AUTHOR]