Search

Lidar data, which can describe the 3D spatial information of the environment in the form of point clouds, play an important role in autonomous driving and related downstream tasks such as 3D object detection. However, unlike for images, collecting and labeling lidar data is often very expensive. As an effective means to increase the quantity of annotated data for training deep learning models, data augmentation (DA) has been widely used in the image field, but studies on augmenting lidar point clouds are only at the beginning stage. In this article, we propose AdaMix, a novel framework for lidar DA via adaptive resampling of multiscale object mixup. AdaMix contains two different object mixup schemes, i.e., object-level and part-level mixup, to augment the lidar data with the existing object instances from different scenes. For object-level mixup, a learning-based point upsampling operation is employed to obtain a set of dense objects, such as vehicles and pedestrians. For part-level mixup, parts from different vehicles are composed together and upsampled to generate vehicles of complete and dense shapes. To mix the dense objects into a new scene, AdaMix introduces a novel projection-based downsampling method to adaptively downsample the objects based on the location generated from a location sampling module. We evaluate the performance of AdaMix with several 3D object detection models on the KITTI dataset. Experimental results demonstrate that AdaMix consistently surpasses state-of-the-art lidar DA methods in improving the average precision of vehicle and pedestrian detection. [ABSTRACT FROM AUTHOR]

Industrial semiconductor electrodeposition plating cells require recirculation of process chemicals with consistent flow and minimal contaminants to prevent defects from developing during film deposition. This manuscript investigates how recirculation nozzle quality and nozzle machining can affect bath chemical uniformity. Computational fluid dynamics simulations are utilized to visualize bath chemical velocities based on variable nozzle conditions in four case studies. Results show that strict quality control of inlet nozzles, in conjunction with proper mounting angles, induce laminar bath flow. Greater fluid uniformity and laminar flow translate to a reduction of in-line defects and increased wafer yield. [ABSTRACT FROM AUTHOR]

Recently audio-driven talking face video generation has attracted considerable attention. However, very few researches address the issue of emotional editing of these talking face videos with continuously controllable expressions, which is a strong demand in the industry. The challenge is that speech-related expressions and emotion-related expressions are often highly coupled. Meanwhile, traditional image-to-image translation methods cannot work well in our application due to the coupling of expressions with other attributes such as poses, i.e., translating the expression of the character in each frame may simultaneously change the head pose due to the bias of the training data distribution. In this paper, we propose a high-quality facial expression editing method for talking face videos, allowing the user to control the target emotion in the edited video continuously. We present a new perspective for this task as a special case of motion information editing, where we use a 3DMM to capture major facial movements and an associated texture map modeled by a StyleGAN to capture appearance details. Both representations (3DMM and texture map) contain emotional information and can be continuously modified by neural networks and easily smoothed by averaging in coefficient/latent spaces, making our method simple yet effective. We also introduce a mouth shape preservation loss to control the trade-off between lip synchronization and the degree of exaggeration of the edited expression. Extensive experiments and a user study show that our method achieves state-of-the-art performance across various evaluation criteria. [ABSTRACT FROM AUTHOR]

This article proposes a data-driven approach for generating personalized smile style images for neutral expressions, which aims to produce diverse smile styles while preserving individual features. Unlike other generator models that require expensive manual facial attribute labeling, we designed an auxiliary expression attention Siamese network (EASN) to extract identity-irrelevant facial expression attention regions and guide the proposed two-stage style-expression generative adversarial network (style-exprGAN). The first generator stage generates the overall facial geometry and virtual smile features, while the second stage refines the image quality. Additionally, we introduced traditional geometry warping methods to include registered neutral expression images for consistent transformation and realistic texture fusion. Results show that the proposed method effectively synthesizes realistic and diverse smile styles while preserving individual features. Furthermore, we demonstrate the potential of our data-driven approach by applying the generated personalized smile style images to image augmentation tasks, improving the stability and robustness of facial recognition models. [ABSTRACT FROM AUTHOR]

Exploring effective approaches to enhance face-to-face interactions and interpersonal relationships is an important topic in the applications of affective computing. According to the co-actualization model, we propose a face-to-face co-participation serious game for relationship enhancement, with a focus on battling COVID-19. Moreover, a prototype system is developed using an immersive virtual environment and a low-cost brain-computer interface. Through this system, a dynamic flow experience enhancement tool is utilized to involve partners in the cooperative task. To evaluate the system performance, two studies are conducted with schoolmates as participants. Study 1 compares the cooperative and competitive modes, and demonstrates that the former elicited higher level of decision-making challenge and affections, which are beneficial for forming relationships. Study 2 further examines the effect of the dynamic flow enhancement tool in the cooperative task and the results show its effectiveness in promoting flow experience, perceived closeness, and intimacy in relationships. Given this short-term participation, participants felt a greater sense of closeness and intimacy than they had before the test. In conclusion, our proposed system is effective in enhancing schoolmate relationships. [ABSTRACT FROM AUTHOR]

This paper presents a 3D tolerance analysis approach for linear dimensions applied to planar faces in an assembly. The assembly variations are generated and visualized as an explicit geometrical stack-up of the component variations using the solid modeller Solidworks®. The feature variations are obtained by adapting the geometric solid model of each component, either by offsetting the target planar face or by tilting it within the tolerance zone. A concept of Oriented Minimum Bounding Box (OMBB) is introduced to generate individual component variations with any generalized shape of the target planar face. The analysis of the OMBB extents, the tilting angles and the corresponding pivot points has revealed symmetry in these data. Rigorous mathematical formulations have been implemented in this study to handle the general case of large and small displacements. An approach is suggested to evaluate the functional dimensions, the target face's centroid and normal for each assembly variation. Functional dimensions of the assembly variations obtained by the software '3DCS Variation Analyst' are found to deviate from those obtained by the proposed approach by up to 40% of the assembly tolerance size. 3DCS tool has also failed to detect out-of-specification assembly variations, which were identified by the proposed approach. [ABSTRACT FROM AUTHOR]

Earthquakes present worldwide risk to economic and human safety. The 2023 earthquakes in Türkiye provided a reminder of the potential for catastrophic consequences with 50,700 deaths and 15.7 million people affected. The ability to predict ground motions and infrastructure damage for earthquakes continues to be a challenging problem for scientists and engineers. Until now, estimates of ground motions have been performed empirically by looking at sparse data from past earthquakes. This approach can provide statistical information on intensity amplitudes but cannot inform site-specific ground motions essential to developing the most effective resilience. Interest has grown in large-scale computational models to simulate earthquakes at regional scale. The U.S. Department of Energy EarthQuake SIMulation (EQSIM) framework was developed for regional-scale earthquake simulations at unprecedented fidelity, taking advantage of emerging GPU-accelerated systems. This article describes the EQSIM workflow and demonstrates regional-scale simulations with the new computational capability available to scientists in their quest to mitigate future disasters. [ABSTRACT FROM AUTHOR]

In this article, we address the problem of 4D facial expressions generation. This is usually addressed by animating a neutral 3D face to reach an expression peak, and then get back to the neutral state. In the real world though, people show more complex expressions, and switch from one expression to another. We thus propose a new model that generates transitions between different expressions, and synthesizes long and composed 4D expressions. This involves three sub-problems: (1) modeling the temporal dynamics of expressions, (2) learning transitions between them, and (3) deforming a generic mesh. We propose to encode the temporal evolution of expressions using the motion of a set of 3D landmarks, that we learn to generate by training a manifold-valued GAN (Motion3DGAN). To allow the generation of composed expressions, this model accepts two labels encoding the starting and the ending expressions. The final sequence of meshes is generated by a Sparse2Dense mesh Decoder (S2D-Dec) that maps the landmark displacements to a dense, per-vertex displacement of a known mesh topology. By explicitly working with motion trajectories, the model is totally independent from the identity. Extensive experiments on five public datasets show that our proposed approach brings significant improvements with respect to previous solutions, while retaining good generalization to unseen data. [ABSTRACT FROM AUTHOR]

Recognising continuous emotions and action unit (AU) intensities from face videos, requires a spatial and temporal understanding of expression dynamics. Existing works primarily rely on 2D face appearance features to extract such dynamics. This work focuses on a promising alternative based on parametric 3D face alignment models, which disentangle different factors of variation, including expression-induced shape variations. We aim to understand how expressive 3D face shapes are in estimating valence-arousal and AU intensities compared to the state-of-the-art 2D appearance-based models. We benchmark five recent 3D face models: ExpNet, 3DDFA-V2, RingNet, DECA, and EMOCA. In valence-arousal estimation, expression features of 3D face models consistently surpassed previous works and yielded an average concordance correlation of. 745 and. 574 on SEWA and AVEC 2019 CES corpora, respectively. We also study how 3D face shapes performed on AU intensity estimation on BP4D and DISFA datasets, and report that 3D face features were on par with 2D appearance features in recognising AUs 4, 6, 10, 12, and 25, but not the entire set of AUs. To understand this discrepancy, we conduct a correspondence analysis between valence-arousal and AUs, which points out that accurate prediction of valence-arousal may require the knowledge of only a few AUs. [ABSTRACT FROM AUTHOR]

One of the primary difficulties in treating patients with genetic syndromes is diagnosing their condition. Many syndromes are associated with characteristic facial features that can be imaged and utilized by computer-assisted diagnosis systems. In this work, we develop a novel 3D facial surface modeling approach with the objective of maximizing diagnostic model interpretability within a flexible deep learning framework. Therefore, an invertible normalizing flow architecture is introduced to enable both inferential and generative tasks in a unified and efficient manner. The proposed model can be used (1) to infer syndrome diagnosis and other demographic variables given a 3D facial surface scan and (2) to explain model inferences to non-technical users via multiple interpretability mechanisms. The model was trained and evaluated on more than 4700 facial surface scans from subjects with 47 different syndromes. For the challenging task of predicting syndrome diagnosis given a new 3D facial surface scan, age, and sex of a subject, the model achieves a competitive overall top-1 accuracy of 71%, and a mean sensitivity of 43% across all syndrome classes. We believe that invertible models such as the one presented in this work can achieve competitive inferential performance while greatly increasing model interpretability in the domain of medical diagnosis.

To ensure post OPC data quality, examination based on estimated resist contours at resist bottom alone is insufficient, reliable prediction of lithography performance within process window must rely on complete information of on-wafer resist 3D structures. In this regard, resist 3D structure model, in particular, the through focus resist 3D structure model, with full chip capability will be the ultimate model in demand. To develop machine learning resist 3D structure models,we have proposed the physics-based information encoding scheme, together with carefully chosen deep convolution neural network and model training strategies. Our proposed through focus resist 3D structure model is based on conditional U-net structure with first five eigen images as model’s main inputs and the focus setting as the conditional input. The average normalized cross correlation (NCC) or mean structure similarity index between ground truth and model predicted resist 3D structures can reach 0.92. With single GPU (Tesla M60), it takes 6.1ms for the model to produce resist 3D structure covering area of 1.8umx1.8 $\mu {\mathrm{ m}}$. The model is fast enough and can be engineered for full chip implementation. The model can extend the capability of detecting lithography process window aware resist loss related hotspots. [ABSTRACT FROM AUTHOR]

The Visual Multimethod Assessment Fusion (VMAF) algorithm has recently emerged as a state-of-the-art approach to video quality prediction, that now pervades the streaming and social media industry. However, since VMAF requires the evaluation of a heterogeneous set of quality models, it is computationally expensive. Given other advances in hardware-accelerated encoding, quality assessment is emerging as a significant bottleneck in video compression pipelines. Towards alleviating this burden, we propose a novel Fusion of Unified Quality Evaluators (FUNQUE) framework, by enabling computation sharing and by using a transform that is sensitive to visual perception to boost accuracy. Further, we expand the FUNQUE framework to define a collection of improved low-complexity fused-feature models that advance the state-of-the-art of video quality performance with respect to both accuracy, by 4.2% to 5.3%, and computational efficiency, by factors of 3.8 to 11 times! [ABSTRACT FROM AUTHOR]

The emergence of FluidFM technology has presented a novel opportunity for selective aspiration and distribution of liquids at the sub-micron scale. However, precise control over droplet generation volume remains a challenging issue. This study investigates the impact of dynamic parameters between the tip and surface on droplet formation. Initially, we establish a finite element model to simulate the process from the formation of a liquid bridge between the hollow cantilever probe and the substrate to the rupture resulting in droplet formation. Additionally, we propose an image-based method for quantifying droplet volume. Subsequently, we construct a comprehensive experimental framework to delve into the influence of these parameters on microdroplet formation by monitoring the dynamic interaction parameters between the probe tip and the liquid surface. Finally, we optimize the parameters during the droplet generation process, enabling the formation of uniform arrays of microdroplets under controlled experimental conditions. Our study provides a reference for automation parameters of serial experiments and improves experimental throughput. [ABSTRACT FROM AUTHOR]

Cardiovascular disease (CVD) is the leading cause of mortality worldwide and its incidence is rising due to an aging population. The development and progression of CVD is directly linked to adverse vascular hemodynamics and biomechanics, whose in-vivo measurement remains challenging but can be simulated numerically and experimentally. The ability to evaluate these parameters in patient-specific CVD cases is crucial to better predict future disease progression, risk of adverse events, and treatment efficacy. While significant progress has been made toward patient-specific hemodynamic simulations, blood vessels are often assumed to be rigid, which does not consider the compliant mechanical properties of vessels whose malfunction is implicated in disease. In an effort to simulate the biomechanics of flexible vessels, fluid-structure interaction (FSI) simulations have emerged as promising tools for the characterization of hemodynamics within patient-specific cardiovascular anatomies. Since FSI simulations combine the blood's fluid domain with the arterial structural domain, they pose novel challenges for their experimental validation. This paper reviews the scientific work related to FSI simulations for patient-specific arterial geometries and the current standard of FSI model validation including the use of compliant arterial phantoms, which offer novel potential for the experimental validation of FSI results. [ABSTRACT FROM AUTHOR]

ObjectiveAtherosclerosis is a chronic immuno-inflammatory condition emerging in arteries and considered the cause of a myriad of cardiovascular diseases. Atherosclerotic lesion characterization through invasive imaging modalities is essential in disease evaluation and determining intervention strategy. Recently, electrical properties of the lesions have been utilized in assessing its vulnerability mainly owing to its capability to differentiate lipid content existing in the lesion, albeit with limited detection resolution. Electrical impedance tomography is the natural extension of conventional spectrometric measurement by incorporating larger number of interrogating electrodes and advanced algorithm to achieve imaging of target objects and thus provides significantly richer information. It is within this context that we develop Outward Electrical Impedance Tomography (OEIT), aimed at intravascular imaging for atherosclerotic lesion characterization.MethodsWe utilized flexible electronics to establish the 32-electrode OEIT device with outward facing configuration suitable for imaging of vessels. We conducted comprehensive studies through simulation model and ex vivo setup to demonstrate the functionality of OEIT.ResultsQuantitative characterization for OEIT regarding its proximity sensing and conductivity differentiation was achieved using well-controlled experimental conditions. Imaging capability for OEIT was further verified with phantom setup using porcine aorta to emulate in vivo environment.ConclusionWe have successfully demonstrated a novel tool for intravascular imaging, OEIT, with unique advantages for atherosclerosis detection.SignificanceThis study demonstrates for the first time a novel electrical tomography-based platform for intravascular imaging, and we believe it paves the way for further adaptation of OEIT for intravascular detection in more translational settings and offers great potential as an alternative imaging tool for medical diagnosis.

Using a series of detector measurements taken at different locations to localize a source of radiation is a well-studied problem. The source of radiation is sometimes constrained to a single point-like source, in which case the location of the point source can be found using techniques such as maximum likelihood. Recent advancements have shown the ability to locate point sources in 2-D and even 3-D but few have studied the effect of intervening material on the problem. In this work, we examine gamma-ray data taken from a freely moving system and develop voxelized 3-D models of the scene using data from its onboard light detection and ranging (LiDAR) unit. Ray casting is used to compute the distance each gamma ray travels through the scene material, which is then used to calculate attenuation assuming a single attenuation coefficient for solids within the geometry. Parameter estimation using maximum likelihood is performed to simultaneously find the attenuation coefficient, source activity, and source position that best match the data. Using a simulation, we validate the ability of this method to reconstruct the true location and activity of a source, along with the true attenuation coefficient of the structure it is inside, and then we apply the method to measured data with sources and find good agreement.

Cardiovascular disease (CVD) threatens the lives of many and affects their productivity. Wearable sensors can enable continuous monitoring of hemodynamic parameters to improve the diagnosis and management of CVD. Bio-Impedance (Bio-Z) is an effective non-invasive sensor for arterial pulse wave monitoring based on blood volume changes in the artery due to the deep penetration of its current signal inside the tissue. However, the measured data are significantly affected by the placement of electrodes relative to the artery and the electrode configuration. In this work, we created a Bio-Z simulation platform that models the tissue, arterial pulse wave, and Bio-Z sensing configuration using a 3D circuit model based on a time-varying impedance grid. A new method is proposed to accurately simulate the different tissue types such as blood, fat, muscles, and bones in a 3D circuit model in addition to the pulsatile activity of the arteries through a variable impedance model. This circuit model is simulated in SPICE and can be used to guide design decisions (i.e. electrode placement relative to the artery and electrode configuration) to optimize the monitoring of pulse wave prior to experimentation. We present extensive simulations of the arterial pulse waveform for different sensor locations, electrode sizes, current injection frequencies, and artery depths. These simulations are validated by experimental Bio-Z measurements.

Automatic Feature Recognition (AFR) is considered as the key connection technique of the integration of Computer Aided Design (CAD) and Computer Aided Process Planning (CAPP). At present, there is a lack of a systematic method to identify and evaluate the local features of 3D CAD models. The process information such as topological structure, shape and size, tolerance and surface roughness should be considered. Therefore, a novel Model Based Definition (MBD) based on 3D CAD model AFR and similarity evaluation are proposed in this paper. A Multi-Dimensional Attributed Adjacency Matrix (MDAAM) based on MBD is established based on the fully consideration of the topological structure, shape and size, surface roughness, tolerance and other process information of the B-rep model. Based on the MDAAM, a two-stage model local feature similarity evaluation method is proposed, which combines the methods of optimal matching and adjacency judgment. First, the faces of source feature and target model are used as independent sets to construct a bipartite graph. Secondly, supplement the vertices in the independent set of source feature to make the number of vertices in two independent sets equal. Thirdly, based on MDAAM data, the weighted complete bipartite graph is constructed with the face similarity between two independent sets as the weight. Fourthly, Kuhn-Munkres algorithm is used to calculate the optimal matching between the faces of source feature and target model. Fifthly, the adjacency between matching faces in target model is judged. Finally, the similarity between matching faces of the two models is calculated, which is used as the similarity evaluation result. The effectiveness of this method is verified by three applications.

Metaverse is a novel innovative platform that connects users worldwide in the distributed virtual environment. People share their interests, opinions, and resources on this virtual reality platform. With this, we come to know that besides other fundamental techniques, the language generation method is also a necessity to regulate the VR environment. There are several types of language generation methods in 3D, including neural learning, such as GRU, RNN, and GPT-3, and transfer learning. This paper proposes a recurrent voting generator (RVG) system that understands the 3D text of a book and performs emotional analytics within a metaverse space. The proposed model RVG evaluates emotions through three algorithms such as the first module is a recurrent sentiment generator (RSG) that analyzes emotions and calculates and generates the distributions. The second module is the sentiment decomposition (SD) that optimizes higher dimensions in Big Data. And, the third module is the compound voting learning (CVL) module that performs calculations with an emphasis on optimal performance. The dataset used to evaluate RVG is based on the content of movie reviews and books. The performance evaluation shows that the proposed approach outperforms better compared to the existing 2D RNN models in the metaverse. [ABSTRACT FROM AUTHOR]

Instead of predicting just one emotion for one activity (e.g., video watching), fine-grained emotion recognition enables more temporally precise recognition. Previous works on fine-grained emotion recognition require segment-by-segment, fine-grained emotion labels to train the recognition algorithm. However, experiments to collect these labels are costly and time-consuming compared with only collecting one emotion label after the user watched that stimulus (i.e., the post-stimuli emotion labels). To recognize emotions at a finer granularity level when trained with only post-stimuli labels, we propose an emotion recognition algorithm based on Deep Multiple Instance Learning (EDMIL) using physiological signals. EDMIL recognizes fine-grained valence and arousal (V-A) labels by identifying which instances represent the post-stimuli V-A annotated by users after watching the videos. Instead of fully-supervised training, the instances are weakly-supervised by the post-stimuli labels in the training stage. The V-A of instances are estimated by the instance gains, which indicate the probability of instances to predict the post-stimuli labels. We tested EDMIL on three different datasets, CASE, MERCA and CEAP-360VR, collected in three different environments: desktop, mobile and HMD-based Virtual Reality, respectively. Recognition results validated with the fine-grained V-A self-reports show that for subject-independent 3-class classification (high/neutral/low), EDMIL obtains promising recognition accuracies: 75.63% and 79.73% for V-A on CASE, 70.51% and 67.62% for V-A on MERCA and 65.04% and 67.05% for V-A on CEAP-360VR. Our ablation study shows that all components of EDMIL contribute to both the classification and regression tasks. Our experiments also show that (1) compared with fully-supervised learning, weakly-supervised learning can reduce the problem of overfitting caused by the temporal mismatch between fine-grained annotations and physiological signals, (2) instance segment lengths between 1-2 s result in the highest recognition accuracies and (3) EDMIL performs best if post-stimuli annotations consist of less than 30% or more than 60% of the entire video watching. [ABSTRACT FROM AUTHOR]

To compensate for the repositioning error (RE) introduced by the radar position offset in discontinuous ground-based synthetic aperture radar (GBSAR) monitoring, a new mathematical framework for modeling the baseline error based on the Taylor expansion is developed in this letter. And then, a novel 3-D model called multiparameter nonlinear trigonometric model (MNTM) is proposed to accurately compensate for the RE. Furthermore, to improve the compensating efficiency, we further develop an efficient 2-D model called linear trigonometric model (LTM). Both simulation and field experiments verify the superiority and feasibility of the proposed methods, which measure the displacement with sub-millimeter accuracy. [ABSTRACT FROM AUTHOR]

This letter addresses the challenges associated with interpreting synthetic aperture radar (SAR) images, which provide limited visual information compared with optical images. We propose a new method for generating virtual data and an SAR-to-optical image translation neural network to recognize targets in SAR images. The creation of virtual data involves classifying it into targets and backgrounds. The target data generate a virtual optical image using a 3-D model, while the background data are synthesized by combining the target with the existing SAR image. For image translation using the virtual dataset, we design a modified dense nested U-net that converts images for target recognition. By incorporating the proposed translation network into the YOLO v4 detection algorithm, we verify the impact of virtual optical images on target recognition. The experimental results demonstrate that our proposed method outperforms the conventional approach, which relies solely on SAR target image data for learning. [ABSTRACT FROM AUTHOR]

Multigrid (MG) methods are among the best choice for stable and efficient 3D forward modeling of electromagnetic (EM) fields over large area due to their linear dependence of computational time on grid size. However, as the frequency decreases to near zero and/or the grid is increasingly stretched, MG solvers for EM modeling based on the curl-curl equations converge slowly or even diverge. In this letter, we develop an efficient MG algorithm combined with a two-color plane Gauss-Seidel (GS) smoother for finite difference forward modeling of EM fields particularly at low frequencies. In this algorithm, we group different planes of the grid nodes into two colors. In each color, the components attached to different planes are totally decoupled and can be solved simultaneously, which can be distributed to different processors. The Dublin Test Model 1 is used to verify the accuracy of our algorithm and examine the numerical performance of our method against MG algorithms based on a four-color cell-block GS smoother and the Bi-Conjugate Gradient stabilized (BICGstab) smoother (as four-color cell-block GS MG and BICGstab-MG, respectively), and BICGstab and Quasi-Minimal Residual (QMR) both preconditioned with block incomplete lower-upper (blockILU) decomposition (as blockILU-BICGstab and blockILU-QMR, respectively). The numerical test based on OpenMP shows the good parallelization of our algorithm. Grids stretched to different degrees are designed to examine its ability to handle grid-stretching. The numerical performance comparison indicates its remarkable dominance in efficiency and stability. [ABSTRACT FROM AUTHOR]

Street trees providing ecological and cultural functions may block traffic signs from the driver’s view if they are not properly maintained. Aiming at the problems of low automation and poor refinement of the detection of tree areas blocking traffic signboards and the inability to make pruning simulation and quantification of the occlude branches, in this letter, the detection of trees to be maintained and the simulation of branch pruning are studied based on mobile laser scanning (MLS) point clouds. First, the tree areas that block the cantilever traffic signs from the driver’s view are automatically and accurately detected based on the line of sight analysis. Then, via structured modeling, the pruning effect of trees under different pruning degrees is simulated in 3-D, and four indicators (the number of pruning positions, the pruning branch volume, the pruning ratio of branch volume, and the pruning ratio of leaves) are proposed to quantify the pruning. The application in four different scenes proves the effectiveness of the proposed method. This research is helpful to automatically and accurately identify the street tree branches to be maintained and provides an important reference for the formulation of branch pruning schemes that can comprehensively balance the pruning cost, tree ecological benefits, and cultural benefits. [ABSTRACT FROM AUTHOR]

Existing point cloud modeling datasets primarily express the modeling precision by pose or trajectory precision rather than the point cloud modeling effect itself. Under this demand, we first independently construct a set of LiDAR system with an optical stage, and then we build an HPMB dataset based on the constructed LiDAR system, a High-Precision, Multi-Beam, real-world dataset. Second, we propose a modeling evaluation method based on HPMB for object-level modeling to overcome this limitation. In addition, the existing point cloud modeling methods tend to generate continuous skeletons of the global environment, hence lacking attention to the shape of complex objects. To tackle this challenge, we propose a novel learning-based joint framework, DSMNet, for high-precision 3-D surface modeling from sparse point cloud frames. DSMNet comprises density-aware point cloud registration (PCR) and geometry-aware point cloud sampling (PCS) to effectively learn the implicit structure feature of sparse point clouds. Extensive experiments demonstrate that DSMNet outperforms the state-of-the-art methods in PCS and PCR on the Multi-View Partial Point Cloud (MVP) database. Furthermore, the experiments on the open-source KITTI and our proposed HPMB datasets show that DSMNet can be generalized as a postprocessing of simultaneous localization and mapping (SLAM), thereby improving modeling precision in environments with sparse point clouds. [ABSTRACT FROM AUTHOR]

This letter presents a new decomposition approach with independent physical scattering models for odd- and even-bounce mechanisms. The rationale behind the method is to provide orthogonality between the odd- and even-bounce scattering components by removing their correlation from a rank-deficient residue coherency matrix by two successive unitary rotations. The rotated residue coherency matrix is then decomposed into the sum of two rank-1 orthogonal Hermitian positive semi-definite matrices. We show that these two orthogonal matrices consistently depict a single-type scattering phenomenon. Therefore, without branching criteria, one can independently compute the nonnegative odd- and even-bounce scattering powers. We used two full polarimetric SAR data to validate the effectiveness of the proposed method. We first affirm the orthogonality of the proposed scattering models. Then, we perform decomposition to derive the scattering power components and compare them with conventional and state-of-the-art methods. The quantitative analysis supports the merit of the proposed method. [ABSTRACT FROM AUTHOR]

Purer contributions obtained from three diagonal elements of the coherency matrix are expected. This makes polarimetric synthetic aperture radar (PolSAR) easier to understand and to use in some applications, since the main information is aggregated into three mutually independent polarization channels. Therefore, we propose a method to decouple different polarization channels (DDPCs) in this letter. In such a case, all the nondiagonal elements of the coherency matrix are turned to zero, and three uncorrelated polarization channels are obtained. For this purpose, two rotation angles with physical meaning are proposed. They are responsible for rotating $T_{12}$ to zero. Radarsat-2 data are used to validate the proposed method. The results show that the proposed rotation angles are effective for land-cover classification. Furthermore, three polarization channels obtained from the DDPC method can mitigate the overestimation of volume scattering power for oriented buildings. [ABSTRACT FROM AUTHOR]

Conflating/stitching 2.5-D raster digital surface models (DSMs) into a large one has been a running practice in geoscience applications; however, conflating full-3-D mesh models, such as those from oblique photogrammetry, is extremely challenging. In this letter, we propose a novel approach to address this challenge by conflating multiple full-3-D oblique photogrammetric models into a single and seamless mesh for high-resolution site modeling. Given two or more individually collected and created photogrammetric meshes, we first propose to create a virtual camera field [with a panoramic field of view (FoV)] to incubate virtual spaces represented by truncated signed distance field (TSDF), an implicit volumetric field friendly for linear 3-D fusion; then, we adaptively leverage the truncated bound of meshes in TSDF to conflate them into a single and accurate full 3-D site model. With drone-based 3-D meshes, we show that our approach significantly improves upon traditional methods for model conflations, to drive new potentials to create excessively large and accurate full 3-D mesh models in support of geoscience and environmental applications. [ABSTRACT FROM AUTHOR]

For the predominant classification performance, the convolutional neural network (CNN) is becoming quite popular in the hyperspectral image (HSI) classification. However, quite a few parameters must be updated in the training procedure. The overfitting problem is also exacerbated by HSI’s high dimensionality and limited training samples. Therefore, an end-to-end model DWTCNN that deeply integrates 3-D discrete wavelet transform (DWT) and CNN is proposed. The 3-D DWT is introduced for the intrinsic feature collection in multiresolution, and the 3-D DWT modulated kernel with predefined parameters will not increase the trainable parameters of DWTCNN. The impact of 3D-DWT at different scale levels on classification performance is also studied. The amalgamation of 3-D DWT reduces the feature extraction burden of 3-D CNN while increasing the feature extraction capacity of the network. With a compact and light structure, our model is easier to train with less memory and limited training samples. Compared with other methods, our model has a considerable advantage in computational efficiency while maintaining good classification accuracy and clear physical meaning interpretability. Experimental results on three public datasets also demonstrate the superiority of our approach in the training process and classification performance. [ABSTRACT FROM AUTHOR]

This article establishes the actual movement mathematical model of CNC machine tools for the precision processing of jewelry. Through analyzing the general geometric error analysis model of CNC machine tools with less than five axes and the method of solving precision CNC instructions, the operating principle of the CNC machine tools is studied. At the same time, we use a transformation matrix to express the relationship between the various moving bodies. The article abstracts the complex motion relationship between entities as the relationship between mathematical matrices. The experimental results show that the theoretical method proposed in this paper can increase the machining accuracy of the machine tool by more than 50%.

Wire and arc additive manufacturing (WAAM) is a promising method for directly manufacturing parts with complex shapes. However, the accuracy of the existing welding parameter planning methods would dramatically decrease when the bead geometry changes dynamically due to the long-term dependence, strong coupling, and hysteresis properties of the WAAM process. To this end, a nonautoregressive dynamic model is proposed to predict the bead geometry, and an adaptive model predictive control (aMPC) method is proposed to plan welding parameters to achieve high manufacturing accuracy. First, in the proposed dynamic model, the long-term dynamic characteristics of the WAAM process are modeled by a resample long short-term memory network by considering the fluidity of the welding pool, which is the crucial factor of dynamic characteristics of the welding process. Second, in the proposed aMPC method, the strong coupling is addressed by high multiobjective performance, and the hysteresis is considered by a long control horizon. Thus, the aMPC method can reduce the control latency and improve the manufacturing accuracy for varying geometry beads. The proposed methods are validated by experiments, which indicate that the proposed methods are effective. [ABSTRACT FROM AUTHOR]

Manufacturing companies are currently undergoing a digitalization transformation in which digitally enabled, new, and innovative advanced service offerings are being launched. These so-called “digital services” represent a shift in the business logic of manufacturing firms, from up-front product sales to advanced service contracts. This business model shift has profound implications for cost structures, risk management, and revenue streams, providing manufacturing companies with the key challenge of rethinking how to capture value. Using a multiple case study of 11 companies, the purpose of this article is to enhance knowledge on how to design new revenue models for digital services. Results reveal a revenue model design framework of key phases and activities that carries implications for the emerging literature on digital servitization, as well as the business model innovation literature. The findings reveal a highly customer-centric, iterative, and agile process where close collaboration with key customers during the early stages guides the framing of revenue models for digital services. For practitioners, it provides hands-on advice on how to implement the design, development, and scaling processes for revenue models in the context of new digital services. [ABSTRACT FROM AUTHOR]

Recent studies of video action recognition can be classified into two categories: the appearance-based methods and the pose-based methods. The appearance-based methods generally cannot model temporal dynamics of large motion well by virtue of optical flow estimation, while the pose-based methods ignore the visual context information such as typical scenes and objects, which are also important cues for action understanding. In this paper, we tackle these problems by proposing a Pose-Appearance Relational Network (PARNet), which models the correlation between human pose and image appearance, and combines the benefits of these two modalities to improve the robustness towards unconstrained real-world videos. There are three network streams in our model, namely pose stream, appearance stream and relation stream. For the pose stream, a Temporal Multi-Pose RNN module is constructed to obtain the dynamic representations through temporal modeling of 2D poses. For the appearance stream, a Spatial Appearance CNN module is employed to extract the global appearance representation of the video sequence. For the relation stream, a Pose-Aware RNN module is built to connect pose and appearance streams by modeling action-sensitive visual context information. Through jointly optimizing the three modules, PARNet achieves superior performances compared with the state-of-the-arts on both the pose-complete datasets (KTH, Penn-Action, UCF11) and the challenging pose-incomplete datasets (UCF101, HMDB51, JHMDB), demonstrating its robustness towards complex environments and noisy skeletons. Its effectiveness on NTU-RGBD dataset is also validated even compared with 3D skeleton-based methods. Furthermore, an appearance-enhanced PARNet equipped with a RGB-based I3D stream is proposed, which outperforms the Kinetics pre-trained competitors on UCF101 and HMDB51. The better experimental results verify the potentials of our framework by integrating various modules. [ABSTRACT FROM AUTHOR]

Deep learning (DL) models have been successfully applied in video-based affective computing, allowing, for instance, to recognize emotions and mood, or to estimate the intensity of pain or stress of individuals based on their facial expressions. Despite the recent advances with state-of-the-art DL models for spatio-temporal recognition of facial expressions associated with depressive behaviour, some key challenges remain in the cost-effective application of 3D-CNNs: (1) 3D convolutions usually employ structures with fixed temporal depth that decreases the potential to extract discriminative representations due to the usually small difference of spatio-temporal variations along different depression levels; and (2) the computational complexity of these models with consequent susceptibility to overfitting. To address these challenges, we propose a novel DL architecture called the Maximization and Differentiation Network (MDN) in order to effectively represent facial expression variations that are relevant for depression assessment. The MDN, operating without 3D convolutions, explores multiscale temporal information using a maximization block that captures smooth facial variations and a difference block that encodes sudden facial variations. Extensive experiments using our proposed MDN with models with 100 and 152 layers result in improved performance while reducing the number of parameters by more than $3\times$ 3 × when compared with 3D ResNet models. Our model also outperforms other 3D models and achieves state-of-the-art results for depression detection. Code available at: https://github.com/wheidima/MDN. [ABSTRACT FROM AUTHOR]

In this article, a new dataset, the driver emotion facial expression (DEFE) dataset for drivers’ spontaneous emotions analysis is introduced. The dataset includes facial expression recordings from 60 participants during driving. After watching a selected video-audio clip to elicit a specific emotion, each participant completed the driving tasks in the same driving scenario and rated his/her emotional responses during the driving processes from the aspects of dimensional emotion method and discrete emotion method. The study also conducted classification experiments to recognize the scales of arousal, valence, dominance, as well as the emotion category and intensity to establish baseline results for the proposed dataset. Furthermore, this paper compared emotion recognition results difference through facial expressions between dynamic driving and static life scenarios. The results showed that dynamic driving and static life datasets were different in emotion recognition results. To further explore the reasons for the difference in emotion recognition results, the analysis from the AU (action unit) presence perspective was studied. The results showed significant differences in the AUs presence of facial expressions between dynamic driving and static life scenarios, indicating that drivers’ facial expressions may be affected by the driving task to influence the recognition of drivers’ emotions through facial expressions. Therefore, to accurately recognize the drivers’ emotions to establish a reliable emotion-aware human-machine interaction system, thereby improving driving safety and comfort, publishing a human emotion dataset specifically for the driver is necessary. The proposed dataset will be publicly available so that researchers worldwide can use it to develop and examine their driver emotion analysis methods. To the best of our knowledge, this is currently the only public driver facial expression dataset. [ABSTRACT FROM AUTHOR]

The power density in modern Integrated Circuits (ICs) is tremendous. For example, Multi-Processor Systems-on-Chip (MPSoCs) nowadays undergo temperature swings of 40 degrees in 100 milliseconds or less, with rapidly emerging and vanishing sub-millimeter hot spots. As such, not only a simulation-based cooling assessment is vital, but one has to simulate the on-chip thermal phenomena jointly with the heat dissipation system - historically, a challenge. In recent years, however, the idea of coupling traditional 3D chip simulators with heat dissipation models written in Equation-Based Modeling (EBM) languages has proven to be a game changer. EBM languages allow one to compose a model by assembling components described in terms of Differential and Algebraic Equations (DAE) and have the simulation code generated automatically. In this article, we take a tutorial viewpoint on the matter just sketched, to put the reader in the position of exploiting the above technology. We also present the first nucleus of a model library for cooling systems, that we release as free software for the scientific and engineering community. [ABSTRACT FROM AUTHOR]

Tissue examination by hand remains an essential technique in clinical practice. The effective application depends on skills in sensorimotor coordination, mainly involving haptic, visual, and auditory feedback. The skills clinicians have to learn can be as subtle as regulating finger pressure with breathing, choosing palpation action, monitoring involuntary facial and vocal expressions in response to palpation, and using pain expressions both as a source of information and as a constraint on physical examination. Patient simulators can provide a safe learning platform to novice physicians before trying real patients. This paper reviews state-of-the-art medical simulators for the training for the first time with a consideration of providing multimodal feedback to learn as many manual examination techniques as possible. The study summarizes current advances in tissue examination training devices simulating different medical conditions and providing different types of feedback modalities. Opportunities with the development of pain expression, tissue modeling, actuation, and sensing are also analyzed to support the future design of effective tissue examination simulators. [ABSTRACT FROM AUTHOR]

Cardiovascular disease is a deadly global health crisis that carries a substantial financial burden. Innovative treatment and management of cardiovascular disease straddles medicine, personalized hemodynamic modeling, machine learning, and modern imaging to help improve patient outcomes and reduce the economic impact. Hemodynamic modeling offers a non-invasive method to provide clinicians with new pre- and post- procedural metrics and aid in the selection of treatment options. Medical imaging is an integral part in clinical workflows for understanding and managing cardiac disease and interventions. Coupling machine learning with modeling, and cardiovascular imaging, provides faster modeling, improved data fidelity, and an enhanced understanding and earlier detection of cardiovascular anomalies, leading to the development of patient-specific diagnostic and predictive tools for characterizing and assessing cardiovascular outcomes. Herein, we provide a scoping review of translational hemodynamic modeling, medical imaging, and machine learning and their applications to cardiovascular interventions. We particularly focus on providing an intuitive understanding of each of these approaches and their ability to support decision making during important clinical milestones. [ABSTRACT FROM AUTHOR]

Power system digital twins (PSDTs) are foreseen as an essential step toward future grids that will positively affect network monitoring, operation, and planning across the whole power industry. Existing frameworks for developing a DT are bounded to particular power system components, applications and/or users. This article proposes a modular framework for the implementation of PSDTs that is flexible, robust, and cost-effective. The modularity in design enables the expansion of the twin beyond power system components and facilitates the integration of multiple services and users, without affecting the functionality of existing modules. Each module can be independently built, modified, replaced, or exchanged with other modules, unlocking multiple, more advanced, dedicated, and multidomain applications for the operation and planning of power systems. The first step toward a PSDT is illustrated based on a real-time compatible model of the Australian National Electricity Market. The model can be seen as the core of one or more distinct modules in the DT, which can be adapted and scaled for particular applications. The EMT real-time compatible model is employed to demonstrate four potential services, including renewable energy integration and “what-if” scenario cases, which are expected applications in forward-looking PSDTs. [ABSTRACT FROM AUTHOR]

Most realtime human pose estimation approaches are based on detecting joint positions. Using the detected joint positions, the yaw and pitch of the limbs can be computed. However, the roll along the limb, which is critical for application such as sports analysis and computer animation, cannot be computed as this axis of rotation remains unobserved. In this paper we therefore introduce orientation keypoints, a novel approach for estimating the full position and rotation of skeletal joints, using only single-frame RGB images. Inspired by how motion-capture systems use a set of point markers to estimate full bone rotations, our method uses virtual markers to generate sufficient information to accurately infer rotations with simple post processing. The rotation predictions improve upon the best reported mean error for joint angles by 48% and achieves 93% accuracy across 15 bone rotations. The method also improves the current state-of-the-art results for joint positions by 14% as measured by MPJPE on the principle dataset, and generalizes well to in-the-wild datasets. [ABSTRACT FROM AUTHOR]

Monocular 3D object detection is an important task in autonomous driving. It can be easily intractable where there exists ego-car pose change w.r.t. ground plane. This is common due to the slight fluctuation of road smoothness and slope. Due to the lack of insight in industrial application, existing methods on open datasets neglect the camera pose information, which inevitably results in the detector being susceptible to camera extrinsic parameters. The perturbation of objects is very popular in most autonomous driving cases for industrial products. To this end, we propose a novel method to capture camera pose to formulate the detector free from extrinsic perturbation. Specifically, the proposed framework predicts camera extrinsic parameters by detecting vanishing point and horizon change. A converter is designed to rectify perturbative features in the latent space. By doing so, our 3D detector works independent of the extrinsic parameter variations and produces accurate results in realistic cases, e.g., potholed and uneven roads, where almost all existing monocular detectors fail to handle. Experiments demonstrate our method yields the best performance compared with the other state-of-the-arts by a large margin on both KITTI 3D and nuScenes datasets. [ABSTRACT FROM AUTHOR]

In this work, we propose a novel method for generating 3D point clouds that leverages the properties of hypernetworks. Contrary to the existing methods that learn only the representation of a 3D object, our approach simultaneously finds a representation of the object and its 3D surface. The main idea of our HyperCloud method is to build a hypernetwork that returns weights of a particular neural network (target network) trained to map points from prior distribution into a 3D shape. As a consequence, a particular 3D shape can be generated using point-by-point sampling from the prior distribution and transforming the sampled points with the target network. Since the hypernetwork is based on an auto-encoder architecture trained to reconstruct realistic 3D shapes, the target network weights can be considered to be a parametrization of the surface of a 3D shape, and not a standard representation of point cloud usually returned by competitive approaches. We also show that relying on hypernetworks to build 3D point cloud representations offers an elegant and flexible framework. To that point, we further extend our method by incorporating flow-based models, which results in a novel HyperFlow approach. [ABSTRACT FROM AUTHOR]

Point cloud analysis without pose priors is very challenging in real applications, as the orientations of point clouds are often unknown. In this paper, we propose a brand new point-set learning framework PRIN, namely, Point-wise Rotation Invariant Network, focusing on rotation invariant feature extraction in point clouds analysis. We construct spherical signals by Density Aware Adaptive Sampling to deal with distorted point distributions in spherical space. Spherical Voxel Convolution and Point Re-sampling are proposed to extract rotation invariant features for each point. In addition, we extend PRIN to a sparse version called SPRIN, which directly operates on sparse point clouds. Both PRIN and SPRIN can be applied to tasks ranging from object classification, part segmentation, to 3D feature matching and label alignment. Results show that, on the dataset with randomly rotated point clouds, SPRIN demonstrates better performance than state-of-the-art methods without any data augmentation. We also provide thorough theoretical proof and analysis for point-wise rotation invariance achieved by our methods. The code to reproduce our results will be made publicly available. [ABSTRACT FROM AUTHOR]

In this work, we reported a theoretical model for a novel piezoelectric micromachined ultrasonic transducer (PMUT). The sensing cells with hexagonal transduction diaphragms in the PMUT array are designed and arranged in a bioinspired honeycomb structure. Based on the classical piece-wise approach commonly used for PMUT with circular diaphragm, an enhanced theoretical model is developed. In this model, the deflection of the honeycomb diaphragm is analytically determined by the volume deflection, rather than the path deflection commonly used in the previous models. The hexagonal diaphragm is converted as a circular one, whose radius is derived by correcting the characteristic size of the hexagonal diaphragm multiplied by a correction factor. The calculated parameters by the reported theoretical model of a transduction cell in the PMUT array, such as resonant frequency and static deflection, matched well with the data obtained by finite element analysis and experiment. Special investigations are made on the effects of top electrode coverage and transduction layer thickness towards PMUT’s performance. The reported theoretical model in this work is suitable not only for PMUTs with circular and hexagonal transduction cells, but also for square and other arbitrary shape diaphragms. This enhanced theoretical model provides an interesting insight on design optimizations of PMUT with various transduction cell geometries. [2022-0093] [ABSTRACT FROM AUTHOR]

We propose a novel deep learning method for shadow removal. Inspired by physical models of shadow formation, we use a linear illumination transformation to model the shadow effects in the image that allows the shadow image to be expressed as a combination of the shadow-free image, the shadow parameters, and a matte layer. We use two deep networks, namely SP-Net and M-Net, to predict the shadow parameters and the shadow matte respectively. This system allows us to remove the shadow effects from images. We then employ an inpainting network, I-Net, to further refine the results. We train and test our framework on the most challenging shadow removal dataset (ISTD). Our method improves the state-of-the-art in terms of mean absolute error (MAE) for the shadow area by 20%. Furthermore, this decomposition allows us to formulate a patch-based weakly-supervised shadow removal method. This model can be trained without any shadow- free images (that are cumbersome to acquire) and achieves competitive shadow removal results compared to state-of-the-art methods that are trained with fully paired shadow and shadow-free images. Last, we introduce SBU-Timelapse, a video shadow removal dataset for evaluating shadow removal methods. [ABSTRACT FROM AUTHOR]