Search

This study introduces a parameter-efficient Hierarchical Spatial Temporal Network (HiSTN) specifically designed for the task of emotion classification using multi-channel electroencephalogram data. The network incorporates a graph hierarchy constructed from bottom-up at various abstraction levels, offering the dual advantages of enhanced task-relevant deep feature extraction and a lightweight design. The model's effectiveness is further amplified when used in conjunction with a proposed unique label smoothing method. Comprehensive benchmark experiments reveal that this combined approach yields high, balanced performance in terms of both quantitative and qualitative predictions. HiSTN, which has approximately 1,000 parameters, achieves mean F1 scores of 96.82% (valence) and 95.62% (arousal) in subject-dependent tests on the rarely-utilized 5-classification task problem from the DREAMER dataset. In the subject-independent settings, the same model yields mean F1 scores of 78.34% for valence and 81.59% for arousal. The adoption of the Sequential Top-2 Hit Rate (Seq2HR) metric highlights the significant enhancements in terms of the balance between model's quantitative and qualitative for predictions achieved through our approach when compared to training with regular one-hot labels. These improvements surpass 50% in subject-dependent tasks and 30% in subject-independent tasks. The study also includes relevant ablation studies and case explorations to further elucidate the workings of the proposed model and enhance its interpretability., Comment: Draft

In this work, a kernel attention module is presented for the task of EEG-based emotion classification with neural networks. The proposed module utilizes a self-attention mechanism by performing a kernel trick, demanding significantly fewer trainable parameters and computations than standard attention modules. The design also provides a scalar for quantitatively examining the amount of attention assigned during deep feature refinement, hence help better interpret a trained model. Using EEGNet as the backbone model, extensive experiments are conducted on the SEED dataset to assess the module's performance on within-subject classification tasks compared to other SOTA attention modules. Requiring only one extra parameter, the inserted module is shown to boost the base model's mean prediction accuracy up to more than 1\% across 15 subjects. A key component of the method is the interpretability of solutions, which is addressed using several different techniques, and is included throughout as part of the dependency analysis., Comment: This preprint has not undergone peer review. The updated version is accepted by MICCAI2022 workshop: iMIMIC - Interpretability of Machine Intelligence in Medical Image Computing

In this work, a parameter-efficient attention module is presented for emotion classification using a limited, or relatively small, number of electroencephalogram (EEG) signals. This module is called the Monotonicity Constrained Attention Module (MCAM) due to its capability of incorporating priors on the monotonicity when converting features' Gram matrices into attention matrices for better feature refinement. Our experiments have shown that MCAM's effectiveness is comparable to state-of-the-art attention modules in boosting the backbone network's performance in prediction while requiring less parameters. Several accompanying sensitivity analyses on trained models' prediction concerning different attacks are also performed. These attacks include various frequency domain filtering levels and gradually morphing between samples associated with multiple labels. Our results can help better understand different modules' behaviour in prediction and can provide guidance in applications where data is limited and are with noises., Comment: A Preprint for the accepted work by MICCAI 2022 workshop: Medical Image Learning with Noisy and Limited Data

This paper reports on the development of reduced models for electron temperature gradient (ETG) driven transport in the pedestal. Model development is enabled by a set of 61 nonlinear gyrokinetic simulations with input parameters taken from the pedestals in a broad range of experimental scenarios. The simulation data has been consolidated in a new database for gyrokinetic simulation data, the Multiscale Gyrokinetic Database (MGKDB), facilitating the analysis. The modeling approach may be considered a generalization of the standard quasilinear mixing length procedure. The parameter eta, the ratio of the density to temperature gradient scale length, emerges as the key parameter for formulating an effective saturation rule. With a single order-unity fitting coefficient, the model achieves an RMS error of 15%. A similar model for ETG particle flux is also described. We also present simple algebraic expressions for the transport informed by an algorithm for symbolic regression.

Image registration is a fundamental step in medical image analysis. Ideally, the transformation that registers one image to another should be a diffeomorphism that is both invertible and smooth. Traditional methods like geodesic shooting approach the problem via differential geometry, with theoretical guarantees that the resulting transformation will be smooth and invertible. Most previous research using unsupervised deep neural networks for registration have used a local smoothness constraint (typically, a spatial variation loss) to address the smoothness issue. These networks usually produce non-invertible transformations with ``folding'' in multiple voxel locations, indicated by a negative determinant of the Jacobian matrix of the transformation. While using a loss function that specifically penalizes the folding is a straightforward solution, this usually requires carefully tuning the regularization strength, especially when there are also other losses. In this paper we address this problem from a different angle, by investigating possible training mechanisms that will help the network avoid negative Jacobians and produce smoother deformations. We contribute two independent ideas in this direction. Both ideas greatly reduce the number of folding locations in the predicted deformation, without making changes to the hyperparameters or the architecture used in the existing baseline registration network.

In this paper, a 1d convolutional neural network is designed for classification tasks of plant leaves. This network based classifier is analyzed in two directions. In the forward direction, the proposed network can be used in two ways: a classifier and an automatic feature extractor. As a classifier, it takes the simple centroid contour distance curve as the single feature and achieves comparable performance with state-of-art methods that usually require multiple extracted features. As a feature extractor, it produces nearly linear separable features, hence can be used together with other classifiers such as support vector machines to provide better performance. The proposed network adopts simple 1d input and is generally applicable for other tasks such as classifying one dimensional time series in an end-to-end fashion without changes. Experiments on some benchmark datasets show this architecture can provide classification accuracies that are comparable or higher than some existing methods. In the backward direction, methods like gradient-weighted class activation mapping and maximum activation map of neurons in the classification layer with respect to inputs are performed to help investigate and further validate that hidden signatures helping trigger the trained classifier's specific decisions can be human interpretable. Code for the paper is available at https://github.com/dykuang/Leaf_Project.

We present a new unsupervised learning algorithm, "FAIM", for 3D medical image registration. With a different architecture than the popular "U-net", the network takes a pair of full image volumes and predicts the displacement fields needed to register source to target. Compared with "U-net" based registration networks such as VoxelMorph, FAIM has fewer trainable parameters but can achieve higher registration accuracy as judged by Dice score on region labels in the Mindboggle-101 dataset. Moreover, with the proposed penalty loss on negative Jacobian determinants, FAIM produces deformations with many fewer "foldings", i.e. regions of non-invertibility where the surface folds over itself. In our experiment, we varied the strength of this penalty and investigated changes in registration accuracy and non-invertibility in terms of number of "folding" locations. We found that FAIM is able to maintain both the advantages of higher accuracy and fewer "folding" locations over VoxelMorph, over a range of hyper-parameters (with the same values used for both networks). Further, when trading off registration accuracy for better invertibility, FAIM required less sacrifice of registration accuracy. Codes for this paper will be released upon publication.

We study a class of mathematical and statistical algorithms with the aim of establishing a computer-based framework for fast and reliable automatic abnormality detection on landmark represented image templates. Under this framework, we apply a landmark-based algorithm for finding a group average as an estimator that is said to best represent the common features of the group in study. This algorithm extracts information of momentum at each landmark through the process of template matching. If ever converges, the proposed algorithm produces a local coordinate system for each member of the observing group, in terms of the residual momentum. We use a Bayesian approach on the collected residual momentum representations for making inference. For illustration, we apply this framework to a small database of brain images for detecting structure abnormality. The brain structure changes identified by our framework are highly consistent with studies in the literature.

The Euler-Poincar\'e (EP) equations describe the geodesic motion on the diffeomorphism group. For template matching (template deformation), the Euler-Lagrangian equation, arising from minimizing an energy function, falls into the Euler-Poincar\'e theory and can be recast into the EP equations. By casting the EP equations in the Lagrangian (or characteristics) form, we formulate the equations as a finite dimensional particle system. The evolution of this particle system describes the geodesic motion of landmark points on a Riemann manifold. In this paper we present a class of novel algorithms that take advantage of the structure of the particle system to achieve a fast matching process between the reference and the target templates. The strong suit of the proposed algorithms includes (1) the efficient feedback control iteration, which allows one to find the initial velocity field for driving the deformation from the reference template to the target one, (2) the use of the conical kernel in the particle system, which limits the interaction between particles and thus accelerates the convergence, and (3) the availability of the implementation of fast-multipole method for solving the particle system, which could reduce the computational cost from $O(N^2)$ to $O(N\log N)$, where $N$ is the number of particles. The convergence properties of the proposed algorithms are analyzed. Finally, we present several examples for both exact and inexact matchings, and numerically analyze the iterative process to illustrate the efficiency and the robustness of the proposed algorithms.

We study a class of partial differential equations (PDEs) in the family of the so-called Euler-Poincar\'e differential systems, with the aim of developing a foundation for numerical algorithms of their solutions. This requires particular attention to the mathematical properties of this system when the associated class of elliptic operators possesses non-smooth kernels. By casting the system in its Lagrangian (or characteristics) form, we first formulate a particles system algorithm in free space with homogeneous Dirichlet boundary conditions for the evolving fields. We next examine the deformation of the system when non-homogeneous "constant stream" boundary conditions are assumed. We show how this simple change at the boundary deeply affects the nature of the evolution, from hyperbolic-like to dispersive with a non-trivial dispersion relation, and examine the potentially regularizing properties of singular kernels offered by this deformation. From the particle algorithm viewpoint, kernel singularities affect the existence and uniqueness of solutions to the corresponding ordinary differential equations systems. We illustrate this with the case when the operator kernel assumes a conical shape over the spatial variables, and examine in detail two-particle dynamics under the resulting lack of Lipschitz-continuity. Curiously, we find that for the conically-shaped kernels the motion of the related two-dimensional waves can become completely integrable under appropriate initial data. This reduction projects the two-dimensional system to the one-dimensional completely integrable Shallow-Water equation [Camassa, R. and Holm, D. D., Phys. Rev. Lett., 71, 1961-1964, 1993], while retaining the full dependence on two spatial dimensions for the single channel solutions.

We present a conservation formulation and a numerical algorithm for the reduced-gravity shallow-water equations on a beta plane, subjected to a constant wind forcing that leads to the formation of double-gyre circulation in a closed ocean basin. The novelty of the paper is that we reformulate the governing equations into a nonlinear hyperbolic conservation law plus source terms. A second-order fractional-step algorithm is used to solve the reformulated equations. In the first step of the fractional-step algorithm, we solve the homogeneous hyperbolic shallow-water equations by the wave-propagation finite volume method. The resulting intermediate solution is then used as the initial condition for the initial-boundary value problem in the second step. As a result, the proposed method is not sensitive to the choice of viscosity and gives high-resolution results for coarse grids, as long as the Rossby deformation radius is resolved. We discuss the boundary conditions in each step, when no-slip boundary conditions are imposed to the problem. We validate the algorithm by a periodic flow on an f-plane with exact solutions. The order-of-accuracy for the proposed algorithm is tested numerically. We illustrate a quasi-steady-state solution of the double-gyre model via the height anomaly and the contour of stream function for the formation of double-gyre circulation in a closed basin. Our calculations are highly consistent with the results reported in the literature. Finally, we present an application, in which the double-gyre model is coupled with the advection equation for modeling transport of a pollutant in a closed ocean basin.