Search

This paper provides insights into deep reinforcement learning (DRL) for process control from the perspective of transfer learning. We analyze the challenges of applying DRL in the field of process industries and the necessity of introducing transfer learning. Furthermore, recommendations and prospects are provided for future research directions on how transfer learning can be integrated with DRL to empower process control., Comment: Final Version of Asian Control Conference (ASCC 2024)

With the rise of deep learning, there has been renewed interest within the process industries to utilize data on large-scale nonlinear sensing and control problems. We identify key statistical and machine learning techniques that have seen practical success in the process industries. To do so, we start with hybrid modeling to provide a methodological framework underlying core application areas: soft sensing, process optimization, and control. Soft sensing contains a wealth of industrial applications of statistical and machine learning methods. We quantitatively identify research trends, allowing insight into the most successful techniques in practice. We consider two distinct flavors for data-driven optimization and control: hybrid modeling in conjunction with mathematical programming techniques and reinforcement learning. Throughout these application areas, we discuss their respective industrial requirements and challenges. A common challenge is the interpretability and efficiency of purely data-driven methods. This suggests a need to carefully balance deep learning techniques with domain knowledge. As a result, we highlight ways prior knowledge may be integrated into industrial machine learning applications. The treatment of methods, problems, and applications presented here is poised to inform and inspire practitioners and researchers to develop impactful data-driven sensing, optimization, and control solutions in the process industries., Comment: 48 pages

The row-column addressed array is an emerging probe for ultrafast 3-D ultrasound imaging. It achieves this with far fewer independent electronic channels and a wider field of view than traditional 2-D matrix arrays, of the same channel count, making it a good candidate for clinical translation. However, the image quality of row-column arrays is generally poor, particularly when investigating tissue. Ultrasound localisation microscopy allows for the production of super-resolution images even when the initial image resolution is not high. Unfortunately, the row-column probe can suffer from imaging artefacts that can degrade the quality of super-resolution images as `secondary' lobes from bright microbubbles can be mistaken as microbubble events, particularly when operated using plane wave imaging. These false events move through the image in a physiologically realistic way so can be challenging to remove via tracking, leading to the production of 'false vessels'. Here, a new type of rolling window image reconstruction procedure was developed, which integrated a row-column array-specific coherence-based beamforming technique with acoustic sub-aperture processing for the purposes of reducing `secondary' lobe artefacts, noise and increasing the effective frame rate. Using an {\it{in vitro}} cross tube, it was found that the procedure reduced the percentage of `false' locations from $\sim$26\% to $\sim$15\% compared to traditional orthogonal plane wave compounding. Additionally, it was found that the noise could be reduced by $\sim$7 dB and that the effective frame rate could be increased to over 4000 fps. Subsequently, {\it{in vivo}} ultrasound localisation microscopy was used to produce images non-invasively of a rabbit kidney and a human thyroid.

As a useful and efficient alternative to generic model-based control scheme, data-driven predictive control is subject to bias-variance trade-off and is known to not perform desirably in face of uncertainty. Through the connection between direct data-driven control and subspace predictive control, we gain insight into the reason being the lack of causality as a main cause for high variance of implicit prediction. In this article, we seek to address this deficiency by devising a novel causality-informed formulation of direct data-driven control. Built upon LQ factorization, an equivalent two-stage reformulation of regularized data-driven control is first derived, which bears clearer interpretability and a lower complexity than generic forms. This paves the way for deriving a two-stage causality-informed formulation of data-driven predictive control, as well as a regularized form that balances between control cost minimization and implicit identification of multi-step predictor. Since it only calls for block-triangularization of a submatrix in LQ factorization, the new causality-informed formulation comes at no excess cost as compared to generic ones. Its efficacy is investigated based on numerical examples and application to model-free control of a simulated industrial heating furnace. Empirical results corroborate that the proposed method yields obvious performance improvement over existing formulations in handling stochastic noise and process nonlinearity.

The Organic Rankine Cycle (ORC) is widely used in industrial waste heat recovery due to its simple structure and easy maintenance. However, in the context of smart manufacturing in the process industry, traditional model-based optimization control methods are unable to adapt to the varying operating conditions of the ORC system or sudden changes in operating modes. Deep reinforcement learning (DRL) has significant advantages in situations with uncertainty as it directly achieves control objectives by interacting with the environment without requiring an explicit model of the controlled plant. Nevertheless, direct application of DRL to physical ORC systems presents unacceptable safety risks, and its generalization performance under model-plant mismatch is insufficient to support ORC control requirements. Therefore, this paper proposes a Sim2Real transfer learning-based DRL control method for ORC superheat control, which aims to provide a new simple, feasible, and user-friendly solution for energy system optimization control. Experimental results show that the proposed method greatly improves the training speed of DRL in ORC control problems and solves the generalization performance issue of the agent under multiple operating conditions through Sim2Real transfer.

We propose a robust nonlinear model predictive control design with generalized zone tracking (ZMPC) in this work. The proposed ZMPC has guaranteed convergence into the target zone in the presence of bounded disturbance. The proposed approach achieves this by modifying the actual target zone such that the effect of disturbances is rejected. A control invariant set (CIS) inside the modified target zone is used as the terminal set, which ensures the closed-loop stability of the proposed controller. Detailed closed-loop stability analysis is presented. Simulation studies based on a continuous stirred tank reactor (CSTR) are performed to validate the effectiveness of the proposed ZMPC.

Minimum miscibility pressure (MMP) prediction plays an important role in design and operation of nitrogen based enhanced oil recovery processes. In this work, a comparative study of statistical and machine learning methods used for MMP estimation is carried out. Most of the predictive models developed in this study exhibited superior performance over correlation and predictive models reported in literature., Comment: 42 pages, 24 figures

Super-resolution ultrasound can image microvascular structure and flow at sub-wave-diffraction resolution based on localising and tracking microbubbles. Currently, tracking microbubbles accurately under limited imaging frame rates and high microbubble concentrations remains a challenge, especially under the effect of cardiac pulsatility and in highly curved vessels. In this study, an acceleration-incorporated microbubble motion model is introduced into a Kalman tracking framework. The tracking performance was evaluated using simulated microvasculature with different microbubble motion parameters and acquisition frame rates, and in vivo human breast tumour ultrasound datasets. The simulation results show that the acceleration-based method outperformed the non-acceleration-based method at different levels of acceleration and acquisition frame rates and achieved significant improvement in true positive rate (up to 10.03%), false negative rate (up to 28.61%) and correctly pairing fraction (up to 170.14%). The proposed method can also reduce errors in vasculature reconstruction via the acceleration-based nonlinear interpolation, compared with linear interpolation (up to 19 um). The tracking results from temporally downsampled low frame rate in vivo datasets from human breast tumours show that the proposed method has better microbubble tracking performance than the baseline method, if using results from the initial high frame data as reference. Finally, the acceleration estimated from tracking results also provides a spatial speed gradient map that may contain extra valuable diagnostic information., Comment: 15 pages, 10 figures

Micro-vascular flow in the myocardium is of significant importance clinically but remains poorly understood. Up to 25% of patients with symptoms of coronary heart diseases have no obstructive coronary arteries and have suspected microvascular diseases. However, such microvasculature is difficult to image in vivo with existing modalities due to the lack of resolution and sensitivity. Here, we demonstrate the feasibility of transthoracic super-resolution ultrasound localisation microscopy (SRUS/ULM) of myocardial microvasculature and hemodynamics in a large animal model and in patients, using a cardiac phased array probe with a customised data acquisition and processing pipeline. A multi-level motion correction strategy was proposed. A tracking framework incorporating multiple features and automatic parameter initialisations was developed to reconstruct microcirculation. In two patients with impaired myocardial function, we have generated SRUS images of myocardial vascular structure and flow with a resolution that is beyond the wave-diffraction limit (half a wavelength), using data acquired within a breath hold. Myocardial SRUS/ULM has potential to improve the understanding of myocardial microcirculation and the management of patients with cardiac microvascular diseases., Comment: 22 pages, 10 figures

This paper presents a distributed data-driven predictive control (DDPC) approach using the behavioral framework. It aims to design a network of controllers for an interconnected system with linear time-invariant (LTI) subsystems such that a given global (network-wide) cost function is minimized while desired control performance (e.g., network stability and disturbance rejection) is achieved using dissipativity in the quadratic difference form (QdF). By viewing dissipativity as a behavior and integrating it into the control design as a virtual dynamical system, the proposed approach carries out the entire design process in a unified framework with a set-theoretic viewpoint. This leads to an effective data-driven distributed control design, where the global design goal can be achieved by distributed optimization based on the local QdF conditions. The approach is illustrated by an example throughout the paper.

We reconstruct the spectral pairing (SP) theories to enable analytical descriptions of eigenstate spatiotemporal orders in translation-invariant systems without prethermal conditions. It is shown that the strong Ising interactions and drivings alone stabilize a class of ``cat scar" eigenstates with tunable patterns, which lead to {\em local} discrete time crystal (DTC) dynamics. They exhibit Fock space localization and long-range correlations robust against generic perturbations in a disorder-free scenario. In particular, we introduce a symmetry indicator method to enumerate cat scars, with which a set of unexpected inhomogeneous scar patterns are identified in addition to the ferromagnetic scars found before. These scars enforce DTC dynamics with rigid inhomogeneous patterns, offering a viable way to verify underlying eigenstate properties experimentally. Further, we prove rigorously that the strong Ising interactions enforce a selection rule for perturbations of different orders, which imposes an exponential suppression of spin fluctuations for Floquet eigenstates. Based on this property, three analytical scaling relations are proved to characterize the amplitudes, Fock space localization, and lifetime for DTC dynamics associated with cat scars. We further provide two practical methods to check whether certain DTC phenomena are dominated by single-spin dynamics or due to genuine interaction effects., Comment: 29 pages, 18 figures, major updates that include more detailed discussions of Floquet resonance and comparisons with other DTC-type systems

Over the last ten years, we have seen a significant increase in industrial data, tremendous improvement in computational power, and major theoretical advances in machine learning. This opens up an opportunity to use modern machine learning tools on large-scale nonlinear monitoring and control problems. This article provides a survey of recent results with applications in the process industry., Comment: IFAC World Congress 2020

As a significant ingredient regarding health status, data-driven state-of-health (SOH) estimation has become dominant for lithium-ion batteries (LiBs). To handle data discrepancy across batteries, current SOH estimation models engage in transfer learning (TL), which reserves apriori knowledge gained through reusing partial structures of the offline trained model. However, multiple degradation patterns of a complete life cycle of a battery make it challenging to pursue TL. The concept of the stage is introduced to describe the collection of continuous cycles that present a similar degradation pattern. A transferable multi-stage SOH estimation model is proposed to perform TL across batteries in the same stage, consisting of four steps. First, with identified stage information, raw cycling data from the source battery are reconstructed into the phase space with high dimensions, exploring hidden dynamics with limited sensors. Next, domain invariant representation across cycles in each stage is proposed through cycling discrepancy subspace with reconstructed data. Third, considering the unbalanced discharge cycles among different stages, a switching estimation strategy composed of a lightweight model with the long short-term memory network and a powerful model with the proposed temporal capsule network is proposed to boost estimation accuracy. Lastly, an updating scheme compensates for estimation errors when the cycling consistency of target batteries drifts. The proposed method outperforms its competitive algorithms in various transfer tasks for a run-to-failure benchmark with three batteries., Comment: This paper has been accepted for publication in IEEE Transactions on Industrial Informatics

Brain tumor segmentation (BTS) in magnetic resonance image (MRI) is crucial for brain tumor diagnosis, cancer management and research purposes. With the great success of the ten-year BraTS challenges as well as the advances of CNN and Transformer algorithms, a lot of outstanding BTS models have been proposed to tackle the difficulties of BTS in different technical aspects. However, existing studies hardly consider how to fuse the multi-modality images in a reasonable manner. In this paper, we leverage the clinical knowledge of how radiologists diagnose brain tumors from multiple MRI modalities and propose a clinical knowledge-driven brain tumor segmentation model, called CKD-TransBTS. Instead of directly concatenating all the modalities, we re-organize the input modalities by separating them into two groups according to the imaging principle of MRI. A dual-branch hybrid encoder with the proposed modality-correlated cross-attention block (MCCA) is designed to extract the multi-modality image features. The proposed model inherits the strengths from both Transformer and CNN with the local feature representation ability for precise lesion boundaries and long-range feature extraction for 3D volumetric images. To bridge the gap between Transformer and CNN features, we propose a Trans&CNN Feature Calibration block (TCFC) in the decoder. We compare the proposed model with five CNN-based models and six transformer-based models on the BraTS 2021 challenge dataset. Extensive experiments demonstrate that the proposed model achieves state-of-the-art brain tumor segmentation performance compared with all the competitors.

Many multivariate statistical analysis methods and their corresponding probabilistic counterparts have been adopted to develop process monitoring models in recent decades. However, the insightful connections between them have rarely been studied. In this study, a generalized probabilistic monitoring model (GPMM) is developed with both random and sequential data. Since GPMM can be reduced to various probabilistic linear models under specific restrictions, it is adopted to analyze the connections between different monitoring methods. Using expectation maximization (EM) algorithm, the parameters of GPMM are estimated for both random and sequential cases. Based on the obtained model parameters, statistics are designed for monitoring different aspects of the process system. Besides, the distributions of these statistics are rigorously derived and proved, so that the control limits can be calculated accordingly. After that, contribution analysis methods are presented for identifying faulty variables once the process anomalies are detected. Finally, the equivalence between monitoring models based on classical multivariate methods and their corresponding probabilistic graphic models is further investigated. The conclusions of this study are verified using a numerical example and the Tennessee Eastman (TE) process. Experimental results illustrate that the proposed monitoring statistics are subject to their corresponding distributions, and they are equivalent to statistics in classical deterministic models under specific restrictions., Comment: 12 pages, 4 figures, 3 tables

We analytically identify a new class of quantum scars protected by spatiotemporal translation symmetries, dubbed Floquet-Bloch scars. They distinguish from previous (quasi-)static scars by a rigid spectral pairing only possible in Floquet systems, where strong interaction and drivings equalize the quasienergy corrections to all scars and maintain their spectral spacings against generic bilinear perturbations. Scars then enforce the spatial localization and rigid discrete time crystal (DTC) oscillations as verified numerically in a trimerized kagome lattice model relevant to recent cold atom experiments. Our analytical solutions offer a potential scheme to understand the mechanisms for more generic translation-invariant DTCs., Comment: 6+11 pages

Water scarcity is an urgent issue to be resolved and improving irrigation water-use efficiency through closed-loop control is essential. The complex agro-hydrological system dynamics, however, often pose challenges in closed-loop control applications. In this work, we propose a two-layer neural network (NN) framework to approximate the dynamics of the agro-hydrological system. To minimize the prediction error, a linear bias correction is added to the proposed model. The model is employed by a model predictive controller with zone tracking (ZMPC), which aims to keep the root zone soil moisture in the target zone while minimizing the total amount of irrigation. The performance of the proposed approximation model framework is shown to be better compared to a benchmark long-short-term-memory (LSTM) model for both open-loop and closed-loop applications. Significant computational cost reduction of the ZMPC is achieved with the proposed framework. To handle the tracking offset caused by the plant-model-mismatch of the proposed NN framework, a shrinking target zone is proposed for the ZMPC. Different hyper-parameters of the shrinking zone in the presence of noise and weather disturbances are investigated, of which the control performance is compared to a ZMPC with a time-invariant target zone.