Search

OBJECTIVE: This study evaluates regularization variants in logistic regression (L1, L2, ElasticNet, Adaptive L1, Adaptive ElasticNet, Broken adaptive ridge [BAR], and Iterative hard thresholding [IHT]) for discrimination and calibration performance, focusing on both internal and external validation. MATERIALS AND METHODS: We use data from 5 US claims and electronic health record databases and develop models for various outcomes in a major depressive disorder patient population. We externally validate all models in the other databases. We use a train-test split of 75%/25% and evaluate performance with discrimination and calibration. Statistical analysis for difference in performance uses Friedmans test and critical difference diagrams. RESULTS: Of the 840 models we develop, L1 and ElasticNet emerge as superior in both internal and external discrimination, with a notable AUC difference. BAR and IHT show the best internal calibration, without a clear external calibration leader. ElasticNet typically has larger model sizes than L1. Methods like IHT and BAR, while slightly less discriminative, significantly reduce model complexity. CONCLUSION: L1 and ElasticNet offer the best discriminative performance in logistic regression for healthcare predictions, maintaining robustness across validations. For simpler, more interpretable models, L0-based methods (IHT and BAR) are advantageous, providing greater parsimony and calibration with fewer features. This study aids in selecting suitable regularization techniques for healthcare prediction models, balancing performance, complexity, and interpretability.

We have previously shown that bilingual Spanish and English-learning infants can segment English iambs, two-syllable words with final stress (e.g., guiTAR), earlier than their monolingual peers. This is consistent with accelerated development in bilinguals and was attributed to bilingual infants' increased exposure to iambs through Spanish; about 10% of English content words start with an unstressed syllable, compared to 40% in Spanish. Here, we evaluated whether increased exposure to a stress pattern alone is sufficient to account for acceleration in bilingual infants. In English, 90% of content words start with a stressed syllable (e.g., KINGdom), compared to 60% in Spanish. However, we found no evidence for accelerated segmentation of Spanish trochees by Spanish-English bilingual infants compared to their monolingual Spanish-learning peers. Based on this finding, we argue that merely increased exposure to a linguistic feature in one language does not result in accelerated development in the other. Instead, only the acquisition of infrequent patterns in one language may be accelerated due to the additive effects of the other language.

We consider the stability of bound‐state solutions of a family of regularized nonlinear Schrödinger equations which were introduced by Dumas et al. as models for the propagation of laser beams. Among these bound‐state solutions are ground states, which are defined as solutions of a variational problem. We give a sufficient condition for existence and orbital stability of ground states, and use it to verify that ground states exist and are stable over a wider range of nonlinearities than for the nonregularized nonlinear Schrödinger equation. We also give another sufficient and almost necessary condition for stability of general bound states, and show that some stable bound states exist which are not ground states. [ABSTRACT FROM AUTHOR]

This article proposes penalized Mallow's model averaging (pMMA) in the linear regression framework given non nested candidate models. Compared to the MMA, additional constraints are imposed on model weights. We introduce a general framework and allow for non convex constraints such as SCAD, MCP, and TLP. We establish the asymptotic optimality of our proposed penalized MMA (pMMA) estimator and show that the pMMA can achieve a higher sparsity level than the classic MMA. A coordinate-wise descent algorithm has been developed to compute the pMMA estimator efficiently. We conduct simulation and empirical studies to show that our pMMA estimator produces a more sparse weight vector than the MMA, but with better out-of-sample performance. [ABSTRACT FROM AUTHOR]

New epidemics encourage the development of new mathematical models of the spread and forecasting of infectious diseases. Statistical epidemiology data are characterized by incomplete and inexact time series, which leads to an unstable and non-unique forecasting of infectious diseases. In this paper, a model of a conditional generative adversarial neural network (CGAN) for modeling and forecasting COVID-19 in St. Petersburg is constructed. It takes 20 processed historical statistics as a condition and is based on the solution of the minimax problem. The CGAN builds a short-term forecast of the number of newly diagnosed COVID-19 cases in the region for 5 days ahead. The CGAN approach allows modeling the distribution of statistical data, which allows obtaining the required amount of training data from the resulting distribution. When comparing the forecasting results with the classical differential SEIR-HCD model and a recurrent neural network with the same input parameters, it was shown that the forecast errors of all three models are in the same range. It is shown that the prediction error of the bagging model based on three models is lower than the results of each model separately. [ABSTRACT FROM AUTHOR]

In the face of escalating threats to Armenian cultural heritage in Artsakh due to armed conflicts and destruction, urgent measures are imperative to safeguard this invaluable legacy. The Digital Humanities Institute at the EPFL (École polytechnique fédérale de Lausanne, Switzerland) is leading an effort to digitize the Armenian epigraphic heritage in Artsakh, where inscriptions and toponyms serve as cultural identifiers. Documenting endangered inscriptions and using the , and tags safeguard historical integrity. Integrating GeoNames and Pleiades enhances accessibility and accuracy, bridging perspectives and preserving heritage. The project's bilingual nature amplifies complexity, as English and Armenian representations are layered with intricate grammar nuances. Lemmatization and regularization ensure consistent proper names, particularly across diverse grammatical cases. In a landscape where cultural heritage preservation is needed, the marriage of external resources, Armenian toponyms, lemmatization and regularization ensures historical accuracy and accessibility. This academic set-up clarifies the methodologies and standards used, emphasizing the preservation of Armenian cultural heritage through digital epigraphy. The set-up of the corpus allows for future enrichment by incorporating additional inscriptions, more detailed historical background information and etymological clarifications about the toponyms. This flexibility ensures that the corpus can continue to grow and provide comprehensive insights into the cultural heritage of Artsakh. [ABSTRACT FROM AUTHOR]

We study symmetric periodic orbits near collision in a nonautonomous restricted planar four-body problem. The restricted problem consists of a massless particle moving under the gravitational influence due to three bodies with the same positive mass (the primaries), following the figure-eight choreography. We use regularized coordinates, in order to deal numerically with motions near collision between the massless particle and one of the primaries. By means of reversing symmetries, we characterize the symmetric periodic orbits near collision. The initial conditions for these orbits were computed by solving numerically some boundary value problems. We explain theoretically, and confirm numerically, how different parts of the diagram of initial conditions are related due to the symmetry of the figure-eight choreography. [ABSTRACT FROM AUTHOR]

How to improve discriminative feature learning is central in classification. Existing works address this problem by explicitly increasing inter-class separability and intra-class compactness by constructing positive and negative pairs for contrastive learning or posing tighter class separating margins. These methods do not exploit the similarity between different classes as they adhere to independent identical distributions assumption in data. In this paper, we embrace the real-world data distribution setting in that some classes share semantic overlaps due to their similar appearances or concepts. Regarding this hypothesis, we propose a novel regularization to improve discriminative learning. We first calibrate the estimated highest likelihood of one sample based on its semantically neighboring classes, then encourage the overall likelihood predictions to be deterministic by imposing an adaptive exponential penalty. As the gradient of the proposed method is roughly proportional to the uncertainty of the predicted likelihoods, we name it adaptive discriminative regularization (ADR), trained along with a standard cross entropy loss in classification. Extensive experiments demonstrate that it can yield consistent and non-trivial performance improvements in a variety of visual classification tasks (over 10 benchmarks). Furthermore, we find it is robust to long-tailed and noisy label data distribution. Its flexible design enables its compatibility with mainstream classification architectures and losses. [ABSTRACT FROM AUTHOR]

A controlled system of differential equations under the action of an unknown disturbance is considered. The problem discussed in the paper consists in constructing algorithms for forming a control that provides the realization of a prescribed motion for any admissible disturbance. Namely these algorithms should provide the closeness in the metric of the space of differentiable functions of a phase trajectory of a given controlled system and some etalon trajectory of an analogous system functioning when any outer actions are absent. As the space of admissible disturbances, we take the space of measurable square integrable (with respect to the Euclidean norm) functions. The cases of inaccurate measurements of phase trajectories of both systems at all times and at discrete times are under study. Two computer oriented algorithms for solving the problem are designed. The algorithms are based on the (well-known in the theory of guaranteed control) method of extremal shift. In the process, its local (at each time of control correction) regularization is performed by the method of smoothing functional (the Tikhonov method). In addition, estimates for algorithm's convergence rate are presented. [ABSTRACT FROM AUTHOR]

The aim of this paper is to improve the parameter estimation accuracy of the system to be identified by using measurements from a known system. By introducing the transfer gain matrix and setting the effective identification criterion, a novel transfer sparse identification method is raised, which deals with the sparse issues more precise. Besides, the unbiased form is given in the parameter analysis and the recursion form can prevent the dimension catastrophe related problems. Moreover, in order to test the effects of the transfer and avoid bad performance, a negative transfer analysis condition is carried out. Finally, the simulation verifies the enhancements and benefits of the proposed transfer sparse identification method, confirming that the transfer performance outperforms better than that of no transfer, especially on the zero parameters identification. [ABSTRACT FROM AUTHOR]

In this paper, we propose a general distributionally robust regression model based on distributionally robust optimization theory. The proposed model has a piecewise linear loss function and elastic net penalty term, and it generalizes many other regression models. We prove the piecewise linear property of the optimal solutions to this model, which enables us to develop a solution path algorithm for the hyperparameter tuning. A Doubly regularized Least Absolute Deviations (DrLAD) regression model is proposed based on this framework, and a solution path algorithm is developed to speed up the tuning of two hyperparameters in this model. Numerical experiments are implemented to validate the performance of this model and the computational efficiency of the solution path algorithm. [ABSTRACT FROM AUTHOR]

One of the most important needs in neuroimaging is brain dynamic source imaging with high spatial and temporal resolution. EEG source imaging estimates the underlying sources from EEG recordings, which provides enhanced spatial resolution with intrinsically high temporal resolution. To ensure identifiability in the underdetermined source reconstruction problem, constraints on EEG sources are essential. This paper introduces a novel method for estimating source activities based on spatio-temporal constraints and a dynamic source imaging algorithm. The method enhances time resolution by incorporating temporal evolution of neural activity into a regularization function. Additionally, two spatial regularization constraints based on L 1 and L 2 norms are applied in the transformed domain to address both focal and spread neural activities, achieved through spatial gradient and Laplacian transform. Performance evaluation, conducted quantitatively using synthetic datasets, discusses the influence of parameters such as source extent, number of sources, correlation level, and SNR level on temporal and spatial metrics. Results demonstrate that the proposed method provides superior spatial and temporal reconstructions compared to state-of-the-art inverse solutions including STRAPS, sLORETA, SBL, dSPM, and MxNE. This improvement is attributed to the simultaneous integration of transformed spatial and temporal constraints. When applied to a real auditory ERP dataset, our algorithm accurately reconstructs brain source time series and locations, effectively identifying the origins of auditory evoked potentials. In conclusion, our proposed method with spatio-temporal constraints outperforms the state-of-the-art algorithms in estimating source distribution and time courses. [ABSTRACT FROM AUTHOR]

The Tsetlin Machine (TM) is a novel machine learning approach that implements propositional logic to perform various tasks such as classification and regression. The TM not only achieves competitive accuracy in these tasks but also provides results that are explainable and easy to implement using simple hardware. The TM learns using clauses based on the features of the data, and final classification is done using a combination of these clauses. In this paper, we propose the novel idea of adding regularizers to the TM, referred to as Regularized TM (RegTM), to improve generalization. Regularizers have been widely used in machine learning to enhance accuracy. We explore different regularization strategies and their influence on performance. We demonstrate the feasibility of our methodology through various experiments on benchmark datasets. [ABSTRACT FROM AUTHOR]

This study employs a two-dimensional and incompressible flow of Herschel–Bulkley visco-plastic materials in order to investigate the hydrodynamic forces that are acting on a barrier that is located close to the inlet of a channel. As the benchmark configuration, the flow domain that has been selected is a channel that still contains the impediment. The two important parameters of the Herschel–Bulkley Model (HBM) are the yield stress τ y and power law index n. Obtaining special situations within the HBM, such as Newtonian, power-law, and Bingham fluids, can be accomplished by assigning certain values to these parameters at the appropriate times. Utilizing a numerical strategy grounded in the Finite Element Method (FEM), we tackle the nonlinearity of the governing equations as well as the viscosity models. As a result of this nonlinearity, FEM becomes an essential tool. The generation of a refined hybrid mesh is done in order to guarantee accuracy in the computations. The stable finite element pair ( ℙ 2 ∕ ℙ 1 ) has been selected for discretization purposes. The discretized nonlinear system is linearized with Newton's method and subsequently, a direct linear solver PARDISO has been employed in the inner iterations. The pressure, velocity, and viscosity profiles are plotted for various values of n and Bingham number (Bn). In addition, the velocity behavior is observed along the y-direction in a channel through line graphs. Code validation is done as a special case Bn = 0 and a good agreement is found with the results available in the literature. Finally, a correlation analysis has been performed for the drag coefficient C d and lift coefficient C l . [ABSTRACT FROM AUTHOR]

Over the past few years, there has been a surge in research attention towards tasks involving graph data, largely due to the impressive performance demonstrated by graph neural networks (GNNs) in handling such information. Currently, out-of-distribution (OOD) detection in graphs is a hot research topic. The goal of graph OOD detection is to identify nodes or new graphs that differ from the training data distribution, primarily in terms of attributes and structures. OOD detection is crucial for enhancing the stability, security, and robustness of models. In various applications, such as biological networks and financial fraud, graph OOD detection can help models identify anomalies or unforeseen situations, thereby enabling appropriate responses. In node-level OOD detection, existing models typically only consider first-order neighbors. This paper introduces graph diffusion to the OOD detection task for the first time, proposing the HOOD model, a graph diffusion-based OOD node detection algorithm. Specifically, the original graph is processed through graph diffusion to obtain a new graph that can directly capture high-order neighbor information, overcoming the limitation that message passing must go through first-order neighbors. The new graph is then sparsified using a top-k approach. Based on entropy information, regularization is employed to ensure the uncertainty of OOD nodes, thereby giving these nodes higher scores and enabling the model to effectively detect OOD nodes while ensuring the accuracy of in-distribution node classification. Experimental results demonstrate that the HOOD model outperforms existing methods in both node classification and OOD detection tasks on multiple benchmarks, highlighting its robustness and effectiveness. [ABSTRACT FROM AUTHOR]

Count regression models, particularly negative binomial regression (NBR), are widely used in various fields, including biometrics, ecology, and insurance. Over-dispersion is likely when dealing with count data, and NBR has gained attention as an effective tool to address this challenge. However, multicollinearity among covariates and the presence of outliers can lead to inflated confidence intervals and inaccurate predictions in the model. This study proposes a comprehensive approach integrating robust and regularization techniques to handle the simultaneous impact of multicollinearity and outliers in the negative binomial regression model (NBRM). We investigate the estimators' performance through extensive simulation studies and provide analytical comparisons. The simulation results and the theoretical comparisons demonstrate the superiority of the proposed robust hybrid KL estimator (M-NBKLE) with predictive accuracy and stability when multicollinearity and outliers exist. We illustrate the application of our methodology by analyzing a forestry dataset. Our findings complement and reinforce the simulation and theoretical results. [ABSTRACT FROM AUTHOR]

The primary goal of this article is to extend the reciprocal LASSO for applications to binary and survival outcomes. We consider the least squares approximation (LSA) as a solver for the reciprocal LASSO problem. The LSA is a general theoretical framework that includes generalized linear models, Cox regression, and many others as special cases. By applying LSA to reciprocal LASSO regularization, we transfer the original reciprocal LASSO problem into an asymptotically equivalent least squares problem. While the existing literature on reciprocal LASSO has mostly focused on linear models, our algorithm can be easily implemented for general likelihoods, providing a flexible framework for variable selection using reciprocal penalties. To handle the computational burden of implementing the resulting procedure, we employ a scalable stochastic search method called Simplified Shotgun Stochastic Search with Screening (S5), which is easy to implement, without requiring any sophisticated optimization package other than a linear equation solver. We examine the effectiveness of our procedure through Monte Carlo simulations and real data analyses. [ABSTRACT FROM AUTHOR]

This article addresses the deconvolution problem of estimating a square-integrable probability density from observations contaminated with additive measurement errors having a known density. The estimator begins with a density estimate of the contaminated observations and minimizes a reconstruction error penalized by an integrated squared mth derivative. Theory for deconvolution has mainly focused on kernel- or wavelet-based techniques, but other methods including spline-based techniques and this smoothness-penalized estimator have been found to outperform kernel methods in simulation studies. This article fills in some of these gaps by establishing asymptotic guarantees for the smoothness-penalized approach. Consistency is established in mean integrated squared error, and rates of convergence are derived for Gaussian, Cauchy, and Laplace error densities, attaining some lower bounds already in the literature. The assumptions are weak for most results; the estimator can be used with a broader class of error densities than the deconvoluting kernel. Our application example estimates the density of the mean cytotoxicity of certain bacterial isolates under random sampling; this mean cytotoxicity can only be measured experimentally with additive error, leading to the deconvolution problem. We also describe a method for approximating the solution by a cubic spline, which reduces to a quadratic program. for this article are available online. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Guangdong University of Technology is the property of Journal of Guangdong University of Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Purpose: To improve image quality, mitigate quantification biases and variations for free‐breathing liver proton density fat fraction (PDFF) and R2*$$ {\mathrm{R}}_2^{\ast } $$ quantification accelerated by radial k‐space undersampling. Methods: A free‐breathing multi‐echo stack‐of‐radial MRI method was developed with compressed sensing with multidimensional regularization. It was validated in motion phantoms with reference acquisitions without motion and in 11 subjects (6 patients with nonalcoholic fatty liver disease) with reference breath‐hold Cartesian acquisitions. Images, PDFF, and R2*$$ {\mathrm{R}}_2^{\ast } $$ maps were reconstructed using different radial view k‐space sampling factors and reconstruction settings. Results were compared with reference‐standard results using Bland–Altman analysis. Using linear mixed‐effects model fitting (p < 0.05 considered significant), mean and SD were evaluated for biases and variations of PDFF and R2*$$ {\mathrm{R}}_2^{\ast } $$, respectively, and coefficient of variation on the first echo image was evaluated as a surrogate for image quality. Results: Using the empirically determined optimal sampling factor of 0.25 in the accelerated in vivo protocols, mean differences and limits of agreement for the proposed method were [−0.5; −33.6, 32.7] s−1 for R2*$$ {\mathrm{R}}_2^{\ast } $$ and [−1.0%; −5.8%, 3.8%] for PDFF, close to those of a previous self‐gating method using fully sampled radial views: [−0.1; −27.1, 27.0] s−1 for R2*$$ {\mathrm{R}}_2^{\ast } $$ and [−0.4%; −4.5%, 3.7%] for PDFF. The proposed method had significantly lower coefficient of variation than other methods (p < 0.001). Effective acquisition time of 64 s or 59 s was achieved, compared with 171 s or 153 s for two baseline protocols with different radial views corresponding to sampling factor of 1.0. Conclusion: This proposed method may allow accelerated free‐breathing liver PDFF and R2*$$ {\mathrm{R}}_2^{\ast } $$ mapping with reduced biases and variations. [ABSTRACT FROM AUTHOR]

This study examines the planning challenges and opportunities in conserving national trails, focusing on the Israel National Trail. National trails offer crucial benefits, including health promotion, social connectivity, cultural education, ecological conservation, and economic growth. Despite their value, these trails face sustainability challenges from environmental impacts, resource constraints, and competing interests, often exacerbated by weak statutory protections. Using a case study approach, this study analyzes three decades of case law and planning committee decisions, uncovering real-life conflicts and challenges in trail conservation. This study uncovers issues like difficulties in preserving the trail's national identity, risks of route diversions, failure to realize ecological and economic potential, and inefficiencies in planning due to a lack of statutory footing. This study contributes to the emerging field of national trail research by proposing a retrospective methodology for identifying sustainability challenges based on practical realities and by emphasizing the need for comprehensive and transparent planning to ensure the sustainable future of national trails. Legal case studies prove to be a valuable tool for identifying conservation challenges and a significant database for developing a sustainable planning strategy. This methodology serves as a useful and globally relevant tool. [ABSTRACT FROM AUTHOR]

The paper aims to propose a two-step iterated method, which is derived from a regularized dynamical system of extragradient-type in terms of time discretizing, for solving an equilibrium problem. We prove that the iterative sequence generated by the method converges strongly to a solution of the equilibrium problem. Some numerical experiments are given to illustrate and compare the behavior of the new method with several other methods. [ABSTRACT FROM AUTHOR]

Semantic nuclei segmentation is a challenging area of computer vision. Accurate nuclei segmentation can help medics in diagnosing many diseases. Automatic nuclei segmentation can help medics in diagnosing many diseases such as cancer by providing automatic tissue analysis. Deep learning algorithms allow automatic feature extraction from medical images, however, hematoxylin and eosin (H&E) stained images are challenging due to variability in staining and textures. Using pre-trained models in deep learning speeds up development and improves their performance. This paper compares Deeplabv3+ and U-Net deep learning methods with the pre-trained models ResNet-50 and EfficientNetB4 embedded in their architecture. In addition, different regularization and dropout parameters are applied to prevent overtraining. The experiment was conducted on the PanNuke dataset consisting of nearly 8,000 histological images and annotated nuclei. As a result, the ResNet50-based DeepLabV3+ model with L2 regularization of 0.02 and dropout of 0.7 showed efficiency with dice coefficient (DCS) of 0.8356, intersection over union (IOU) of 0.7280, and loss of 0.3212 on the test set. [ABSTRACT FROM AUTHOR]

Waste generation has seen a significant surge in the last decade, presenting an urgent need for efficient and sustainable waste management strategies. The mounting piles of landfill waste underscore the criticality of segregating recyclable from non-recyclable waste, a measure that could alleviate numerous global environmental challenges. The increasing necessity to differentiate waste into specific categories such as paper, plastic, and metal is becoming evident; these categories demand distinct disposal and recycling methods. Addressing this automatic detection and segregation issue calls for an automated garbage classification system, achievable through computer vision. This paper introduces a deep learning and computer vision-based Smart Ensembled Framework for Waste Management (SEFWaM) to categorize garbage into various classes for improved waste management. This innovative model employs a fusion of transfer learning—a deep learning technique and boosting to devise an ensembled approach for efficient waste classification. Our proposed approach, trained on the Trashnet 2.0 dataset, has demonstrated superior performance over competing algorithms in terms of Weighted Macro Precision (WMP), Weighted Macro Recall (WMR), Weighted Macro F1-Score (WMF), and test accuracy. The proposed SEFWaM model achieves an accuracy of 94.2% for the considered dataset, proving its superior efficiency over other deep learning-based models. This research thereby contributes to a pressing environmental issue by offering an automated, efficient solution for waste management. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) are prominent in predictive analytics for accurately forecasting target variables. However, inherent uncertainties necessitate constructing prediction intervals for reliability. The existing literature often lacks practical methodologies for creating predictive intervals, especially for time series with trends and seasonal patterns. This paper explicitly details a practical approach integrating dual-output Monte Carlo Dropout (MCDO) with DNNs to approximate predictive means and variances within a Bayesian framework, enabling forecast interval construction. The dual-output architecture employs a custom loss function, combining mean squared error with Softplus-derived predictive variance, ensuring non-negative variance values. Hyperparameter optimization is performed through a grid search exploring activation functions, dropout rates, epochs, and batch sizes. Empirical distributions of predictive means and variances from the MCDO demonstrate the results of the dual-output MCDO DNNs. The proposed method achieves a significant improvement in forecast accuracy, with an RMSE reduction of about 10% compared to the seasonal autoregressive integrated moving average model. Additionally, the method provides more reliable forecast intervals, as evidenced by a higher coverage proportion and narrower interval widths. A case study on Thailand's durian export data showcases the method's utility and applicability to other datasets with trends and/or seasonal components. [ABSTRACT FROM AUTHOR]

We consider a regularization of the Lagrange multiplier rule (LMR) in the nondifferential form in a convex constrained optimization problem with an operator equality constraint in a Hilbert space and a finite number of functional inequality constraints. The objective functional of the problem is assumed to be strongly convex, and the convex closed set of its admissible elements also belongs to a Hilbert space. The constraints of the problem contain additively included parameters, which enables using the so-called perturbation method to study it. The main purpose of the regularized LMR is the stable generation of generalized minimizing sequences (GMSs), which approximate the exact solution of the problem using extremals of the regular Lagrange functional. The regularized LMR itself can be interpreted as a GMS-generating (regularizing) operator, which assigns to each set of input data of the constrained optimization problem the extremal of its corresponding regular Lagrange functional, in which the dual variable is generated following one or another procedure for stabilizing the dual problem. The main attention is paid to (1) studying the connection between the dual regularization procedure and the subdifferential properties of the value function of the original problem; (2) proving the convergence of this procedure in the case of solvability of the dual problem; (3) an appropriate update of the regularized LMR; (4) obtaining the classical LMR as a limiting version of its regularized analog. [ABSTRACT FROM AUTHOR]

A quadratic minimization problem is considered in Hilbert spaces under constraints given by a linear operator equation and a convex quadratic inequality. The main feature of the problem statement is that the practically available approximations to the exact linear operators specifying the criterion and the constraints converge to them only strongly pointwise rather than in the uniform operator norm, which makes it impossible to justify the use of the classical regularization methods. We propose a regularization method that is applicable in the presence of error estimates for approximate operators in pairs of other operator norms, which are weaker than the original ones. For each of the operators, the pair of corresponding weakened operator norms is obtained by strengthening the norm in the domain of the operator and weakening the norm in its range. The weakening of operator norms usually makes it possible to estimate errors in operators where this was fundamentally impossible in the original norms, for example, in the finite-dimensional approximation of a noncompact operator. From the original optimization formulation, a transition is made to the problem of finding a saddle point of the Lagrange function. The proposed numerical method for finding a saddle point is an iterative regularized extragradient two-stage procedure. At the first stage of each iteration, an approximation to the optimal value of the criterion is refined; at the second stage, the approximate solution with respect to the main variable is refined. Compared to the methods previously developed by the authors and working under similar information conditions, this method is preferable for practical implementation, since it does not require the gradient step size to converge to zero. The main result of the work is the proof of the strong convergence of the approximations generated by the method to one of the exact solutions to the original problem in the norm of the original space. [ABSTRACT FROM AUTHOR]

We formulate a data-independent latent space regularization constraint for general unsupervised autoencoders. The regularization relies on sampling the autoencoder Jacobian at Legendre nodes, which are the centers of the Gauss–Legendre quadrature. Revisiting this classic allows us to prove that regularized autoencoders ensure a one-to-one re-embedding of the initial data manifold into its latent representation. Demonstrations show that previously proposed regularization strategies, such as contractive autoencoding, cause topological defects even in simple examples, as do convolutional-based (variational) autoencoders. In contrast, topological preservation is ensured by standard multilayer perceptron neural networks when regularized using our approach. This observation extends from the classic FashionMNIST dataset to (low-resolution) MRI brain scans, suggesting that reliable low-dimensional representations of complex high-dimensional datasets can be achieved using this regularization technique. [ABSTRACT FROM AUTHOR]

Background: Reducing proton treatment time improves patient comfort and decreases the risk of error from intrafractional motion, but must be balanced against clinical goals and treatment plan quality. Purpose: To improve the delivery efficiency of spot scanning proton therapy by simultaneously reducing the number of spots and energy layers using the reweighted l1$l_1$ regularization method. Methods: We formulated the proton treatment planning problem as a convex optimization problem with a cost function consisting of a dosimetric plan quality term plus a weighted l1$l_1$ regularization term. We iteratively solved this problem and adaptively updated the regularization weights to promote the sparsity of both the spots and energy layers. The proposed algorithm was tested on four head‐and‐neck cancer patients, and its performance, in terms of reducing the number of spots and energy layers, was compared with existing standard l1$l_1$ and group l2$l_2$ regularization methods. We also compared the effectiveness of the three methods (l1$l_1$, group l2$l_2$, and reweighted l1$l_1$) at improving plan delivery efficiency without compromising dosimetric plan quality by constructing each of their Pareto surfaces charting the trade‐off between plan delivery and plan quality. Results: The reweighted l1$l_1$ regularization method reduced the number of spots and energy layers by an average over all patients of 40%$40\%$ and 35%$35\%$, respectively, with an insignificant cost to dosimetric plan quality. From the Pareto surfaces, it is clear that reweighted l1$l_1$ provided a better trade‐off between plan delivery efficiency and dosimetric plan quality than standard l1$l_1$ or group l2$l_2$ regularization, requiring the lowest cost to quality to achieve any given level of delivery efficiency. Conclusions: Reweighted l1$l_1$ regularization is a powerful method for simultaneously promoting the sparsity of spots and energy layers at a small cost to dosimetric plan quality. This sparsity reduces the time required for spot scanning and energy layer switching, thereby improving the delivery efficiency of proton plans. [ABSTRACT FROM AUTHOR]

Extracting the useful information has been used almost everywhere in many fields of mathematics and applied mathematics. It is a classical ill-posed problem due to the unstable dependence of approximations on small perturbation of the data. The traditional regularization methods depend on the choice of the regularization parameter, which are closely related to an available accurate upper bound of noise level; thus it is not appropriate for the randomly distributed noise with big or unknown variance. In this paper, a purely data driven statistical regularization method is proposed, effectively extracting the information from randomly noisy observations. The rigorous upper bound estimation of confidence interval of the error in L 2 norm is established, and some numerical examples are provided to illustrate the effectiveness and computational performance of the method. [ABSTRACT FROM AUTHOR]

In this paper, based on the Karush–Kuhn–Tucker (KKT) conditions of ℓ 0 regularization, we propose a communication-efficient distributed learning approach for high-dimensional and sparse generalized linear models with massive data sets stored across different machines. This proposed method is a support detection and root finding method for generalized linear models in a distributed form. In each round of the proposed method, the support set is first determined by the primal and dual information reduced to the master machine, then the reduced maximum likelihood estimator is obtained by the gradient descent method, among which it only suffices to calculate the gradient vectors on each machine and communicate them instead of the data. We give the optimal ℓ ∞ -norm error bound for the sequences generated by the proposed algorithm and show that this ℓ ∞ -norm error bound decays exponentially to the optimal order. Moreover, we show that the oracle estimator can be recovered if the target signal is not less than the detectable level. In addition, an adaptive version of the proposed algorithm is developed to estimate the sparsity level. Simulation studies illustrate the superior performance of the proposed methods. [ABSTRACT FROM AUTHOR]

Bayesian regularized composite quantile regression approaches with bridge and reciprocal bridge penalties are adopted to conduct variable selection in composite quantile regression. Two MCMC algorithms were developed for posterior inference using the normal-exponential mixture representation of the asymmetric Laplace distribution. Gamma prior is placed on the regularization parameter. The parameters of the Gamma prior are treated as unknowns and estimate them along with the other parameters. The performance of the proposed composite quantile regression methods over the existing methods is then illustrated via simulated studies and a real data set. [ABSTRACT FROM AUTHOR]

We consider piecewise smooth vector fields Z = (Z + , Z -) defined in R n where both vector fields are tangent to the switching manifold Σ along a submanifold M ⊂ Σ . We shall see that, under suitable assumptions, Filippov convention gives rise to a unique sliding mode on M, governed by what we call the tangential sliding vector field. Here, we will provide the necessary and sufficient conditions for characterizing such a vector field. Additionally, we prove that the tangential sliding vector field is conjugated to the reduced dynamics of a singular perturbation problem arising from the Sotomayor–Teixeira regularization of Z around M. Finally, we analyze several examples where tangential sliding vector fields can be observed, including a model for intermittent treatment of HIV. [ABSTRACT FROM AUTHOR]

While biological intelligence grows organically as new knowledge is gathered throughout life, Artificial Neural Networks forget catastrophically whenever they face a changing training data distribution. Rehearsal-based Continual Learning (CL) approaches have been established as a versatile and reliable solution to overcome this limitation; however, sudden input disruptions and memory constraints are known to alter the consistency of their predictions. We study this phenomenon by investigating the geometric characteristics of the learner's latent space and find that replayed data points of different classes increasingly mix up, interfering with classification. Hence, we propose a geometric regularizer that enforces weak requirements on the Laplacian spectrum of the latent space, promoting a partitioning behavior. Our proposal, called Continual Spectral Regularizer for Incremental Learning (CaSpeR-IL), can be easily combined with any rehearsal-based CL approach and improves the performance of SOTA methods on standard benchmarks. • We study the geometry of a model's latent space in a Continual Learning setting. • We propose Continual Spectral Regularizer, a geometrically motivated regularizer. • We combine CaSpeR with SOTA rehearsal-based CL approaches in standard benchmarks. • We compare our proposal with recent contrastive-based CL approaches. • We reveal that CaSpeR achieves increased accuracy and reduced forgetting. [ABSTRACT FROM AUTHOR]

Outliers usually occur during industrial data collection process and can lead to poor system identification performance. This paper considers the parameter estimation probelm for Hammerstein nonlinear systems whose output measurements are contaminated with additive outliers. By incorporating a lasso-type penalty on the outlier vector, a regularization based least squares identification algorithm that can interactively estimate the system parameters and outliers is proposed based on the auxiliary model identification idea. To further reduce the impact of outliers on the identification effect, each observation is assigned an individual weight, which decreases as the number of iterations increases once the corresponding data is determined to be an outlier. Finally, the outliers can be excluded from the contaminated data without any prior knowledge and the estimation bias is reduced. A numerical example and a benchmark system are given to confirm the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

We propose a novel methodology for forecasting spatio-temporal data using supervised semi-nonnegative matrix factorization (SSNMF) with frequency regularization. Matrix factorization is employed to decompose spatio-temporal data into spatial and temporal components. To improve clarity in the temporal patterns, we introduce a nonnegativity constraint on the time domain along with regularization in the frequency domain. Specifically, regularization in the frequency domain involves selecting features in the frequency space, making an interpretation in the frequency domain more convenient. We propose two methods in the frequency domain: soft and hard regularizations, and provide convergence guarantees to first-order stationary points of the corresponding constrained optimization problem. While our primary motivation stems from geophysical data analysis based on GRACE (Gravity Recovery and Climate Experiment) data, our methodology has the potential for wider application. Consequently, when applying our methodology to GRACE data, we find that the results with the proposed methodology are comparable to previous research in the field of geophysical sciences but offer clearer interpretability. [ABSTRACT FROM AUTHOR]

The hole-drilling method is one of the most widespread techniques to measure residual stresses. Since the introduction of the Integral Method to evaluate non-uniform stress distributions, there has been a considerable improvement in the instrumentation technology, as step increments of about 10 microns are now achievable. However, that spatial resolution makes the ill-posedness of the problem stand out among other sources of uncertainty. As the solution becomes totally dominated by noise, an additional regularization of the problem is needed to obtain meaningful results. Tikhonov regularization is the most common option, as it is also prescribed by the hole-drilling ASTM E837 standard, but it has only been studied in the reference case of a hole with no eccentricity with respect to the strain rosette. A recent work by Schajer addresses the eccentricity problem by defining a correction strategy that transforms strain measurements, allowing one to obtain the solution with the usual decoupled equations. In this work, Tikhonov regularization is applied to the eccentric hole case through the influence functions approach, in order to avoid the introduction of new error-compensating functions and bias-prone interpolations. Some useful general considerations for a practical implementation of the procedure and an experimental test case on an aluminum specimen are presented. [ABSTRACT FROM AUTHOR]

This paper explores a novel high-dimensional sparse multiplicative model, which deal with data with positive responses, particularly in economical and biomedical researches. The proposed regularized method is conducted on the least product relative error (LPRE), and can be applied on various penalties including adaptive Lasso, SCAD, and MCP. An adjusted ADMM algorithm is adopted to obtain the estimators based on LPRE loss. Additionally, we prove the consistency and compute the convergence rates of the estimator. To validate the effectiveness of the proposed method, we conduct extensive numerical studies and real data analysis, yielding valuable insights and practical applications, utilizing well-known datasets of the Boston housing data and gold price data. [ABSTRACT FROM AUTHOR]

We consider the problem of denoising with the help of prior information taken from a database of clean signals or images. Denoising with variational methods is very efficient if a regularizer well-adapted to the nature of the data is available. Thanks to the maximum a posteriori Bayesian framework, such regularizer can be systematically linked with the distribution of the data. With deep neural networks (DNN), complex distributions can be recovered from a large training database. To reduce the computational burden of this task, we adapt the compressive learning framework to the learning of regularizers parametrized by DNN. We propose two variants of stochastic gradient descent (SGD) for the recovery of deep regularization parameters from a heavily compressed database. These algorithms outperform the initially proposed method that was limited to low-dimensional signals, each iteration using information from the whole database. They also benefit from classical SGD convergence guarantees. Thanks to these improvements we show that this method can be applied for patch-based image denoising. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) are prominent in predictive analytics for accurately forecasting target variables. However, inherent uncertainties necessitate constructing prediction intervals for reliability. The existing literature often lacks practical methodologies for creating predictive intervals, especially for time series with trends and seasonal patterns. This paper explicitly details a practical approach integrating dual-output Monte Carlo Dropout (MCDO) with DNNs to approximate predictive means and variances within a Bayesian framework, enabling forecast interval construction. The dual-output architecture employs a custom loss function, combining mean squared error with Softplus-derived predictive variance, ensuring non-negative variance values. Hyperparameter optimization is performed through a grid search exploring activation functions, dropout rates, epochs, and batch sizes. Empirical distributions of predictive means and variances from the MCDO demonstrate the results of the dual-output MCDO DNNs. The proposed method achieves a significant improvement in forecast accuracy, with an RMSE reduction of about 10% compared to the seasonal autoregressive integrated moving average model. Additionally, the method provides more reliable forecast intervals, as evidenced by a higher coverage proportion and narrower interval widths. A case study on Thailand’s durian export data showcases the method’s utility and applicability to other datasets with trends and/or seasonal components.

Global in time weak solution to a regularized periodic three-dimensional Boussinesq system is proved to exist in energy spaces. This solution depends continuously on the initial data. In particular, it is unique. The main novelty is the global in time aspect of this solution. The proofs use the coupling between the temperature and the velocity of the fluid, energy methods, and compactness argument.

Earth's magnetic field is generated through the dynamo action in the liquid outer core. Using observational data from geomagnetic satellites and observatories, we can construct geomagnetic models that describe spatial and temporal variations of the core field. Due to the limited temporal and spatial resolution of observations and incomplete parameterization of the model, the inversion problem of geomagnetic field modeling is non-unique. Therefore, we need to incorporate regularization constraints based on prior information to alleviate the non-uniqueness of the inversion problem for geomagnetic modeling. For the core field, the non-uniqueness of modeling inversion manifests in the temporal variations of the field. Studies on the influence of regularization can help us build more reliable core field models, especially on how to suppress artificial secular variations through regularizations. In this study, we construct a series of models based on different regularization parameters within the framework of CHAOS modeling to explore the impact of regularization on the core field modeling. Our results indicate that it is necessary to use regularization constraints for the core field modeling based solely on satellite data. By comparing models with different regularization intensities of the third time derivative of the core field, we show that regularization of the third time derivative with appropriate intensity can effectively suppress artificial signals caused by overfitting of the model to observational data. However, too strong regularizations will reduce the temporal resolution of small-scale secular variations.

In this work, we introduce a novel two-point Landweber-type method to solve the non-smooth ill-posed problems in Banach spaces. The method comprises of inner solvers and two-point acceleration strategy based on iteratively regularized Landweber iteration method. With respect to penalty terms in every step, we use inner solvers to solve certain minimization problems. However, if the minimization problems are explicitly solvable, the inner solvers are same as the exact solvers. The design of the method is such that it allows various types of noises in the given data of non-smooth inverse problems. We study the convergence analysis of our method along with the convergence rates and error estimates by using the standard assumptions that hold without the reflexivity of the image space and G âteaux differentiability of the forward operator. We highlight that this paper is the first advancement in the direction of deducing the convergence rates of a two-point Landweber method. Finally, we present the numerical simulations to show the practicality of the method. [ABSTRACT FROM AUTHOR]

To enhance the accuracy of breast cancer diagnosis, current practices rely on biopsies and microscopic examinations. However, this approach is known for being time-consuming, tedious, and costly. While convolutional neural networks (CNNs) have shown promise for their efficiency and high accuracy, training them effectively becomes challenging in real-world learning scenarios such as class imbalance, small-scale datasets, and label noises. Angular margin-based softmax losses, which concentrate on the angle between features and classifiers embedded in cosine similarity at the classification layer, aim to regulate feature representation learning. Nevertheless, the cosine similarity’s lack of a heavy tail impedes its ability to compactly regulate intra-class feature distribution, limiting generalization performance. Moreover, these losses are constrained to target classes when margin penalties are applied, which may not always optimize effectiveness. Addressing these hurdles, we introduce an innovative approach termed MF-BAM (Mises-Fisher Similarity-based Boosted Additive Angular Margin Loss), which extends beyond traditional cosine similarity and is anchored in the von Mises-Fisher distribution. MF-BAM not only penalizes the angle between deep features and their corresponding target class weights but also considers angles between deep features and weights associated with non-target classes. Through extensive experimentation on the BreaKHis dataset, MF-BAM achieves outstanding accuracies of 99.92%, 99.96%, 100.00%, and 98.05% for magnification levels of ×40, ×100, ×200, and ×400, respectively. Furthermore, additional experiments conducted on the BACH dataset for breast cancer classification, as well as on the LFW and YTF datasets for face recognition, affirm the generalization capability of our proposed loss function. [ABSTRACT FROM AUTHOR]

Summary: In this article we deal with the problem of determining a non‐homogeneous N$$ N $$‐valued heat conductivity profile in a steady‐state heat conduction boundary‐value problem with mixed Dirichlet‐Neumann boundary conditions over a bounded domain in ℝn$$ {\mathbb{R}}^n $$, from the knowledge of the temperature field over the whole domain. In a previous work we developed a method based on a variational approach of the PDE leading to an optimality equation which is then projected into a finite dimensional space. Discretization of the optimality equation then yields a linear although severely ill‐posed equation which is then regularized via appropriate ad‐hoc penalizers based upon a‐priori information about the conductivities of all materials present. This process results in a generalized Tikhonov‐Phillips functional whose global minimizer yields our approximate solution to the inverse problem. In our previous work we showed that this approach yields quite satisfactory results in the cases of two different conductivities. We considered here an appropriate extension of that approach for the N$$ N $$ materials case and show a few numerical examples for the case N=3$$ N=3 $$ in which the method is able to produce very good reconstructions of the exact solution. [ABSTRACT FROM AUTHOR]