Search

Traditional low-rank approximation is a powerful tool to compress the huge data matrices that arise in simulations of partial differential equations (PDE), but suffers from high computational cost and requires several passes over the PDE data. The compressed data may also lack interpretability thus making it difficult to identify feature patterns from the original data. To address these issues, we present an online randomized algorithm to compute the interpolative decomposition (ID) of large-scale data matrices {\em in situ}. Compared to previous randomized IDs that used the QR decomposition to determine the column basis, we adopt a streaming ridge leverage score-based column subset selection algorithm that dynamically selects proper basis columns from the data and thus avoids an extra pass over the data to compute the coefficient matrix of the ID. In particular, we adopt a single-pass error estimator based on the non-adaptive Hutch++ algorithm to provide real-time error approximation for determining the best coefficients. As a result, our approach only needs a single pass over the original data and thus is suitable for large and high-dimensional matrices stored outside of core memory or generated in PDE simulations. A strategy to improve the accuracy of the reconstructed data gradient, when desired, within the ID framework is also presented. We provide numerical experiments on turbulent channel flow and ignition simulations, and on the NSTX Gas Puff Image dataset, comparing our algorithm with the offline ID algorithm to demonstrate its utility in real-world applications.

Objective: The transmit encoding model for synthetic aperture imaging is a robust and flexible framework for understanding the effects of acoustic transmission on ultrasound image reconstruction. Our objective is to use machine learning (ML) to construct scanning sequences, parameterized by time delays and apodization weights, that produce high-quality B-mode images. Approach: We use a custom ML model in PyTorch with simulated RF data from Field II to probe the space of possible encoding sequences for those that minimize a loss function that describes image quality. This approach is made computationally feasible by a novel formulation of the derivative for delay-and-sum beamforming. Main Results: When trained for a specified experimental setting (imaging domain, hardware restrictions, etc.), our ML model produces optimized encoding sequences that, when deployed in the REFoCUS imaging framework, improve a number of standard quality metrics over conventional sequences including resolution, field of view, and contrast. We demonstrate these results experimentally on both wire targets and a tissue-mimicking phantom. Significance: This work demonstrates that the set of commonly used encoding schemes represent only a narrow subset of those available. Additionally, it demonstrates the value for ML tasks in synthetic transmit aperture imaging to consider the beamformer within the model, instead of purely as a post-processing step., Comment: 31 pages, 12 figures

Bayesian optimization is a widely used technique for optimizing black-box functions, with Expected Improvement (EI) being the most commonly utilized acquisition function in this domain. While EI is often viewed as distinct from other information-theoretic acquisition functions, such as entropy search (ES) and max-value entropy search (MES), our work reveals that EI can be considered a special case of MES when approached through variational inference (VI). In this context, we have developed the Variational Entropy Search (VES) methodology and the VES-Gamma algorithm, which adapts EI by incorporating principles from information-theoretic concepts. The efficacy of VES-Gamma is demonstrated across a variety of test functions and read datasets, highlighting its theoretical and practical utilities in Bayesian optimization scenarios.

Recent years have seen many successful applications of machine learning (ML) to facilitate fluid dynamic computations. As simulations grow, generating new training datasets for traditional offline learning creates I/O and storage bottlenecks. Additionally, performing inference at runtime requires non-trivial coupling of ML framework libraries with simulation codes. This work offers a solution to both limitations by simplifying this coupling and enabling in situ training and inference workflows on heterogeneous clusters. Leveraging SmartSim, the presented framework deploys a database to store data and ML models in memory, thus circumventing the file system. On the Polaris supercomputer, we demonstrate perfect scaling efficiency to the full machine size of the data transfer and inference costs thanks to a novel co-located deployment of the database. Moreover, we train an autoencoder in situ from a turbulent flow simulation, showing that the framework overhead is negligible relative to a solver time step and training epoch.

Quantifying the uncertainty of quantities of interest (QoIs) from physical systems is a primary objective in model validation. However, achieving this goal entails balancing the need for computational efficiency with the requirement for numerical accuracy. To address this trade-off, we propose a novel bi-fidelity formulation of variational auto-encoders (BF-VAE) designed to estimate the uncertainty associated with a QoI from low-fidelity (LF) and high-fidelity (HF) samples of the QoI. This model allows for the approximation of the statistics of the HF QoI by leveraging information derived from its LF counterpart. Specifically, we design a bi-fidelity auto-regressive model in the latent space that is integrated within the VAE's probabilistic encoder-decoder structure. An effective algorithm is proposed to maximize the variational lower bound of the HF log-likelihood in the presence of limited HF data, resulting in the synthesis of HF realizations with a reduced computational cost. Additionally, we introduce the concept of the bi-fidelity information bottleneck (BF-IB) to provide an information-theoretic interpretation of the proposed BF-VAE model. Our numerical results demonstrate that BF-VAE leads to considerably improved accuracy, as compared to a VAE trained using only HF data, when limited HF data is available.

This study examined whether the association between cybervictimization and internalizing symptoms was moderated by adolescents' emotional connectedness to their social media. Participants were 288 adolescents (54.9% male participants) with (n = 151) and without (n = 137) attention-deficit/hyperactivity disorder (ADHD) between the ages of 13 and 15 years (M = 14.09, SD = 0.36). Adolescents reported on social integration and emotional connection (SIEC) to social media and parents reported on their impression of their adolescent's SIEC to social media. Adolescents also reported on cybervictimization experiences and internalizing symptoms. Adolescents with ADHD had higher cybervictimization scores than adolescents without ADHD and were also more likely to report multiple experiences of cybervictimization over the past month. Emotional investment in social media moderated the relations between cybervictimization and internalizing symptoms such that cybervictimization was associated with higher anxiety and depression symptoms at higher levels of emotional investment in social media. Results were consistent across both parent and adolescent report of social integration and emotional connection to social media. These findings indicate that cybervictimization may be associated with negative outcomes specifically among adolescents with a strong emotional connection to their social media use. [This paper was published in "Research on Child and Adolescent Psychopathology" v50 p363-374 2022.]

Objectives/Background: This study examined whether dispositional mindfulness moderates the association between brooding rumination and sleep problems in adolescents with and without attention-deficit/hyperactivity disorder (ADHD). Participants/Methods: Participants were 137 adolescents (ages 13-15 years; 64% male). Approximately half (47.4%; n=65) were diagnosed with ADHD. Adolescents provided ratings of their dispositional mindfulness and brooding rumination. Both adolescents and parents provided ratings of adolescents' sleep problems. Results: In analyses controlling for sex, race, study site, and group (ADHD vs. comparison), brooding rumination was associated with more adolescent- and parent-reported sleep problems only at low levels of dispositional mindfulness. This effect did not differ for adolescents with or without ADHD and was also unchanged when controlling for internalizing psychopathology symptoms. Conclusions: Our findings showed that dispositional mindfulness might buffer against the negative impact of brooding rumination on adolescent sleep. These findings may have important clinical implications and underscore the potential benefit of including mindfulness and other cognitive-behavioral approaches when treating sleep problems in adolescents. [This paper was published in "Sleep Medicine" v90 p131-134 2023.]

It is unknown whether sluggish cognitive tempo (SCT) is prospectively associated with depression in adolescence, and possible processes linking SCT to depression remain unexamined. Using a longitudinal study with three timepoints over a two-year period, the current study tested the indirect effects of SCT on depression via peer victimization, specifically physical, relational, and verbal victimization. Participants were 302 adolescents (M[subscript age] = 13.17 years; 44.7% female participants; 81.8% White; 52% with ADHD). In the fall of 8th grade, adolescents and parents completed measures of adolescents' SCT and ADHD symptoms. Adolescents completed a measure of peer victimization in spring of 8th grade and a measure of depressive symptoms in 10th grade. Models examining indirect effects were conducted with and without control of baseline ADHD and/or depressive symptoms. Across analyses, adolescent and parent ratings of SCT symptoms uniquely predicted greater depressive symptoms two years later when controlling for adolescent sex, study site, and either 8th grade depressive or ADHD symptoms. Further, adolescents' self-reported 8th grade SCT symptoms predicted 10th grade depressive symptoms via verbal victimization when controlling for 8th grade ADHD symptoms, but not in analyses incorporating 8th grade depressive symptoms. Findings underscore the predictive association of SCT on depressive symptoms, the possible role of adverse peer relationships as a mechanism linking SCT to depression, and the importance of considering ADHD and depressive symptoms in research on longitudinal correlates of SCT. [This paper was published in "Research on Child and Adolescent Psychopathology" v50 p809-822 2022.]

Objective: Researchers have speculated that the COVID-19 pandemic may expand the academic performance gap experienced by at-risk students. We examined learning experiences during the 2020-2021 school year and the impact the pandemic has had on high school student grade point average (GPA), including predictors of change in GPA from 2019-2020 to 2020-2021. Method: Participants were 238 adolescents (55.5% male), 49.6% with attention-deficit/hyperactivity disorder (ADHD), in the United States. Adolescents reported on their GPAs via online surveys. Results: GPA significantly decreased on average from 2019-2020 to 2020-2021 school year. ADHD status and biological sex significantly moderated change--students with ADHD and male students reported decreased GPA, whereas students without ADHD and female students' GPA did not change. Low income and Black/Latinx students had lower GPAs in both school years. Conclusion: It is imperative that additional supports be provided for at-risk students to help them catch up on missed learning during the pandemic. [This paper was published in "Journal of Attention Disorders" v26 p1011-1017 2022.]

Parallel imaging with linear predictability takes advantage of information present in multiple receive coils to accurately reconstruct the image with fewer samples. Commonly used algorithms based on linear predictability include GRAPPA and SPIRiT. We present a sufficient condition for reconstruction based on the direction of undersampling and the arrangement of the sensing coils. This condition is justified theoretically and examples are shown using real data. We also propose a metric based on the fully-sampled auto-calibration region which can show which direction(s) of undersampling will allow for a good quality image reconstruction.

The goal of this study is to assess the feasibility of airway geometry as a biomarker for ASD. Chest CT images of children with a documented diagnosis of ASD as well as healthy controls were identified retrospectively. 54 scans were obtained for analysis, including 31 ASD cases and 23 age and sex-matched controls. A feature selection and classification procedure using principal component analysis (PCA) and support vector machine (SVM) achieved a peak cross validation accuracy of nearly 89% using a feature set of 8 airway branching angles. Sensitivity was 94%, but specificity was only 78%. The results suggest a measurable difference in airway branchpoint angles between children with ASD and the control population. Under review at Scientific Reports

We present a new convolution layer for deep learning architectures which we call QuadConv -- an approximation to continuous convolution via quadrature. Our operator is developed explicitly for use on non-uniform, mesh-based data, and accomplishes this by learning a continuous kernel that can be sampled at arbitrary locations. Moreover, the construction of our operator admits an efficient implementation which we detail and construct. As an experimental validation of our operator, we consider the task of compressing partial differential equation (PDE) simulation data from fixed meshes. We show that QuadConv can match the performance of standard discrete convolutions on uniform grid data by comparing a QuadConv autoencoder (QCAE) to a standard convolutional autoencoder (CAE). Further, we show that the QCAE can maintain this accuracy even on non-uniform data. In both cases, QuadConv also outperforms alternative unstructured convolution methods such as graph convolution., Comment: 26 pages, 18 figures, 5 tables

Least squares regression is a ubiquitous tool for building emulators (a.k.a. surrogate models) of problems across science and engineering for purposes such as design space exploration and uncertainty quantification. When the regression data are generated using an experimental design process (e.g., a quadrature grid) involving computationally expensive models, or when the data size is large, sketching techniques have shown promise to reduce the cost of the construction of the regression model while ensuring accuracy comparable to that of the full data. However, random sketching strategies, such as those based on leverage scores, lead to regression errors that are random and may exhibit large variability. To mitigate this issue, we present a novel boosting approach that leverages cheaper, lower-fidelity data of the problem at hand to identify the best sketch among a set of candidate sketches. This in turn specifies the sketch of the intended high-fidelity model and the associated data. We provide theoretical analyses of this bi-fidelity boosting (BFB) approach and discuss the conditions the low- and high-fidelity data must satisfy for a successful boosting. In doing so, we derive a bound on the residual norm of the BFB sketched solution relating it to its ideal, but computationally expensive, high-fidelity boosted counterpart. Empirical results on both manufactured and PDE data corroborate the theoretical analyses and illustrate the efficacy of the BFB solution in reducing the regression error, as compared to the non-boosted solution., Comment: 36 pages, 9 figures

Approximate solutions to large least squares problems can be computed efficiently using leverage score-based row-sketches, but directly computing the leverage scores, or sampling according to them with naive methods, still requires an expensive manipulation and processing of the design matrix. In this paper we develop efficient leverage score-based sampling methods for matrices with certain Kronecker product-type structure; in particular we consider matrices that are monotone lower column subsets of Kronecker product matrices. Our discussion is general, encompassing least squares problems on infinite domains, in which case matrices formally have infinitely many rows. We briefly survey leverage score-based sampling guarantees from the numerical linear algebra and approximation theory communities, and follow this with efficient algorithms for sampling when the design matrix has Kronecker-type structure. Our numerical examples confirm that sketches based on exact leverage score sampling for our class of structured matrices achieve superior residual compared to approximate leverage score sampling methods., Comment: 33 pages, 5 figures

In this editorial perspective we consider the potential conceptual and empirical overlap between the research on mind wandering, particularly in its pathological extreme, and that on sluggish cognitive tempo (SCT) as it has diverged from research on attention-deficit/hyperactivity disorder (ADHD). The more advanced state of research findings on the nature and correlates of mind wandering relative to that of SCT is used to suggest a variety of avenues of investigation into SCT, such as its phenomenology, positive and negative correlates, research methods, theory building, and potential to inform interventions. These and other avenues drawn from the field of mind wandering are likely to prove fruitful in further revealing the nature of SCT and its relationship to mind wandering. [This paper was published in "JCPP Advances" v1 e12002 2021.]

Objective: To conduct a systematic review of the measures designed to assess sluggish cognitive tempo (SCT) since the first SCT scale using careful test-construction procedures was published in 2009. Methods: The MEDLINE (PubMed), Embase, PsychINFO, and Web of Science databases were searched from September 2009 through December 2019. Articles reporting on the reliability (internal consistency, test-retest, and inter-rater reliability), structural validity (an aspect of construct validity focused on items' convergent and discriminant validity), concurrent and longitudinal external validity, invariance, or intervention/experimental findings were included. Results: Seventy-six studies met full criteria for data extraction and inclusion. Nine measures for assessing SCT were identified (seven assessing parent-, teacher-, and/or self-report in children and two assessing self- and/or collateral-informant report in adults). Each measure has demonstrated acceptable to excellent reliability. All or at least the majority of SCT items on each measure also had structural validity (high loadings on an SCT factor and low loadings on an ADHD inattention factor). Studies have supported the invariance of SCT across sex and time, and there is also initial evidence of invariance across informants, ADHD and non-ADHD youth, and ADHD presentations. The Child and Adolescent Behavior Inventory (CABI), Child Concentration Inventory, Second Edition (CCI-2), and the Barkley Adult ADHD Rating Scale-IV (BAARS-IV) have particularly strong support for assessing parent/teacher-reported, youth self-reported, and adult self-reported SCT, respectively. Conclusion: The SCT measures included in this review share numerous positive properties, have promising psychometric support, and have proven useful for examining the external correlates of SCT across the life span. Although substantial progress has been made over the last decade, work remains to be done to further improve the assessment of SCT and key directions for future research are provided. [This paper was published in "Journal of the American Academy of Child and Adolescent Psychiatry" v60 p690-709 2021.]

Study Objectives: To prospectively examine changes in adolescent sleep before and during the COVID-19 pandemic in adolescents with and without ADHD. Methods: Participants were 122 adolescents (ages 15-17; 61% male; 48% with ADHD). Parents reported on adolescents' sleep duration and difficulties initiating and maintaining sleep (DIMS); adolescents reported on sleep patterns, sleep duration, delayed sleep/wake behaviors, and daytime sleepiness before (September 2019-February 2020) and during (May-June 2020) COVID-19. Adolescents also reported on their health behaviors, COVID-19-related negative affect, and difficulties concentrating due to COVID-19. Results: Parents reported adolescents had more DIMS during COVID-19 than before COVID-19, with clinically-elevated rates increasing from 24% to 36%. Both bedtimes and waketimes shifted later during COVID-19, and adolescents reported more delayed sleep/wake behaviors. Adolescents also reported less daytime sleepiness and longer school night sleep duration during COVID-19. In considering differences between adolescents with and without ADHD, adolescents with ADHD did not experience an increase in school night sleep duration and were less likely to obtain recommended sleep duration during COVID-19. In the full sample, controlling for ADHD status, COVID-19-related sadness/loneliness was associated with increases in DIMS, and spending less time outside and more COVID-19-related worries/fears were associated with increases in delayed sleep/wake behaviors during COVID-19. Conclusions: COVID-19 had negative and positive impacts on adolescent sleep. Adolescents with ADHD did not experience the benefit of increased school night sleep duration during COVID-19 like adolescents without ADHD. Negative affect and health behaviors may be useful intervention targets for reducing negative impacts of COVID-19 for adolescent sleep. [This paper was published in "SLEEP" v44 Article zsab054 2021.]

Background: The impact of chronic stressors like the COVID-19 pandemic is likely to be magnified in adolescents with pre-existing mental health risk, such as attention-deficit/hyperactivity disorder (ADHD). This study examined changes in and predictors of adolescent mental health from before to during the COVID-19 pandemic in the Southeastern and Midwestern United States. Methods: Participants include 238 adolescents (132 males; ages 15-17; 118 with ADHD). Parents and adolescents provided ratings of mental health symptoms shortly before the COVID-19 pandemic and in spring and summer 2020. Results: Adolescents on average experienced an increase in depression, anxiety, sluggish cognitive tempo, inattentive, and oppositional/defiant symptoms from pre-COVID-19 to spring 2020; however, with the exception of inattention, these symptoms decreased from spring to summer 2020. Adolescents with ADHD were more likely than adolescents without ADHD to experience an increase in inattentive, hyperactive/impulsive, and oppositional/defiant symptoms. Adolescents with poorer pre-COVID-19 emotion regulation abilities were at-risk for experiencing increases in all mental health symptoms relative to adolescents with better pre-COVID-19 emotion regulation abilities. Interactive risk based on ADHD status and pre-COVID-19 emotion regulation abilities was found for inattention and hyperactivity/impulsivity, such that adolescents with ADHD and poor pre-COVID-19 emotion regulation displayed the highest symptomatology across timepoints. Lower family income related to increases in inattention but higher family income related to increases in oppositional/defiant symptoms. Conclusions: The early observed increases in adolescent mental health symptoms during the COVID-19 pandemic do not on average appear to be sustained following the lift of stay-at-home orders, though studies evaluating mental health across longer periods of time are needed. Emotion dysregulation and ADHD increase risk for sustained negative mental health functioning and highlight the need for interventions for these populations during chronic stressors. Results and clinical implications should be considered within the context of our predominately White, middle class sample. [This paper was published in "Journal of Child Psychology and Psychiatry" v62 p1132-1139 2021.]

Objective: Although peer difficulties and sluggish cognitive tempo (SCT) are related, studies have yet to examine environmental factors that may advance further understanding this association. The current study tested whether peer difficulties, specifically social competence and peer victimization, interacted with school support, a component of school climate, in relation to adolescents' SCT symptoms. Further, we explored whether these relations would be differentially associated with SCT in adolescents with and without attention-deficit/hyperactivity disorder (ADHD). Method: Adolescents (N = 288; M[subscript age] = 14.08, 45% female, 82.6% White; 52% with ADHD) completed measures of social competence, peer victimization, school climate support, and SCT and ADHD inattentive (IN) symptoms. Parents also reported on adolescents' social competence, SCT, and ADHD-IN symptoms. Results: Results indicated that adolescent and parent ratings of lower social competence were both associated with higher adolescent-reported SCT symptoms in the context of low, but not high, school support. Relational and non-physical victimization were associated with higher self-reported SCT symptoms in the context of low school support. Lower adolescent- and parent-reported social competence were also related to higher parent-reported SCT symptoms, with these associations not moderated by school support. These results remained after controlling for demographics and ADHD-IN symptoms and were similar across adolescents with and without ADHD. Conclusions: Findings from the current study are the first to provide evidence that peer difficulties and school climate are jointly related to adolescents' self-reported SCT and underscore the importance of continued research investigating social adversity and environmental factors in relation to SCT. [This paper will be published in the "Journal of Clinical Child & Adolescent Psychology."]

The Adult Concentration Inventory (ACI) is an adult self-report measure of sluggish cognitive tempo (SCT) developed following a meta-analysis identifying items distinguishing SCT from ADHD inattention. To date, only one study conducted in college students has examined the structural and external validity of the ACI. The current study evaluated the convergent and discriminative validity of the ACI in a community sample of adults, in addition to testing unique associations with internalizing symptoms, daily life executive functions, and sleep. Adults (N = 286; M[subscript age] = 44.45; 83.6% female) completed ratings of SCT, ADHD symptom dimensions, and external correlates. An a priori two-factor model with cross-loadings found 10 of the 16 SCT items to have high loadings on the SCT factor and low loadings on the ADHD inattention factor. SCT was uniquely associated with higher internalizing symptoms, time management and self-organization difficulties, poorer sleep quality, shorter sleep duration, lower sleep efficiency, and more daytime sleepiness. These findings replicate and extend support for the ACI in assessing SCT in adults. [This paper will be published in "Assessment."]

Estimating the fidelity with a target state is important in quantum information tasks. Many fidelity estimation techniques present a suitable measurement scheme to perform the estimation. In contrast, we present techniques that allow the experimentalist to choose a convenient measurement setting. Our primary focus lies on a method that constructs an estimator with nearly minimax optimal confidence intervals for any specified measurement setting. We demonstrate, through a combination of theoretical and numerical results, various desirable properties of the method: robustness against experimental imperfections, competitive sample complexity, and accurate estimates in practice. We compare this method with Maximum Likelihood Estimation and the associated Profile Likelihood method, a Semi-Definite Programming based approach, direct fidelity estimation, quantum state verification, and classical shadows. Our method can also be used for estimating the expectation value of any observable with the same guarantees., Comment: 30 pages, 3 figures; Version accepted in Physical Review A

As quantum devices become more complex and the requirements on these devices become more demanding, it is crucial to be able to verify the performance of such devices in a scalable and reliable fashion. A cornerstone task in this challenge is quantifying how close an experimentally prepared quantum state is to the desired one. Here we present a method to construct an estimator for the quantum state fidelity that is compatible with any measurement protocol. Our method provides a confidence interval on this estimator that is guaranteed to be nearly minimax optimal for the specified measurement protocol. For a well-chosen measurement scheme, our method is competitive in the number of measurement outcomes required for estimation. We demonstrate our method using simulations and experimental data from a trapped-ion quantum computer and compare the results to state-of-the-art techniques. Our method can be easily extended to estimate the expectation value of any observable, such as entanglement witnesses., Comment: 6 pages, 1 figure; Version accepted in Physical Review Letters

Estimation of linear functionals from observed data is an important task in many subjects. Juditsky & Nemirovski [The Annals of Statistics 37.5A (2009): 2278-2300] propose a framework for non-parametric estimation of linear functionals in a very general setting, with nearly minimax optimal confidence intervals. They compute this estimator and the associated confidence interval by approximating the saddle-point of a function. While this optimization problem is convex, it is rather difficult to solve using existing off-the-shelf optimization software. Furthermore, this computation can be expensive when the estimators live in a high-dimensional space. We propose a different algorithm to construct this estimator. Our algorithm can be used with existing optimization software and is much cheaper to implement even when the estimators are in a high-dimensional space, as long as the Hellinger affinity (or the Bhattacharyya coefficient) for the chosen parametric distribution can be efficiently computed given the parameters. We hope that our algorithm will foster the adoption of this estimation technique to a wider variety of problems with relative ease., Comment: 10 pages, no figures

This study aimed to examine changes in depression and anxiety symptoms from before to during the first six months of the COVID-19 pandemic in a sample of 1,339 adolescents (9-18 years old, 59% female) from three countries. We also examined if age, race/ethnicity, disease burden, or strictness of government restrictions moderated change in symptoms. Data from 12 longitudinal studies (10 U.S., 1 Netherlands, 1 Peru) were combined. Linear mixed effect models showed that depression, but not anxiety, symptoms increased significantly (median increase=28%). The most negative mental health impacts were reported by multiracial adolescents and those under 'lockdown' restrictions. Policy makers need to consider these impacts by investing in ways to support adolescents' mental health during the pandemic. [This paper was published in "Journal of Research on Adolescence."]

Objective: This longitudinal study investigated the predictors of and changes in psychological trauma during the COVID-19 pandemic. Method: Participants included 236 adolescents (130 males; M age = 16.74 years in spring 2020; 49.6% diagnosed with attention-deficit hyperactivity disorder; 16.1% diagnosed with an anxiety or depressive disorder) in the United States who completed online questionnaires at four timepoints (spring 2020, summer 2020, fall 2020, spring 2021). Results: Repeated measures ANOVA showed that psychological trauma was highest during stay-at-home orders in spring 2020, and decreased for a majority of adolescents by the summer of 2020. However, [approximately]20% of adolescents exhibited moderate-to-clinical levels of psychological trauma at each timepoint. Four groups were identified based on the presence of psychological trauma symptoms: (1) "resilient group" (normal range across all timepoints; 60.6%), (2) "moderate fluctuating group" (moderate range at one or more timepoints; 18.2%), (3) "severe fluctuating group" (clinical range at one or more timepoints; 14.0%), and (4) "chronic psychological trauma group" (moderate or clinical range across all timepoints; 7.2%). Females, adolescents with pre-existing internalizing disorders, and participants whose families were most impacted by the pandemic were more susceptible to experiencing psychological trauma symptoms. Conclusions: Findings highlight at-risk populations and suggest that the COVID-19 pandemic resulted in psychological trauma symptoms for approximately 20% of adolescents at some point during the first year of the pandemic. There is critical need to provide mental health services to adolescents, such as through school-based services, to reduce the negative long-term psychological impact of the pandemic. [This paper was published in "Psychological Trauma: Theory, Research, Practice, and Policy."]

Introduction: This study examined whether adolescents with and without attention-deficit/hyperactivity disorder (ADHD) differed in affect variability and whether variability in positive and negative affect was associated with functional outcomes. Method: Participants were 302 adolescents (12-14 years, "M"age=13.17, 55% male; 54% diagnosed with ADHD; 82% white) and their caregivers who each completed the 10-item Positive and Negative Affect Scale via daily diaries for approximately two weeks. Multi-informant ratings of emotional, behavioral, social, and academic outcomes were assessed. Results: Adolescents with ADHD were found to experience greater variability in self- and parent-reported positive affect, fear, and distress. For adolescents with ADHD, greater variability in self- and parent-reported positive affect, fear, and distress were associated with more internalizing symptoms, greater variability in parent-reported positive affect was associated with worse social functioning, and greater variability in self- and parent-reported fear was associated with more externalizing symptoms. In contrast, greater variability in self- and parent-reported positive affect, fear, and distress were associated with better social functioning in adolescents without ADHD. Limitations: Future work should examine affect variability in adolescents with ADHD within the same day rather than across days. The limited age range and demographic diversity of our sample may limit generalizability of findings. Conclusions: Findings suggest the significant affect variability found among children with ADHD is also present in adolescents with ADHD and is associated with social and behavior impairment. Interventions for adolescents with ADHD should target emotion regulation abilities to help reduce the extremes of and shifts in affective experiences in this population. [This paper was published in "Journal of Affective Disorders" v274 p500-507 2020.]

Objective: The objective of this study was to compare caffeine consumption in the morning, afternoon, and evening in adolescents with and without ADHD and examine associations with sleep functioning. Methods: Participants were 302 adolescents (ages 12-14) with (n=140) and without (n=162) ADHD. Adolescents wore actigraph watches to assess total sleep time and wake after sleep onset and reported on sleep-wake problems and the number of caffeinated beverages consumed per day in the morning, afternoon, and evening. Parents reported on adolescents' difficulties initiating and maintaining sleep. Chi-square analyses, odds ratios, and path analyses were conducted. Results: Analyses controlled for sex, medication status, and pubertal development. Adolescents with ADHD were 2.47 times more likely to consume caffeine in the afternoon and evening than adolescents without ADHD. Path analyses indicated significant associations between afternoon caffeine use and more self-reported sleep problems for adolescents with and without ADHD, and an association between evening caffeine use and self-reported sleep problems only in adolescents with ADHD. Afternoon caffeine use was associated with parent-reported sleep problems in adolescents with ADHD but not in adolescents without ADHD. Caffeine use was not associated with actigraphy-assessed sleep. Conclusions: This is the first study to show that adolescents with ADHD consume more caffeine than peers during later times of the day. Additionally, caffeine use is more consistently associated with poorer subjective sleep functioning in adolescents with ADHD compared to adolescents without ADHD. Pediatricians and mental health professionals should assess for caffeine use in adolescents with ADHD and co-occurring sleep problems. [This paper was published in "Journal of Pediatric Psychology" v45 n6 p643-653 2020.]

A growing number of studies support the internal and external validity of youth self-reported sluggish cognitive tempo (SCT) symptoms. However, no study has examined SCT in adolescents without ADHD, examined whether adolescent self-reported SCT is distinct from adolescent self-reported ADHD inattention (ADHD-IN), or evaluated whether links between SCT and internalizing problems differ for adolescents with or without ADHD. The present study is the first to (1) determine the convergent and discriminant validity of self-reported SCT and ADHD-IN symptoms in both adolescents with and without ADHD, (2) test the invariance of SCT and ADHD-IN symptoms across ADHD and comparison groups, (3) examine SCT as uniquely related to a range of internalizing-relevant domains, and (4) evaluate if the association between SCT with internalizing correlates differs for adolescents with or without ADHD. Participants were adolescents (M[subscript age]=13 years) with (n=162) and without (n=140) ADHD. Adolescents and parents completed measures of internalizing symptoms and emotion dysregulation; adolescents completed measures of rumination and suicidal ideation. Analyses indicated that 13 of the 15 SCT items demonstrated convergent and discriminant validity from ADHD-IN, and SCT and ADHD-IN demonstrated invariance across the ADHD and comparison groups and across sex. SCT, but not ADHD-IN, was uniquely associated with greater adolescent-reported internalizing symptoms and suicidal ideation. Both SCT and ADHD-IN were uniquely associated with adolescent-reported emotion dysregulation and parent-reported internalizing symptoms. Only ADHD-IN was uniquely associated with parent-reported emotion dysregulation. Findings support the differentiation of adolescent-reported SCT and ADHD-IN and demonstrate associations between SCT and increased internalizing problems in adolescents with and without ADHD. [This article is published in "Journal of Abnormal Child Psychology" v48 p391-406 2020.]

The present study evaluates differences in self-reported intrinsic and extrinsic academic motivation and amotivation between eighth-grade adolescents with (n = 162) and without (n = 140) ADHD. This study also examines associations between motivation and academic functioning with objective (i.e., grade point average, standardized reading and math scores) and cross-rater measurement (i.e., parent-reported homework performance). Multivariate analysis of variance controlling for sex, intelligence, and medication status found that adolescents with ADHD exhibited a significant motivational deficit compared to adolescents without ADHD across all areas of academic motivation, including intrinsic motivation (d = 0.49), extrinsic motivation (d = 0.43), and amotivation (d = 0.42). To examine whether motivation was differentially associated with academic impairment in the ADHD and comparison groups, a multi-group path analysis was conducted controlling for sex, intelligence, and medication status. Findings showed that motivation was differentially associated with academic impairment for adolescents with and without ADHD. For the comparison group, higher amotivation was associated with poorer homework performance and lower intrinsic motivation was associated with lower reading accuracy. In the ADHD group, higher amotivation was associated with poorer homework performance and math fluency, higher extrinsic motivation was associated with better homework performance and higher GPA, and higher intrinsic motivation was associated with higher reading accuracy. This study builds upon previous research in demonstrating that adolescents with ADHD have academic motivational deficits when compared to their peers without ADHD. Research is needed to understand the longitudinal interplay of academic motivation and academic functioning, with an eye towards developing or modifying interventions to increase academic motivation and academic success. [This paper was published in "Journal of Abnormal Child Psychology" v48 p237-249 2020.]

Purpose: To examine remote learning practices and difficulties during initial stay-at-home orders during the COVID-19 pandemic in adolescents with and without ADHD. Methods: Participants were 238 adolescents (132 males; 118 with ADHD) ages 15.64-17.99 years and their parents. Adolescents and parents completed questionnaires in May/June 2020 when in-person schools were closed in the United States. Results: Twenty-two percent of families incurred financial costs to support remote learning, and only 59% of school-based services received prior to COVID-19 continued during COVID-19 remote learning. Adolescents with ADHD had fewer routines and more remote learning difficulties than adolescents without ADHD. Parents of adolescents with ADHD had less confidence in managing remote learning and more difficulties in supporting home learning and home-school communication. Thirty-one percent of parents of adolescents with ADHD with an Individualized Education Program (IEP) or receiving academic accommodations (504 Plan) reported remote learning to be very challenging, compared to 18% of parents of adolescents with ADHD without an IEP/504 Plan and only 4% of parents of adolescents with neither ADHD nor an IEP/504 Plan. Fewer adolescent routines, higher negative affect, and more difficulty concentrating because of COVID-19 were each associated with greater adolescent remote learning difficulties only in adolescents with ADHD. Conclusions: This study provides initial findings of the nature and impact of remote learning during the COVID-19 pandemic. It is imperative for schools and communities to provide the necessary supports to adolescents, particularly those with mental health and/or learning difficulties, and to their parents. Implications and Contributions: The start of the COVID-19 pandemic included the near-universal closing of schools. This study describes the remote learning practices, continuation of school services, financial burden to families, and remote learning difficulties in adolescents with and without ADHD when initial stay-at-home orders were in place in the United States. [This paper was published in "Journal of Adolescent Health" v67 n6 p769-777 2020.]

The coronavirus disease 2019 (COVID-19) pandemic presents tremendous challenges to child and adolescent health. It is expected that the COVID-19 crisis, including the disease and prolonged social distancing, will have a major impact on youth well-being. It is easy to envision how COVID-19 will be the impetus for a host of lingering negative outcomes. Schools are closed, businesses are shuttered, and families are adjusting to 24/7 interaction, while caregivers simultaneously navigate parenting, financial, and professional challenges and uncertainties. As Zhou and colleagues recently reported in this journal, Chinese adolescents have experienced very high rates of anxiety and depression during the COVID-19 outbreak. Risk is real and warrants attention. And yet a sole focus on risk will miss resilience processes that can advance science, services, education, and policy aimed at understanding how children and adolescents respond to crisis. As such, we appreciate that Zhou and colleagues also examined factors that may reduce risk for psychological distress. As Masten observed two decades ago, "resilience does not come from rare or special qualities, but from the everyday magic of the ordinary, normative human resources in… children, in their families and relationships, and in their communities". This ordinary magic in children, families, and communities has a crucial role in the scientific and public health response to COVID-19. [This paper will be published in "European Child and Adolescent Psychiatry."]

It is anticipated that the novel coronavirus disease 2019 (COVID-19) pandemic and associated societal response will have wide-ranging impacts on youth development and mental health. Sleep is crucial for child and adolescent health and well-being, and the potential for sleep problems to emerge or worsen during and following the pandemic is high. This may be particularly true for children and adolescents who are at heightened risk for the onset of sleep and mental health disturbances and for those whom developmental changes impacting sleep are rapidly occurring. Youth with pre-existing psychopathologies (including anxiety and depression) and neurodevelopmental conditions (including attention-deficit/hyperactivity disorder and autism spectrum disorder) could be especially vulnerable to disturbed sleep during this period of change and uncertainty. It is thus imperative that sleep considerations be part of research and clinical initiatives aimed at understanding and mitigating the impact of the COVID-19 pandemic in children and adolescents. This article considers ways in which the pandemic may impact sleep, including research and clinical implications. [This paper was published in "Journal of Child Psychology and Psychiatry" v61 p757-759 2020.]

To advance the research examining the sluggish cognitive tempo (SCT) construct, a key priority has been to develop assessment tools that are reliable and valid. The current study builds upon existing work by conducting the most thorough psychometric evaluation to date of the teacher-reported Child and Adolescent Behavior Inventory (CABI) SCT and attention-deficit/hyperactivity disorder inattention (ADHD-IN) modules in a large sample of elementary students. Participants were 7,613 students (grades 2-5; 50.3% boys) attending 24 elementary schools in three school districts. Teachers (N = 398) provided ratings of SCT, ADHD-IN, academic impairment, and social impairment. An a priori two-factor model with cross-loadings found the SCT items to demonstrate excellent structural validity with ADHD-IN items. The measurement properties of the SCT and ADHD-IN constructs were also invariant across sex and grade. SCT and ADHD-IN were both uniquely associated with academic and social impairment. Graded response item response theory analysis indicated that the SCT and ADHD-IN scales provided a high level of information and precision. The current study replicates and extends previous research and provides the strongest psychometric evidence to date of teacher-rated SCT using the CABI. The teacher-report CABI may be especially useful in the school-based screening of SCT and ADHD-IN. [This paper was published in "Psychological Assessment" v32 p1047-1056 2020.]

Idier et al. [IEEE Trans. Comput. Imaging 4(1), 2018] propose a method which achieves superresolution in the microscopy setting by leveraging random speckle illumination and knowledge about statistical second order moments for the illumination patterns and model noise. This is achieved without any assumptions on the sparsity of the imaged object. In this paper, we show that their technique can be extended to photoacoustic tomography. We propose a simple algorithm for doing the reconstruction which only requires a small number of linear algebra steps. It is therefore much faster than the iterative method used by Idier et al. We also propose a new representation of the imaged object based on Dirac delta expansion functions., Comment: 5 pages, 5 figures; accepted to WASPAA 2021

Scalar optically-pumped magnetometers (OPMs) are being developed in small packages with high sensitivities. The high common-mode rejection ratio of these sensors allows for detection of very small signals in the presence of large background fields making them ideally suited for brain imaging applications in unshielded environments. Despite a flurry of activity around the topic, questions remain concerning how well a dipolar source can be localized under such conditions, especially when using few sensors. In this paper, we investigate the source localization capabilities using an array of scalar OPMs in the presence of a large background field while varying dipole strength, sensor count, and forward model accuracy. We also consider localization performance as the orientation angle of the background field changes. Our results are validated experimentally through accurate localization using a phantom virtual array mimicking a current dipole in a conducting sphere in a large background field. Our results are intended to give researchers a general sense of the capabilities and limitations of scalar OPMs for magnetoencephalography systems., Comment: 18 pages, 9 figures

In this paper we consider regression problems subject to arbitrary noise in the operator or design matrix. This characterization appropriately models many physical phenomena with uncertainty in the regressors. Although the problem has been studied extensively for ordinary/total least squares, and via models that implicitly or explicitly assume Gaussianity, less attention has been paid to improving estimation for regression problems under general uncertainty in the design matrix. To address difficulties encountered when dealing with distributions of sums of random variables, we rely on the saddle point method to estimate densities and form an approximate log-likelihood to maximize. We show that the proposed method performs favorably against other classical methods., Comment: 21 pages, 4 figures

Stochastic gradient Langevin dynamics (SGLD) has gained the attention of optimization researchers due to its global optimization properties. This paper proves an improved convergence property to local minimizers of nonconvex objective functions using SGLD accelerated by variance reductions. Moreover, we prove an ergodicity property of the SGLD scheme, which gives insights on its potential to find global minimizers of nonconvex objectives.

We propose a new modeling framework for highly-multivariate spatial processes that synthesizes ideas from recent multiscale and spectral approaches with graphical models. The basis graphical lasso writes a univariate Gaussian process as a linear combination of basis functions weighted with entries of a Gaussian graphical vector whose graph is estimated from optimizing an $\ell_1$ penalized likelihood. This paper extends the setting to a multivariate Gaussian process where the basis functions are weighted with Gaussian graphical vectors. We motivate a model where the basis functions represent different levels of resolution and the graphical vectors for each level are assumed to be independent. Using an orthogonal basis grants linear complexity and memory usage in the number of spatial locations, the number of basis functions, and the number of realizations. An additional fusion penalty encourages a parsimonious conditional independence structure in the multilevel graphical model. We illustrate our method on a large climate ensemble from the National Center for Atmospheric Research's Community Atmosphere Model that involves 40 spatial processes.

Sluggish cognitive tempo (SCT) is separable from attention-deficit/hyperactivity disorder (ADHD) and other psychopathologies, and growing evidence demonstrates SCT to be associated with impairment in both children and adults. However, it remains unclear how SCT should optimally be conceptualized. In this article, we argue that multiple models of psychopathology should be leveraged in order to make substantive advances to our understanding of SCT. Both categorical and dimensional approaches should be used, including the Diagnostic and Statistical Manual of Mental Disorders (DSM) nosology, the Research Domain Criteria (RDoC) initiative, and hierarchical models of psychopathology. Studies are needed to determine whether individuals categorized with SCT can be reliably identified and differentiated from individuals without SCT in pathophysiological, neuropsychological, behavioral, and daily life functioning. Studies are also needed to evaluate the validity and utility of SCT as a transdiagnostic and dimensional construct. In considering SCT as a dimensional and potentially transdiagnostic construct, we describe ways in which SCT might be examined within the RDoC framework, including negative valence systems, cognitive systems, and arousal/regulatory systems, as well as within hierarchical models of psychopathology. Conceptualizing SCT within both categorical and dimensional models of psychopathology will help to better understand the causes, developmental pathways, and clinical implications of SCT, both as a construct in its own right and also in relation to other psychopathologies. [This paper was published in "European Child and Adolescent Psychiatry" v28 p603-613 2019.]

The Positive and Negative Affect Schedule (PANAS) has been widely used to assess affect expression. Shortened and adolescent versions of the measure have been created, such as the 10-item PANAS for Children (PANAS-C). However, affect expression often involves substantial intraindividual variability, and no research has examined within-person differences using the 10-item PANAS-C. Moreover, intraindividual variability is a key characteristic of attention-deficit hyperactivity disorder (ADHD), and emotion dysregulation is a key feature of ADHD. The present study examined the factor structure of the 10-item PANAS-C in a sample of adolescents (M[subscript age] = 13.17 years) with (n = 156) and without (n = 139) ADHD. A 3-factor (Positive Affect, Fear, Distress) within and a 2-factor (Positive Affect, Negative Affect) between model was found to be best fitting using both parent- and adolescent-report. The model demonstrated configural invariance for the adolescent-report and scalar invariance for the parent-report. These results support the multidimensionality of negative affect in youth with ADHD, even when assessed via the short-version PANAS-C. In future work, it will be important to consider the implications of more discrete types of negative affect expression (fear and distress) found across time at the individual level for assessment and intervention practice. [This paper will be published in "Psychological Assessment."]

Background: Prior studies examining the sleep of adolescents with and without attention-deficit/hyperactivity disorder (ADHD) have relied on mean values such as average sleep duration, which masks intraindividual variability. The objective was to investigate whether adolescents with ADHD have greater intraindividual variability of sleep/wake patterns than adolescents without ADHD using actigraphy and daily sleep diaries. Method: Adolescents (ages 13.17±0.40 years; 45% female) with (n=162) and without (n=140) ADHD were recruited from middle schools at two sites. Participants wore actigraphs and completed sleep diaries for an average of two weeks. Results: Multilevel models were conducted with sex, sleep medication use, ADHD medication use, number of days with data, and social jetlag controlled for in analyses. For actigraphy, adolescents with ADHD had greater variability for time in bed, sleep onset and offset, and wake after sleep onset than adolescents without ADHD. For sleep diary data, adolescents with ADHD had greater variability in bedtime, wake time, sleep duration, sleep onset latency, sleep quality, and night wakings than adolescents without ADHD. Social jetlag was a significant predictor of variability in sleep measures based on both actigraph and daily diaries; however, ADHD status was not associated with social jetlag. Conclusions: This is the first study to show that adolescents with ADHD have more variable sleep/wake patterns than their peers using both objective and subjective sleep measures. Intraindividual variability of sleep/wake patterns may be important for clinicians to assess and monitor as part of treatment. Research is needed to understand the mechanisms underlying increased intraindividual variability of sleep/wake patterns in adolescents with ADHD and potential consequences for daytime functioning. [This article was published in "Journal of Child Psychology and Psychiatry" v60 p1219-1229 2019.]

Background: Children with attention-deficit/hyperactivity disorder (ADHD) experience greater sleep problems than their peers. Although adolescence is generally a developmental period characterized by insufficient sleep, few studies have used a multi-informant, multi-method design, to examine whether sleep differs in adolescents with and without ADHD. Methods: Targeted recruitment was used to enroll an approximately equal number of eighth-grade adolescents (mean age = 13 years) with (n = 162) and without ADHD (n = 140). Adolescents and parents completed global ratings of sleep problems; adolescents, parents, and teachers completed ratings of daytime sleepiness. Adolescents wore actigraphs and completed a daily sleep diary for approximately 2 weeks. Results: Adolescents with ADHD were more likely than adolescents without ADHD to obtain insufficient sleep on school days (per diary) and weekends (per diary and actigraphy). Adolescents with ADHD were also more likely to report falling asleep in class and to have stayed up all night at least twice in the previous 2 weeks (14% and 5% reported all-nighters for ADHD and comparison, respectively). In regression analyses controlling for a number of variables known to impact sleep (e.g. pubertal development, sex, medication use, having an externalizing, anxiety, or depression diagnosis), ADHD remained associated with shorter diary and actigraphy school night sleep duration, adolescent- and parent-reported daytime sleepiness, and parent-reported difficulties initiating and maintaining sleep and total sleep disturbance. Controlling for other variables, the odds of being classified with clinically elevated parent-reported sleep disturbance were 6.20 times greater for adolescents with ADHD. Conclusions: Findings provide some of the clearest evidence yet that adolescents with ADHD experience more sleep problems and sleepiness than their peers without ADHD. It may be especially important to assess for sleep problems in adolescents with ADHD and to evaluate whether existing sleep interventions are effective, or can be optimized, for use in adolescents with ADHD who also have sleep problems. [This paper was published in "Journal of Child Psychology and Psychiatry" v60 p1021-1031 2019.]

Objectives: This study used a multi-informant approach to examine differences in types and rates of technology used by adolescents with and without attention-deficit/hyperactivity disorder (ADHD), associations between technology use and sleep/daytime sleepiness, and whether technology use was differentially related to sleep/daytime sleepiness in adolescents with and without ADHD. Methods: Eighth graders with (n=162) and without (n=140) ADHD were recruited. Adolescents completed questionnaires assessing time spent using technology, sleep-wake problems, school-night time in bed, and daytime sleepiness. Parents and teachers reported on adolescents' technology use and daytime sleepiness, respectively. Results: Adolescents with ADHD had significantly greater total technology, television/movie viewing, video game, and phone/video chatting use than adolescents without ADHD. Adolescents with ADHD engaged in twice as much daily video game use compared to those without ADHD (61 vs. 31 min). Controlling for medication use, ADHD status, pubertal development, sex, and internalizing symptoms, greater parent- and adolescent-reported technology use was associated with more sleep-wake problems and less time in bed. ADHD status did not moderate the relations between technology use and these sleep parameters. In contrast, ADHD status moderated the association between parent-reported technology use and teacher-reported daytime sleepiness, such that this association was significant only for adolescents with ADHD. Conclusions: Technology use, although more prevalent in adolescents with ADHD, is linked with more sleep problems and reduced school-night sleep duration regardless of ADHD status. Technology use is associated with teacher-rated daytime sleepiness only in adolescents with ADHD. Clinicians should consider technology usage when assessing and treating sleep problems. [This paper was published in "Journal of Pediatric Psychology."]

We propose a sampling-based method for computing the tensor ring (TR) decomposition of a data tensor. The method uses leverage score sampled alternating least squares to fit the TR cores in an iterative fashion. By taking advantage of the special structure of TR tensors, we can efficiently estimate the leverage scores and attain a method which has complexity sublinear in the number of input tensor entries. We provide high-probability relative-error guarantees for the sampled least squares problems. We compare our proposal to existing methods in experiments on both synthetic and real data. Our method achieves substantial speedup -- sometimes two or three orders of magnitude -- over competing methods, while maintaining good accuracy. We also provide an example of how our method can be used for rapid feature extraction., Comment: Accepted to ICML 2021

Sketching is a stochastic dimension reduction method that preserves geometric structures of data and has applications in high-dimensional regression, low rank approximation and graph sparsification. In this work, we show that sketching can be used to compress simulation data and still accurately estimate time autocorrelation and power spectral density. For a given compression ratio, the accuracy is much higher than using previously known methods. In addition to providing theoretical guarantees, we apply sketching to a molecular dynamics simulation of methanol and find that the estimate of spectral density is 90% accurate using only 10% of the data., Comment: 17 pages

Stochastic gradient descent is one of the most common iterative algorithms used in machine learning and its convergence analysis is a rich area of research. Understanding its convergence properties can help inform what modifications of it to use in different settings. However, most theoretical results either assume convexity or only provide convergence results in mean. This paper, on the other hand, proves convergence bounds in high probability without assuming convexity. Assuming strong smoothness, we prove high probability convergence bounds in two settings: (1) assuming the Polyak-{\L}ojasiewicz inequality and norm sub-Gaussian gradient noise and (2) assuming norm sub-Weibull gradient noise. In the second setting, as an intermediate step to proving convergence, we prove a sub-Weibull martingale difference sequence self-normalized concentration inequality of independent interest. It extends Freedman-type concentration beyond the sub-exponential threshold to heavier-tailed martingale difference sequences. We also provide a post-processing method that picks a single iterate with a provable convergence guarantee as opposed to the usual bound for the unknown best iterate. Our convergence result for sub-Weibull noise extends the regime where stochastic gradient descent has equal or better convergence guarantees than stochastic gradient descent with modifications such as clipping, momentum, and normalization., Comment: V5: reorganization. No new analysis, but the generalization analysis was removed, the post-processing algorithm was emphasized, and new numerical experiments were run

In this paper we formulate and solve a robust least squares problem for a system of linear equations subject to quantization error in the data matrix. Ordinary least squares fails to consider uncertainty in the operator, modeling all noise in the observed signal. Total least squares accounts for uncertainty in the data matrix, but necessarily increases the condition number of the operator compared to ordinary least squares. Tikhonov regularization or ridge regression is frequently employed to combat ill-conditioning, but requires parameter tuning which presents a host of challenges and places strong assumptions on parameter prior distributions. The proposed method also requires selection of a parameter, but it can be chosen in a natural way, e.g., a matrix rounded to the 4th digit uses an uncertainty bounding parameter of 0.5e-4. We show here that our robust method is theoretically appropriate, tractable, and performs favorably against ordinary and total least squares., Comment: 10 pages, 5 figures

We present a stochastic descent algorithm for unconstrained optimization that is particularly efficient when the objective function is slow to evaluate and gradients are not easily obtained, as in some PDE-constrained optimization and machine learning problems. The algorithm maps the gradient onto a low-dimensional random subspace of dimension $\ell$ at each iteration, similar to coordinate descent but without restricting directional derivatives to be along the axes. Without requiring a full gradient, this mapping can be performed by computing $\ell$ directional derivatives (e.g., via forward-mode automatic differentiation). We give proofs for convergence in expectation under various convexity assumptions as well as probabilistic convergence results under strong-convexity. Our method extends the well-known Gaussian smoothing technique to descent in subspaces of dimension greater than one, opening the doors to new analysis of Gaussian smoothing when more than one directional derivative is used at each iteration. We also provide a finite-dimensional variant of a special case of the Johnson-Lindenstrauss lemma. Experimentally, we show that our method compares favorably to coordinate descent, Gaussian smoothing, gradient descent and BFGS (when gradients are calculated via forward-mode automatic differentiation) on problems from the machine learning and shape optimization literature., Comment: arXiv admin note: substantial text overlap with arXiv:1904.01145

This paper investigates online algorithms for smooth time-varying optimization problems, focusing first on methods with constant step-size, momentum, and extrapolation-length. Assuming strong convexity, precise results for the tracking iterate error (the limit supremum of the norm of the difference between the optimal solution and the iterates) for online gradient descent are derived. The paper then considers a general first-order framework, where a universal lower bound on the tracking iterate error is established. Furthermore, a method using "long-steps" is proposed and shown to achieve the lower bound up to a fixed constant. This method is then compared with online gradient descent for specific examples. Finally, the paper analyzes the effect of regularization when the cost is not strongly convex. With regularization, it is possible to achieve a non-regret bound. The paper ends by testing the accelerated and regularized methods on synthetic time-varying least-squares and logistic regression problems, respectively., Comment: Main changes: added discussion on regret literature in section 2 and added discussion on lower bounds in section 4