Search

Model fairness is becoming important in class-incremental learning for Trustworthy AI. While accuracy has been a central focus in class-incremental learning, fairness has been relatively understudied. However, naively using all the samples of the current task for training results in unfair catastrophic forgetting for certain sensitive groups including classes. We theoretically analyze that forgetting occurs if the average gradient vector of the current task data is in an "opposite direction" compared to the average gradient vector of a sensitive group, which means their inner products are negative. We then propose a fair class-incremental learning framework that adjusts the training weights of current task samples to change the direction of the average gradient vector and thus reduce the forgetting of underperforming groups and achieve fairness. For various group fairness measures, we formulate optimization problems to minimize the overall losses of sensitive groups while minimizing the disparities among them. We also show the problems can be solved with linear programming and propose an efficient Fairness-aware Sample Weighting (FSW) algorithm. Experiments show that FSW achieves better accuracy-fairness tradeoff results than state-of-the-art approaches on real datasets.

The recent huge advance of Large Language Models (LLMs) is mainly driven by the increase in the number of parameters. This has led to substantial memory capacity requirements, necessitating the use of dozens of GPUs just to meet the capacity. One popular solution to this is storage-offloaded training, which uses host memory and storage as an extended memory hierarchy. However, this obviously comes at the cost of storage bandwidth bottleneck because storage devices have orders of magnitude lower bandwidth compared to that of GPU device memories. Our work, Smart-Infinity, addresses the storage bandwidth bottleneck of storage-offloaded LLM training using near-storage processing devices on a real system. The main component of Smart-Infinity is SmartUpdate, which performs parameter updates on custom near-storage accelerators. We identify that moving parameter updates to the storage side removes most of the storage traffic. In addition, we propose an efficient data transfer handler structure to address the system integration issues for Smart-Infinity. The handler allows overlapping data transfers with fixed memory consumption by reusing the device buffer. Lastly, we propose accelerator-assisted gradient compression/decompression to enhance the scalability of Smart-Infinity. When scaling to multiple near-storage processing devices, the write traffic on the shared channel becomes the bottleneck. To alleviate this, we compress the gradients on the GPU and decompress them on the accelerators. It provides further acceleration from reduced traffic. As a result, Smart-Infinity achieves a significant speedup compared to the baseline. Notably, Smart-Infinity is a ready-to-use approach that is fully integrated into PyTorch on a real system. We will open-source Smart-Infinity to facilitate its use., Comment: Published at HPCA 2024 (Best Paper Award Honorable Mention)

Biased data can lead to unfair machine learning models, highlighting the importance of embedding fairness at the beginning of data analysis, particularly during dataset curation and labeling. In response, we propose Falcon, a scalable fair active learning framework. Falcon adopts a data-centric approach that improves machine learning model fairness via strategic sample selection. Given a user-specified group fairness measure, Falcon identifies samples from "target groups" (e.g., (attribute=female, label=positive)) that are the most informative for improving fairness. However, a challenge arises since these target groups are defined using ground truth labels that are not available during sample selection. To handle this, we propose a novel trial-and-error method, where we postpone using a sample if the predicted label is different from the expected one and falls outside the target group. We also observe the trade-off that selecting more informative samples results in higher likelihood of postponing due to undesired label prediction, and the optimal balance varies per dataset. We capture the trade-off between informativeness and postpone rate as policies and propose to automatically select the best policy using adversarial multi-armed bandit methods, given their computational efficiency and theoretical guarantees. Experiments show that Falcon significantly outperforms existing fair active learning approaches in terms of fairness and accuracy and is more efficient. In particular, only Falcon supports a proper trade-off between accuracy and fairness where its maximum fairness score is 1.8-4.5x higher than the second-best results., Comment: Accepted to VLDB 2024

Channel sounding is essential for the development of radio systems. One flexible strategy is the switched-array-based channel sounding, where antenna elements are activated at different time instants to measure the channel spatial characteristics. Although its hardware complexity is decreased due to fewer radio-frequency (RF) chains, sequentially switching the antenna elements can result in aliasing in the joint estimation of angles and Doppler frequencies of multipath components (MPCs). Therefore, pseudo-random switching has been proposed to mitigate such aliasing and increase estimation accuracy in both angular and Doppler domains. Nevertheless, the increased Doppler resolution could cause additional post-processing complexity of parameter estimation, which is relevant when the Doppler frequencies are not of interest, e.g., for spatial channel modeling. This paper proposes an improved hybrid sequential and random switching scheme. The primary purpose is to maintain the estimation accuracy of angles of MPCs while decreasing the resolution of Doppler frequencies for minimized complexity of channel parameter estimation. A simulated-annealing algorithm is exploited to obtain an optimized switching sequence. The effectiveness of the proposed scheme is also demonstrated with a realistic antenna array., Comment: This paper has been accepted by IEEE Vehicular Technology Conference

Patient-reported outcome (PRO) measures are increasingly collected as a means of measuring healthcare quality and value. The capability to predict such measures enables patient-provider shared decision making and the delivery of patient-centered care. However, due to their voluntary nature, PRO measures often suffer from a high missing rate, and the missingness may depend on many patient factors. Under such a complex missing mechanism, statistical inference of the parameters in prediction models for PRO measures is challenging, especially when flexible imputation models such as machine learning or nonparametric methods are used. Specifically, the slow convergence rate of the flexible imputation model may lead to non-negligible bias, and the traditional missing propensity, capable of removing such a bias, is hard to estimate due to the complex missing mechanism. To efficiently infer the parameters of interest, we propose to use an informative surrogate that can lead to a flexible imputation model lying in a low-dimensional subspace. To remove the bias due to the flexible imputation model, we identify a class of weighting functions as alternatives to the traditional propensity score and estimate the low-dimensional one within the identified function class. Based on the estimated low-dimensional weighting function, we construct a one-step debiased estimator without using any information of the true missing propensity. We establish the asymptotic normality of the one-step debiased estimator. Simulation and an application to real-world data demonstrate the superiority of the proposed method., Comment: 20 pages, 1 figure, 2 tables

As machine learning becomes prevalent, mitigating any unfairness present in the training data becomes critical. Among the various notions of fairness, this paper focuses on the well-known individual fairness, which states that similar individuals should be treated similarly. While individual fairness can be improved when training a model (in-processing), we contend that fixing the data before model training (pre-processing) is a more fundamental solution. In particular, we show that label flipping is an effective pre-processing technique for improving individual fairness. Our system iFlipper solves the optimization problem of minimally flipping labels given a limit to the individual fairness violations, where a violation occurs when two similar examples in the training data have different labels. We first prove that the problem is NP-hard. We then propose an approximate linear programming algorithm and provide theoretical guarantees on how close its result is to the optimal solution in terms of the number of label flips. We also propose techniques for making the linear programming solution more optimal without exceeding the violations limit. Experiments on real datasets show that iFlipper significantly outperforms other pre-processing baselines in terms of individual fairness and accuracy on unseen test sets. In addition, iFlipper can be combined with in-processing techniques for even better results., Comment: 20 pages, 19 figures, 8 tables

Correlated outcomes are common in many practical problems. In some settings, one outcome is of particular interest, and others are auxiliary. To leverage information shared by all the outcomes, traditional multi-task learning (MTL) minimizes an averaged loss function over all the outcomes, which may lead to biased estimation for the target outcome, especially when the MTL model is mis-specified. In this work, based on a decomposition of estimation bias into two types, within-subspace and against-subspace, we develop a robust transfer learning approach to estimating a high-dimensional linear decision rule for the outcome of interest with the presence of auxiliary outcomes. The proposed method includes an MTL step using all outcomes to gain efficiency, and a subsequent calibration step using only the outcome of interest to correct both types of biases. We show that the final estimator can achieve a lower estimation error than the one using only the single outcome of interest. Simulations and real data analysis are conducted to justify the superiority of the proposed method., Comment: 19 pages, 2 figures

In the Earth magnetosphere, sunspots and magnetic cusp fusion devices, the boundary between the plasma and the magnetic field is marked by a diamagnetic current layer with a rapid change in plasma pressure and magnetic field strength. First principles numerical simulations were conducted to investigate this boundary layer with a spatial resolution beyond electron gyroradius while incorporating a global equilibrium structure. The boundary layer thickness is discovered to be on the order of electron gyroradius scale due to a self-consistent electric field suppressing ion gyromotion at the boundary. Formed at the scale of the electron gyroradius, the electric field plays a critical role in determining equilibrium structure and plasma transport. The discovery highlights the necessity to incorporate electron gyroradius scale physics in studies aimed at advancing our understanding of fusion devices, the magnetosphere and sunspots., Comment: 29 pages in MS word, 11 figures

Based on the previously developed Energy Conserving Semi Implicit Method (ECsim) code, we present its cylindrical implementation, called ECsim-CYL, to be used for axially symmetric problems. The main motivation for the development of the cylindrical version is to greatly improve the computational speed by utilizing cylindrical symmetry. The ECsim-CYL discretizes the field equations in two-dimensional cylindrical coordinates using the finite volume method . For the particle mover, it uses a modification of ECsim's mover for cylindrical coordinates by keeping track of all three components of velocity vectors, while only keeping radial and axial coordinates of particle positions. In this paper, we describe the details of the algorithm used in the ECsim-CYL and present a series of tests to validate the accuracy of the code including a wave spectrum in a homogeneous plasmas inside a cylindrical waveguide and free expansion of a spherical plasma ball in vacuum. The ECsim-CYL retains the stability properties of ECsim and conserves the energy within machine precision, while accurately describing the plasma behavior in the test cases., Comment: Submitted to Computer Phys. Comm

We report experimental results validating the concept that plasma confinement is enhanced in a magnetic cusp configuration when beta (plasma pressure/magnetic field pressure) is order of unity. This enhancement is required for a fusion power reactor based on cusp confinement to be feasible. The magnetic cusp configuration possesses a critical advantage: the plasma is stable to large scale perturbations. However, early work indicated that plasma loss rates in a reactor based on a cusp configuration were too large for net power production. Grad and others theorized that at high beta a sharp boundary would form between the plasma and the magnetic field, leading to substantially smaller loss rates. The current experiment validates this theoretical conjecture for the first time and represents critical progress toward the Polywell fusion concept which combines a high beta cusp configuration with an electrostatic fusion for a compact, economical, power-producing nuclear fusion reactor., Comment: 12 pages, figures included. 5 movies in Ancillary files