Search

Recent studies on transfer learning have shown that selectively fine-tuning a subset of layers or customizing different learning rates for each layer can greatly improve robustness to out-of-distribution (OOD) data and retain generalization capability in the pre-trained models. However, most of these methods employ manually crafted heuristics or expensive hyper-parameter searches, which prevent them from scaling up to large datasets and neural networks. To solve this problem, we propose Trainable Projected Gradient Method (TPGM) to automatically learn the constraint imposed for each layer for a fine-grained fine-tuning regularization. This is motivated by formulating fine-tuning as a bi-level constrained optimization problem. Specifically, TPGM maintains a set of projection radii, i.e., distance constraints between the fine-tuned model and the pre-trained model, for each layer, and enforces them through weight projections. To learn the constraints, we propose a bi-level optimization to automatically learn the best set of projection radii in an end-to-end manner. Theoretically, we show that the bi-level optimization formulation could explain the regularization capability of TPGM. Empirically, with little hyper-parameter search cost, TPGM outperforms existing fine-tuning methods in OOD performance while matching the best in-distribution (ID) performance. For example, when fine-tuned on DomainNet-Real and ImageNet, compared to vanilla fine-tuning, TPGM shows $22\%$ and $10\%$ relative OOD improvement respectively on their sketch counterparts. Code is available at \url{https://github.com/PotatoTian/TPGM}., Accepted to CVPR2023

Using multiple spatial modalities has been proven helpful in improving semantic segmentation performance. However, there are several real-world challenges that have yet to be addressed: (a) improving label efficiency and (b) enhancing robustness in realistic scenarios where modalities are missing at the test time. To address these challenges, we first propose a simple yet efficient multi-modal fusion mechanism Linear Fusion, that performs better than the state-of-the-art multi-modal models even with limited supervision. Second, we propose M3L: Multi-modal Teacher for Masked Modality Learning, a semi-supervised framework that not only improves the multi-modal performance but also makes the model robust to the realistic missing modality scenario using unlabeled data. We create the first benchmark for semi-supervised multi-modal semantic segmentation and also report the robustness to missing modalities. Our proposal shows an absolute improvement of up to 10% on robust mIoU above the most competitive baselines. Our code is available at https://github.com/harshm121/M3L

Recent developments for Semi-Supervised Object Detection (SSOD) have shown the promise of leveraging unlabeled data to improve an object detector. However, thus far these methods have assumed that the unlabeled data does not contain out-of-distribution (OOD) classes, which is unrealistic with larger-scale unlabeled datasets. In this paper, we consider a more practical yet challenging problem, Open-Set Semi-Supervised Object Detection (OSSOD). We first find the existing SSOD method obtains a lower performance gain in open-set conditions, and this is caused by the semantic expansion, where the distracting OOD objects are mispredicted as in-distribution pseudo-labels for the semi-supervised training. To address this problem, we consider online and offline OOD detection modules, which are integrated with SSOD methods. With the extensive studies, we found that leveraging an offline OOD detector based on a self-supervised vision transformer performs favorably against online OOD detectors due to its robustness to the interference of pseudo-labeling. In the experiment, our proposed framework effectively addresses the semantic expansion issue and shows consistent improvements on many OSSOD benchmarks, including large-scale COCO-OpenImages. We also verify the effectiveness of our framework under different OSSOD conditions, including varying numbers of in-distribution classes, different degrees of supervision, and different combinations of unlabeled sets., Comment: Project Page is at https://ycliu93.github.io/projects/ossod.html

Adapting large-scale pretrained models to various downstream tasks via fine-tuning is a standard method in machine learning. Recently, parameter-efficient fine-tuning methods show promise in adapting a pretrained model to different tasks while training only a few parameters. Despite their success, most existing methods are proposed in Natural Language Processing tasks with language Transformers, and adaptation to Computer Vision tasks with Vision Transformers remains under-explored, especially for dense vision tasks. Further, in multi-task settings, individually fine-tuning and storing separate models for different tasks is inefficient. In this work, we provide an extensive multi-task parameter-efficient benchmark and examine existing parameter-efficient fine-tuning NLP methods for vision tasks. Our results on four different dense vision tasks showed that existing methods cannot be efficiently integrated due to the hierarchical nature of the Hierarchical Vision Transformers. To overcome this issue, we propose Polyhistor and Polyhistor-Lite, consisting of Decomposed HyperNetworks and Layer-wise Scaling Kernels, to share information across different tasks with a few trainable parameters. This leads to favorable performance improvements against existing parameter-efficient methods while using fewer trainable parameters. Specifically, Polyhistor achieves competitive accuracy compared to the state-of-the-art while only using ~10% of their trainable parameters. Furthermore, our methods show larger performance gains when large networks and more pretraining data are used., Comment: Accepted to NeurIPS 2022; Project Page is at https://ycliu93.github.io/projects/polyhistor.html

With the recent development of Semi-Supervised Object Detection (SS-OD) techniques, object detectors can be improved by using a limited amount of labeled data and abundant unlabeled data. However, there are still two challenges that are not addressed: (1) there is no prior SS-OD work on anchor-free detectors, and (2) prior works are ineffective when pseudo-labeling bounding box regression. In this paper, we present Unbiased Teacher v2, which shows the generalization of SS-OD method to anchor-free detectors and also introduces Listen2Student mechanism for the unsupervised regression loss. Specifically, we first present a study examining the effectiveness of existing SS-OD methods on anchor-free detectors and find that they achieve much lower performance improvements under the semi-supervised setting. We also observe that box selection with centerness and the localization-based labeling used in anchor-free detectors cannot work well under the semi-supervised setting. On the other hand, our Listen2Student mechanism explicitly prevents misleading pseudo-labels in the training of bounding box regression; we specifically develop a novel pseudo-labeling selection mechanism based on the Teacher and Student's relative uncertainties. This idea contributes to favorable improvement in the regression branch in the semi-supervised setting. Our method, which works for both anchor-free and anchor-based methods, consistently performs favorably against the state-of-the-art methods in VOC, COCO-standard, and COCO-additional., Comment: Project Page is at http://ycliu93.github.io/projects/unbiasedteacher2.html

In this paper, we address the multi-robot collaborative perception problem, specifically in the context of multi-view infilling for distributed semantic segmentation. This setting entails several real-world challenges, especially those relating to unregistered multi-agent image data. Solutions must effectively leverage multiple, non-static, and intermittently-overlapping RGB perspectives. To this end, we propose the Multi-Agent Infilling Network: an extensible neural architecture that can be deployed (in a distributed manner) to each agent in a robotic swarm. Specifically, each robot is in charge of locally encoding and decoding visual information, and an extensible neural mechanism allows for an uncertainty-aware and context-based exchange of intermediate features. We demonstrate improved performance on a realistic multi-robot AirSim dataset., Accepted to ICRA 2020 Workshop on "Emerging Learning and Algorithmic Methods for Data Association in Robotics"; associated spotlight talk available at https://www.youtube.com/watch?v=-lEVvtsfz0I&t=16743s

Semi-supervised learning, i.e., training networks with both labeled and unlabeled data, has made significant progress recently. However, existing works have primarily focused on image classification tasks and neglected object detection which requires more annotation effort. In this work, we revisit the Semi-Supervised Object Detection (SS-OD) and identify the pseudo-labeling bias issue in SS-OD. To address this, we introduce Unbiased Teacher, a simple yet effective approach that jointly trains a student and a gradually progressing teacher in a mutually-beneficial manner. Together with a class-balance loss to downweight overly confident pseudo-labels, Unbiased Teacher consistently improved state-of-the-art methods by significant margins on COCO-standard, COCO-additional, and VOC datasets. Specifically, Unbiased Teacher achieves 6.8 absolute mAP improvements against state-of-the-art method when using 1% of labeled data on MS-COCO, achieves around 10 mAP improvements against the supervised baseline when using only 0.5, 1, 2% of labeled data on MS-COCO., Accepted to ICLR 2021; Code is available at https://github.com/facebookresearch/unbiased-teacher

Neural Networks can perform poorly when the training label distribution is heavily imbalanced, as well as when the testing data differs from the training distribution. In order to deal with shift in the testing label distribution, which imbalance causes, we motivate the problem from the perspective of an optimal Bayes classifier and derive a post-training prior rebalancing technique that can be solved through a KL-divergence based optimization. This method allows a flexible post-training hyper-parameter to be efficiently tuned on a validation set and effectively modify the classifier margin to deal with this imbalance. We further combine this method with existing likelihood shift methods, re-interpreting them from the same Bayesian perspective, and demonstrating that our method can deal with both problems in a unified way. The resulting algorithm can be conveniently used on probabilistic classification problems agnostic to underlying architectures. Our results on six different datasets and five different architectures show state of art accuracy, including on large-scale imbalanced datasets such as iNaturalist for classification and Synthia for semantic segmentation. Please see https://github.com/GT-RIPL/UNO-IC.git for implementation., Comment: Accepted to NeurIPS 2020

Few-shot classification aims to learn a classifier to recognize unseen classes during training with limited labeled examples. While significant progress has been made, the growing complexity of network designs, meta-learning algorithms, and differences in implementation details make a fair comparison difficult. In this paper, we present 1) a consistent comparative analysis of several representative few-shot classification algorithms, with results showing that deeper backbones significantly reduce the performance differences among methods on datasets with limited domain differences, 2) a modified baseline method that surprisingly achieves competitive performance when compared with the state-of-the-art on both the \miniI and the CUB datasets, and 3) a new experimental setting for evaluating the cross-domain generalization ability for few-shot classification algorithms. Our results reveal that reducing intra-class variation is an important factor when the feature backbone is shallow, but not as critical when using deeper backbones. In a realistic cross-domain evaluation setting, we show that a baseline method with a standard fine-tuning practice compares favorably against other state-of-the-art few-shot learning algorithms., ICLR 2019. Code: https://github.com/wyharveychen/CloserLookFewShot . Project: https://sites.google.com/view/a-closer-look-at-few-shot/

Continual learning has received a great deal of attention recently with several approaches being proposed. However, evaluations involve a diverse set of scenarios making meaningful comparison difficult. This work provides a systematic categorization of the scenarios and evaluates them within a consistent framework including strong baselines and state-of-the-art methods. The results provide an understanding of the relative difficulty of the scenarios and that simple baselines (Adagrad, L2 regularization, and naive rehearsal strategies) can surprisingly achieve similar performance to current mainstream methods. We conclude with several suggestions for creating harder evaluation scenarios and future research directions. The code is available at https://github.com/GT-RIPL/Continual-Learning-Benchmark, Comment: Continual Learning Workshop, 32nd Conference on Neural Information Processing Systems (NIPS 2018)

We present a novel and unified deep learning framework which is capable of learning domain-invariant representation from data across multiple domains. Realized by adversarial training with additional ability to exploit domain-specific information, the proposed network is able to perform continuous cross-domain image translation and manipulation, and produces desirable output images accordingly. In addition, the resulting feature representation exhibits superior performance of unsupervised domain adaptation, which also verifies the effectiveness of the proposed model in learning disentangled features for describing cross-domain data., Comment: NIPS 2018

The microwave dielectric properties of Nd (1− x ) Yb x (Mg 0.5 Sn 0.5 )O 3 ceramics were examined with a view to exploiting them for mobile communication. The Nd (1− x ) Yb x (Mg 0.5 Sn 0.5 )O 3 ceramics were prepared by the conventional solid-state method with various sintering temperatures. The X-ray diffraction patterns of the Nd 0.96 Yb 0.04 (Mg 0.5 Sn 0.5 )O 3 ceramics revealed no significant variation of phase with sintering temperatures. An apparent density of 6.87 g/cm 3 , a dielectric constant ( e r ) of 19.48, a quality factor ( Q × f ) of 117,300 GHz, and a temperature coefficient of resonant frequency ( τ f ) of −61 ppm/°C were obtained for Nd 0.96 Yb 0.04 (Mg 0.5 Sn 0.5 )O 3 ceramics that were sintered at 1600 °C for 4 h.

Special Issue on Fundamental Aspects and Recent Developments in Multimedia and VLSI Systems

To detect spatially-modulated signals, a receiver needs the channel state information (CSI) of each transmit- receive antenna pair. Although the CSI is never perfect and varies in time, most studies on spatial modulation (SM) systems assume perfectly known CSI and time-invariant channel. The spatial correlations among multiple spatial subchannels, which have to be considered when CSI is imperfect, are also often neglected. In this paper, we release the above assumptions and take the CSI uncertainty along with the spatial-temporal selectivities into account. We derive the channel estimation error aware maximum likelihood (CEEA-ML) detectors as well as several low- complexity alternatives for PSK and QAM signals. As the CSI uncertainty depends on the channel estimator used, we consider both decision feedback and model based estimators in our study. The error rate performance of the ML and some suboptimal detectors is analyzed. Numerical results obtained by simulations and analysis show that the CEEA-ML detectors offer clear performance gain against conventional mismatched SM detectors and, in many cases, the proposed suboptimal detectors incur only minor performance loss., Comment: 15 pages, 9 figures, 4 tables, submitted to IEEE Trans. Veh. Techol. in May 2017

We present two reduced-rank channel estimators for large-scale multiple-input, multiple-output (MIMO) systems and analyze their mean square error (MSE) performance. Taking advantage of the channel's transform domain sparseness, the estimators yield outstanding performance and may offer additional mean angle-of-arrival (AoA) information. It is shown that, for the estimators to be effective, one has to select a proper predetermined unitary basis (transform) and be able to determine the dominant channel rank and the associated subspace. Further MSE analysis reveals the relations among the array size, channel rank, signal-to-noise ratio (SNR), and the estimators' performance. It provides rationales for the proposed rank determination and mean AoA estimation algorithms as well. An angle alignment operation included in one of our channel models is proved to be effective in further reducing the required rank, shifting the dominant basis vectors' range (channel spectrum) and improving the estimator's performance when a suitable basis is used. We also draw insightful analogies among MIMO channel modeling, transform coding, parallel spatial search, and receive beamforming. Computer experiment results are provided to examine the numerical effects of various estimator parameters and the advantages of the proposed channel estimators and rank determination method., Comment: 12 pages, 10 figures, submitted to IEEE Trans. Wireless Commun

A microfluidic card is described for simultaneous and rapid genetic detection of multiple microbial pathogens. The hydrophobic surface of native acrylic and a novel microfluidic mechanism termed “airlock” were used to dispense sample into a series of 64 reaction wells without the use of valves, external pumping peripherals, multiple layers, or vacuum assistance. This airlock mechanism was tested with dilutions of whole human blood, saliva, and urine, along with mock samples of varying viscosities and surface tensions. Samples spiked with genomic DNA (gDNA) or crude lysates from clinical bacterial isolates were tested with loop mediated isothermal amplification assays (LAMP) designed to target virulence and antibiotic resistance genes. Reactions were monitored in real time using the Gene-Z, which is a portable smartphone-driven system. Samples loaded correctly into the microfluidic card in 99.3 % of instances. Amplification results confirmed no carryover of pre-dispensed primer between wells during sample loading, and no observable diffusion between adjacent wells during the 60 to 90 min isothermal reaction. Sensitivity was comparable between LAMP reactions tested within the microfluidic card and in conventional vials. Tests demonstrate that the airlock card works with various sample types, manufacturing techniques, and can potentially be used in many point-of-care diagnostics applications. Electronic supplementary material The online version of this article (doi:10.1007/s10544-015-9994-1) contains supplementary material, which is available to authorized users.

We consider a multiuser (MU) multiple-input multiple-output (MIMO) time-division duplexing (TDD) system in which the base station (BS) is equipped with a large number of antennas for communicating with single-antenna mobile users. In such a system the BS has to estimate the channel state information (CSI) that includes large-scale fading coefficients (LSFCs) and small-scale fading coefficients (SSFCs) by uplink pilots. Although information about the former FCs are indispensable in a MU-MIMO or distributed MIMO system, they are usually ignored or assumed perfectly known when treating the MIMO CSI estimation problem. We take advantage of the large spatial samples of a massive MIMO BS to derive accurate LSFC estimates in the absence of SSFC information. With estimated LSFCs, SSFCs are then obtained using a rank-reduced (RR) channel model which in essence transforms the channel vector into a lower dimension representation. We analyze the mean squared error (MSE) performance of the proposed composite channel estimator and prove that the separable angle of arrival (AoA) information provided by the RR model is beneficial for enhancing the estimator's performance, especially when the angle spread of the uplink signal is not too large., Comment: 11 pages, 9 figures, submitted to IEEE Journal of Selected Topics in Signal Processing