Search

Self-supervised learning (SSL) frameworks consist of pretext task, and loss function aiming to learn useful general features from unlabeled data. The basic idea of most SSL baselines revolves around enforcing the invariance to a variety of data augmentations via the loss function. However, one main issue is that, inattentive or deterministic enforcement of the invariance to any kind of data augmentation is generally not only inefficient, but also potentially detrimental to performance on the downstream tasks. In this work, we investigate the issue from the viewpoint of uncertainty in invariance representation. Uncertainty representation is fairly under-explored in the design of SSL architectures as well as loss functions. We incorporate uncertainty representation in both loss function as well as architecture design aiming for more data-dependent invariance enforcement. The former is represented in the form of data-derived uncertainty in SSL loss function resulting in a generative-discriminative loss function. The latter is achieved by feeding slightly different distorted versions of samples to the ensemble aiming for learning better and more robust representation. Specifically, building upon the recent methods that use hard and soft whitening (a.k.a redundancy reduction), we introduce a new approach GUESS, a pseudo-whitening framework, composed of controlled uncertainty injection, a new architecture, and a new loss function. We include detailed results and ablation analysis establishing GUESS as a new baseline.

Self Supervised learning (SSL) has demonstrated its effectiveness in feature learning from unlabeled data. Regarding this success, there have been some arguments on the role that mutual information plays within the SSL framework. Some works argued for increasing mutual information between representation of augmented views. Others suggest decreasing mutual information between them, while increasing task-relevant information. We ponder upon this debate and propose to revisit the core idea of SSL within the framework of partial information decomposition (PID). Thus, with SSL under PID we propose to replace traditional mutual information with the more general concept of joint mutual information to resolve the argument. Our investigation on instantiation of SSL within the PID framework leads to upgrading the existing pipelines by considering the components of the PID in the SSL models for improved representation learning. Accordingly we propose a general pipeline that can be applied to improve existing baselines. Our pipeline focuses on extracting the unique information component under the PID to build upon lower level supervision for generic feature learning and on developing higher-level supervisory signals for task-related feature learning. In essence, this could be interpreted as a joint utilization of local and global clustering. Experiments on four baselines and four datasets show the effectiveness and generality of our approach in improving existing SSL frameworks.

The success of self-supervised learning (SSL) has been the focus of multiple recent theoretical and empirical studies, including the role of data augmentation (in feature decoupling) as well as complete and dimensional representation collapse. While complete collapse is well-studied and addressed, dimensional collapse has only gain attention and addressed in recent years mostly using variants of redundancy reduction (aka whitening) techniques. In this paper, we further explore a complementary approach to whitening via feature decoupling for improved representation learning while avoiding representation collapse. In particular, we perform feature decoupling by early promotion of useful features via careful feature coloring. The coloring technique is developed based on a Bayesian prior of the augmented data, which is inherently encoded for feature decoupling. We show that our proposed framework is complementary to the state-of-the-art techniques, while outperforming both contrastive and recent non-contrastive methods. We also study the different effects of coloring approach to formulate it as a general complementary technique along with other baselines.

Efficient visual trackers overfit to their training distributions and lack generalization abilities, resulting in them performing well on their respective in-distribution (ID) test sets and not as well on out-of-distribution (OOD) sequences, imposing limitations to their deployment in-the-wild under constrained resources. We introduce SiamABC, a highly efficient Siamese tracker that significantly improves tracking performance, even on OOD sequences. SiamABC takes advantage of new architectural designs in the way it bridges the dynamic variability of the target, and of new losses for training. Also, it directly addresses OOD tracking generalization by including a fast backward-free dynamic test-time adaptation method that continuously adapts the model according to the dynamic visual changes of the target. Our extensive experiments suggest that SiamABC shows remarkable performance gains in OOD sets while maintaining accurate performance on the ID benchmarks. SiamABC outperforms MixFormerV2-S by 7.6\% on the OOD AVisT benchmark while being 3x faster (100 FPS) on a CPU., Comment: WACV 2025

Developing an interpretable system for generating reports in chest X-ray (CXR) analysis is becoming increasingly crucial in Computer-aided Diagnosis (CAD) systems, enabling radiologists to comprehend the decisions made by these systems. Despite the growth of diverse datasets and methods focusing on report generation, there remains a notable gap in how closely these models' generated reports align with the interpretations of real radiologists. In this study, we tackle this challenge by initially introducing Fine-Grained CXR (FG-CXR) dataset, which provides fine-grained paired information between the captions generated by radiologists and the corresponding gaze attention heatmaps for each anatomy. Unlike existing datasets that include a raw sequence of gaze alongside a report, with significant misalignment between gaze location and report content, our FG-CXR dataset offers a more grained alignment between gaze attention and diagnosis transcript. Furthermore, our analysis reveals that simply applying black-box image captioning methods to generate reports cannot adequately explain which information in CXR is utilized and how long needs to attend to accurately generate reports. Consequently, we propose a novel explainable radiologist's attention generator network (Gen-XAI) that mimics the diagnosis process of radiologists, explicitly constraining its output to closely align with both radiologist's gaze attention and transcript. Finally, we perform extensive experiments to illustrate the effectiveness of our method. Our datasets and checkpoint is available at https://github.com/UARK-AICV/FG-CXR., Comment: ACCV 2024

Effective analysis of tabular data still poses a significant problem in deep learning, mainly because features in tabular datasets are often heterogeneous and have different levels of relevance. This work introduces TabSeq, a novel framework for the sequential ordering of features, addressing the vital necessity to optimize the learning process. Features are not always equally informative, and for certain deep learning models, their random arrangement can hinder the model's learning capacity. Finding the optimum sequence order for such features could improve the deep learning models' learning process. The novel feature ordering technique we provide in this work is based on clustering and incorporates both local ordering and global ordering. It is designed to be used with a multi-head attention mechanism in a denoising autoencoder network. Our framework uses clustering to align comparable features and improve data organization. Multi-head attention focuses on essential characteristics, whereas the denoising autoencoder highlights important aspects by rebuilding from distorted inputs. This method improves the capability to learn from tabular data while lowering redundancy. Our research, demonstrating improved performance through appropriate feature sequence rearrangement using raw antibody microarray and two other real-world biomedical datasets, validates the impact of feature ordering. These results demonstrate that feature ordering can be a viable approach to improved deep learning of tabular data., Comment: This paper has been accepted for presentation at the 27th International Conference on Pattern Recognition (ICPR 2024) in Kolkata, India

This work addresses how to efficiently classify challenging histopathology images, such as gigapixel whole-slide images for cancer diagnostics with image-level annotation. We use images with annotated tumor regions to identify a set of tumor patches and a set of benign patches in a cancerous slide. Due to the variable nature of region of interest the tumor positive regions may refer to an extreme minority of the pixels. This creates an important problem during patch-level classification, where the majority of patches from an image labeled as 'cancerous' are actually tumor-free. This problem is different from semantic segmentation which associates a label to every pixel in an image, because after patch extraction we are only dealing with patch-level labels.Most existing approaches address the data imbalance issue by mitigating the data shortage in minority classes in order to prevent the model from being dominated by the majority classes. These methods include data re-sampling, loss re-weighting, margin modification, and data augmentation. In this work, we mitigate the patch-level class imbalance problem by taking a divide-and-conquer approach. First, we partition the data into sub-groups, and define three separate classification sub-problems based on these data partitions. Then, using an information-theoretic cluster-based sampling of deep image patch features, we sample discriminative patches from the sub-groups. Using these sampled patches, we build corresponding deep models to solve the new classification sub-problems. Finally, we integrate information learned from the respective models to make a final decision on the patches. Our result shows that the proposed approach can perform competitively using a very low percentage of the available patches in a given whole-slide image., Comment: 12 pages, 2 figures, Accepted paper for the 27th International Conference on Pattern Recognition (ICPR) 2024

Microscopy data collections are becoming larger and more frequent. Accurate and precise quantitative analysis tools like cell instance segmentation are necessary to benefit from them. This is challenging due to the variability in the data, which requires retraining the segmentation model to maintain high accuracy on new collections. This is needed especially for segmenting cells with elongated and non-convex morphology like bacteria. We propose to reduce the amount of annotation and computing power needed for retraining the model by introducing a few-shot domain adaptation approach that requires annotating only one to five cells of the new data to process and that quickly adapts the model to maintain high accuracy. Our results show a significant boost in accuracy after adaptation to very challenging bacteria datasets., Comment: ISBI 2024

Institutional bias can impact patient outcomes, educational attainment, and legal system navigation. Written records often reflect bias, and once bias is identified; it is possible to refer individuals for training to reduce bias. Many machine learning tools exist to explore text data and create predictive models that can search written records to identify real-time bias. However, few previous studies investigate large language model embeddings and geometric models of biased text data to understand geometry's impact on bias modeling accuracy. To overcome this issue, this study utilizes the RedditBias database to analyze textual biases. Four transformer models, including BERT and RoBERTa variants, were explored. Post-embedding, t-SNE allowed two-dimensional visualization of data. KNN classifiers differentiated bias types, with lower k-values proving more effective. Findings suggest BERT, particularly mini BERT, excels in bias classification, while multilingual models lag. The recommendation emphasizes refining monolingual models and exploring domain-specific biases.

We calculate the thermal Hall conductivity $\kappa_{xy}$ for the columnar valence-bond solid phase of a two-dimensional frustrated antiferromagnet. In particular, we consider the square lattice spin-$1/2$ $J_1$-$J_2$ antiferromagnetic Heisenberg model with an additional Dzyaloshinskii-Moriya interaction between the spins and in the presence of an external magnetic field. We concentrate on the intermediate parameter region of the $J_1$-$J_2$ model, where a quantum paramagnetic phase is stable, and consider a Dzyaloshinskii-Moriya vector pattern associated with the couplings between the spins in the CuO$_2$ planes of the YBCO compound. We describe the columnar valence-bond solid phase within the bond-operator formalism, which allows us to map the Heisenberg model into an effective interacting boson model written in terms of triplet operators. The effective boson model is studied within the harmonic approximation and the triplon excitation bands of the columnar valence-bond solid phase is determined. We then calculate the Berry curvature and the Chern numbers of the triplon excitation bands and, finally, determine the thermal Hall conductivity due to triplons as a function of the temperature. We find that the Dzyaloshinskii-Moriya interaction yields a finite Berry curvature for the triplon bands, but the corresponding Chern numbers vanish. Although the triplon excitations are topologically trivial, the thermal Hall conductivity of the columnar valence-bond solid phase in the square lattice antiferromagnet is finite at low temperatures. We comment on the relations of our results with a {\sl no-go} condition for a thermal Hall effect previously derived for ordered magnets., Comment: 21 pages, 7 figures, final version

Temporal action detection (TAD) involves the localization and classification of action instances within untrimmed videos. While standard TAD follows fully supervised learning with closed-set setting on large training data, recent zero-shot TAD methods showcase the promising open-set setting by leveraging large-scale contrastive visual-language (ViL) pretrained models. However, existing zero-shot TAD methods have limitations on how to properly construct the strong relationship between two interdependent tasks of localization and classification and adapt ViL model to video understanding. In this work, we present ZEETAD, featuring two modules: dual-localization and zero-shot proposal classification. The former is a Transformer-based module that detects action events while selectively collecting crucial semantic embeddings for later recognition. The latter one, CLIP-based module, generates semantic embeddings from text and frame inputs for each temporal unit. Additionally, we enhance discriminative capability on unseen classes by minimally updating the frozen CLIP encoder with lightweight adapters. Extensive experiments on THUMOS14 and ActivityNet-1.3 datasets demonstrate our approach's superior performance in zero-shot TAD and effective knowledge transfer from ViL models to unseen action categories.

Precise 3D environmental mapping is pivotal in robotics. Existing methods often rely on predefined concepts during training or are time-intensive when generating semantic maps. This paper presents Open-Fusion, a groundbreaking approach for real-time open-vocabulary 3D mapping and queryable scene representation using RGB-D data. Open-Fusion harnesses the power of a pre-trained vision-language foundation model (VLFM) for open-set semantic comprehension and employs the Truncated Signed Distance Function (TSDF) for swift 3D scene reconstruction. By leveraging the VLFM, we extract region-based embeddings and their associated confidence maps. These are then integrated with 3D knowledge from TSDF using an enhanced Hungarian-based feature-matching mechanism. Notably, Open-Fusion delivers outstanding annotation-free 3D segmentation for open-vocabulary without necessitating additional 3D training. Benchmark tests on the ScanNet dataset against leading zero-shot methods highlight Open-Fusion's superiority. Furthermore, it seamlessly combines the strengths of region-based VLFM and TSDF, facilitating real-time 3D scene comprehension that includes object concepts and open-world semantics. We encourage the readers to view the demos on our project page: https://uark-aicv.github.io/OpenFusion

Image segmentation remains a pivotal component in medical image analysis, aiding in the extraction of critical information for precise diagnostic practices. With the advent of deep learning, automated image segmentation methods have risen to prominence, showcasing exceptional proficiency in processing medical imagery. Motivated by the Segment Anything Model (SAM)-a foundational model renowned for its remarkable precision and robust generalization capabilities in segmenting 2D natural images-we introduce SAM3D, an innovative adaptation tailored for 3D volumetric medical image analysis. Unlike current SAM-based methods that segment volumetric data by converting the volume into separate 2D slices for individual analysis, our SAM3D model processes the entire 3D volume image in a unified approach. Extensive experiments are conducted on multiple medical image datasets to demonstrate that our network attains competitive results compared with other state-of-the-art methods in 3D medical segmentation tasks while being significantly efficient in terms of parameters. Code and checkpoints are available at https://github.com/UARK-AICV/SAM3D., Comment: Accepted at ISBI 2024

Recently, deep learning has shown to be effective for Electroencephalography (EEG) decoding tasks. Yet, its performance can be negatively influenced by two key factors: 1) the high variance and different types of corruption that are inherent in the signal, 2) the EEG datasets are usually relatively small given the acquisition cost, annotation cost and amount of effort needed. Data augmentation approaches for alleviation of this problem have been empirically studied, with augmentation operations on spatial domain, time domain or frequency domain handcrafted based on expertise of domain knowledge. In this work, we propose a principled approach to perform dynamic evolution on the data for improvement of decoding robustness. The approach is based on distributionally robust optimization and achieves robustness by optimizing on a family of evolved data distributions instead of the single training data distribution. We derived a general data evolution framework based on Wasserstein gradient flow (WGF) and provides two different forms of evolution within the framework. Intuitively, the evolution process helps the EEG decoder to learn more robust and diverse features. It is worth mentioning that the proposed approach can be readily integrated with other data augmentation approaches for further improvements. We performed extensive experiments on the proposed approach and tested its performance on different types of corrupted EEG signals. The model significantly outperforms competitive baselines on challenging decoding scenarios., Comment: ECAI 2023

ChatGPT is a large language model (LLM) created by OpenAI that has been carefully trained on a large amount of data. It has revolutionized the field of natural language processing (NLP) and has pushed the boundaries of LLM capabilities. ChatGPT has played a pivotal role in enabling widespread public interaction with generative artificial intelligence (GAI) on a large scale. It has also sparked research interest in developing similar technologies and investigating their applications and implications. In this paper, our primary goal is to provide a concise survey on the current lines of research on ChatGPT and its evolution. We considered both the glass box and black box views of ChatGPT, encompassing the components and foundational elements of the technology, as well as its applications, impacts, and implications. The glass box approach focuses on understanding the inner workings of the technology, and the black box approach embraces it as a complex system, and thus examines its inputs, outputs, and effects. This paves the way for a comprehensive exploration of the technology and provides a road map for further research and experimentation. We also lay out essential foundational literature on LLMs and GAI in general and their connection with ChatGPT. This overview sheds light on existing and missing research lines in the emerging field of LLMs, benefiting both public users and developers. Furthermore, the paper delves into the broad spectrum of applications and significant concerns in fields such as education, research, healthcare, finance, etc., Comment: The paper was uploaded in error, before it was ready for submission. The paper requires a deep revision, and significant changes and modifications before uploading to the archive

Self-supervised learning (SSL) is now a serious competitor for supervised learning, even though it does not require data annotation. Several baselines have attempted to make SSL models exploit information about data distribution, and less dependent on the augmentation effect. However, there is no clear consensus on whether maximizing or minimizing the mutual information between representations of augmentation views practically contribute to improvement or degradation in performance of SSL models. This paper is a fundamental work where, we investigate role of mutual information in SSL, and reformulate the problem of SSL in the context of a new perspective on mutual information. To this end, we consider joint mutual information from the perspective of partial information decomposition (PID) as a key step in \textbf{reliable multivariate information measurement}. PID enables us to decompose joint mutual information into three important components, namely, unique information, redundant information and synergistic information. Our framework aims for minimizing the redundant information between views and the desired target representation while maximizing the synergistic information at the same time. Our experiments lead to a re-calibration of two redundancy reduction baselines, and a proposal for a new SSL training protocol. Extensive experimental results on multiple datasets and two downstream tasks show the effectiveness of this framework.

Current approaches to novelty or anomaly detection are based on deep neural networks. Despite their effectiveness, neural networks are also vulnerable to imperceptible deformations of the input data. This is a serious issue in critical applications, or when data alterations are generated by an adversarial attack. While this is a known problem that has been studied in recent years for the case of supervised learning, the case of novelty detection has received very limited attention. Indeed, in this latter setting the learning is typically unsupervised because outlier data is not available during training, and new approaches for this case need to be investigated. We propose a new prior that aims at learning a robust likelihood for the novelty test, as a defense against attacks. We also integrate the same prior with a state-of-the-art novelty detection approach. Because of the geometric properties of that approach, the resulting robust training is computationally very efficient. An initial evaluation of the method indicates that it is effective at improving performance with respect to the standard models in the absence and presence of attacks., Comment: CVPR Workshop on Computer Vision in the Wild, 2023

Keypoint detection and matching is a fundamental task in many computer vision problems, from shape reconstruction, to structure from motion, to AR/VR applications and robotics. It is a well-studied problem with remarkable successes such as SIFT, and more recent deep learning approaches. While great robustness is exhibited by these techniques with respect to noise, illumination variation, and rigid motion transformations, less attention has been placed on image distortion sensitivity. In this work, we focus on the case when this is caused by the geometry of the cameras used for image acquisition, and consider the keypoint detection and matching problem between the hybrid scenario of a fisheye and a projective image. We build on a state-of-the-art approach and derive a self-supervised procedure that enables training an interest point detector and descriptor network. We also collected two new datasets for additional training and testing in this unexplored scenario, and we demonstrate that current approaches are suboptimal because they are designed to work in traditional projective conditions, while the proposed approach turns out to be the most effective., Comment: CVPR Workshop on Omnidirectional Computer Vision, 2023

Despite recent significant progress, Multi-Object Tracking (MOT) faces limitations such as reliance on prior knowledge and predefined categories and struggles with unseen objects. To address these issues, Generic Multiple Object Tracking (GMOT) has emerged as an alternative approach, requiring less prior information. However, current GMOT methods often rely on initial bounding boxes and struggle to handle variations in factors such as viewpoint, lighting, occlusion, and scale, among others. Our contributions commence with the introduction of the \textit{Referring GMOT dataset} a collection of videos, each accompanied by detailed textual descriptions of their attributes. Subsequently, we propose $\mathtt{Z-GMOT}$, a cutting-edge tracking solution capable of tracking objects from \textit{never-seen categories} without the need of initial bounding boxes or predefined categories. Within our $\mathtt{Z-GMOT}$ framework, we introduce two novel components: (i) $\mathtt{iGLIP}$, an improved Grounded language-image pretraining, for accurately detecting unseen objects with specific characteristics. (ii) $\mathtt{MA-SORT}$, a novel object association approach that adeptly integrates motion and appearance-based matching strategies to tackle the complex task of tracking objects with high similarity. Our contributions are benchmarked through extensive experiments conducted on the Referring GMOT dataset for GMOT task. Additionally, to assess the generalizability of the proposed $\mathtt{Z-GMOT}$, we conduct ablation studies on the DanceTrack and MOT20 datasets for the MOT task. Our dataset, code, and models are released at: https://fsoft-aic.github.io/Z-GMOT.

Automated cellular instance segmentation is a process utilized for accelerating biological research for the past two decades, and recent advancements have produced higher quality results with less effort from the biologist. Most current endeavors focus on completely cutting the researcher out of the picture by generating highly generalized models. However, these models invariably fail when faced with novel data, distributed differently than the ones used for training. Rather than approaching the problem with methods that presume the availability of large amounts of target data and computing power for retraining, in this work we address the even greater challenge of designing an approach that requires minimal amounts of new annotated data as well as training time. We do so by designing specialized contrastive losses that leverage the few annotated samples very efficiently. A large set of results show that 3 to 5 annotations lead to models with accuracy that: 1) significantly mitigate the covariate shift effects; 2) matches or surpasses other adaptation methods; 3) even approaches methods that have been fully retrained on the target distribution. The adaptation training is only a few minutes, paving a path towards a balance between model performance, computing requirements and expert-level annotation needs., Comment: Accepted in WACV 2023

Person re-identification is a challenging task because of the high intra-class variance induced by the unrestricted nuisance factors of variations such as pose, illumination, viewpoint, background, and sensor noise. Recent approaches postulate that powerful architectures have the capacity to learn feature representations invariant to nuisance factors, by training them with losses that minimize intra-class variance and maximize inter-class separation, without modeling nuisance factors explicitly. The dominant approaches use either a discriminative loss with margin, like the softmax loss with the additive angular margin, or a metric learning loss, like the triplet loss with batch hard mining of triplets. Since the softmax imposes feature normalization, it limits the gradient flow supervising the feature embedding. We address this by joining the losses and leveraging the triplet loss as a proxy for the missing gradients. We further improve invariance to nuisance factors by adding the discriminative task of predicting attributes. Our extensive evaluation highlights that when only a holistic representation is learned, we consistently outperform the state-of-the-art on the three most challenging datasets. Such representations are easier to deploy in practical systems. Finally, we found that joining the losses removes the requirement for having a margin in the softmax loss while increasing performance.

The pandemic of these very recent years has led to a dramatic increase in people wearing protective masks in public venues. This poses obvious challenges to the pervasive use of face recognition technology that now is suffering a decline in performance. One way to address the problem is to revert to face recovery methods as a preprocessing step. Current approaches to face reconstruction and manipulation leverage the ability to model the face manifold, but tend to be generic. We introduce a method that is specific for the recovery of the face image from an image of the same individual wearing a mask. We do so by designing a specialized GAN inversion method, based on an appropriate set of losses for learning an unmasking encoder. With extensive experiments, we show that the approach is effective at unmasking face images. In addition, we also show that the identity information is preserved sufficiently well to improve face verification performance based on several face recognition benchmark datasets.

Self supervised learning (SSL) has become a very successful technique to harness the power of unlabeled data, with no annotation effort. A number of developed approaches are evolving with the goal of outperforming supervised alternatives, which have been relatively successful. One main issue in SSL is robustness of the approaches under different settings. In this paper, for the first time, we recognize the fundamental limits of SSL coming from the use of a single-supervisory signal. To address this limitation, we leverage the power of uncertainty representation to devise a robust and general standard hierarchical learning/training protocol for any SSL baseline, regardless of their assumptions and approaches. Essentially, using the information bottleneck principle, we decompose feature learning into a two-stage training procedure, each with a distinct supervision signal. This double supervision approach is captured in two key steps: 1) invariance enforcement to data augmentation, and 2) fuzzy pseudo labeling (both hard and soft annotation). This simple, yet, effective protocol which enables cross-class/cluster feature learning, is instantiated via an initial training of an ensemble of models through invariance enforcement to data augmentation as first training phase, and then assigning fuzzy labels to the original samples for the second training phase. We consider multiple alternative scenarios with double supervision and evaluate the effectiveness of our approach on recent baselines, covering four different SSL paradigms, including geometrical, contrastive, non-contrastive, and hard/soft whitening (redundancy reduction) baselines. Extensive experiments under multiple settings show that the proposed training protocol consistently improves the performance of the former baselines, independent of their respective underlying principles., Comment: Accepted to WACV 2023

Conventional active learning (AL) frameworks aim to reduce the cost of data annotation by actively requesting the labeling for the most informative data points. However, introducing AL to data hungry deep learning algorithms has been a challenge. Some proposed approaches include uncertainty-based techniques, geometric methods, implicit combination of uncertainty-based and geometric approaches, and more recently, frameworks based on semi/self supervised techniques. In this paper, we address two specific problems in this area. The first is the need for efficient exploitation/exploration trade-off in sample selection in AL. For this, we present an innovative integration of recent progress in both uncertainty-based and geometric frameworks to enable an efficient exploration/exploitation trade-off in sample selection strategy. To this end, we build on a computationally efficient approximate of Thompson sampling with key changes as a posterior estimator for uncertainty representation. Our framework provides two advantages: (1) accurate posterior estimation, and (2) tune-able trade-off between computational overhead and higher accuracy. The second problem is the need for improved training protocols in deep AL. For this, we use ideas from semi/self supervised learning to propose a general approach that is independent of the specific AL technique being used. Taken these together, our framework shows a significant improvement over the state-of-the-art, with results that are comparable to the performance of supervised-learning under the same setting. We show empirical results of our framework, and comparative performance with the state-of-the-art on four datasets, namely, MNIST, CIFAR10, CIFAR100 and ImageNet to establish a new baseline in two different settings., Comment: ACCV 2022

We study the flat-band ferromagnetic phase of a spinfull and time-reversal symmetric Haldane-Hubbard model on a honeycomb lattice within a bosonization formalism for flat-band Z$_2$ topological insulators. Such a study extend our previous one [L. S. G. Leite and R. L. Doretto, Phys. Rev. B {\bf 104}, 155129 (2021)] concerning the flat-band ferromagnetic phase of a correlated Chern insulator described by a Haldane-Hubbard model. We consider the topological Hubbard model at $1/4$ filling of its corresponding noninteracting limit and in the nearly flat band limit of its lower free-electronic bands. We show that it is possible to define boson operators associated with two distinct spin-flip excitations, one that changes (mixed-lattice excitations) and a second one that preserves (same-lattice excitations) the index related with the two triangular sublattices. Within the bosonization scheme, the fermionic model is mapped into an effective interacting boson model, whose quadratic term is considered at the harmonic approximation in order to determine the spin-wave excitation spectrum. For both mixed and same-lattice excitations, we find that the spin-wave spectrum is gapped and has two branches, with an energy gap between the lower and the upper bands at the $K$ and $K'$ points of the first Brillouin zone. Such a behavior is distinct from the one of the corresponding correlated Chern insulator, whose spin-wave spectrum has a Goldstone mode at the center of the first Brillouin zone and Dirac points at $K$ and $K'$ points. We also find some evidences that the spin-wave bands for the same-lattice excitations might be topologically nontrivial even in the completely flat band limit., Comment: 16 pages, 8 figures, companion paper to our previous arXiv:2106.00468, final version

The study of signatures of aging in terms of genomic biomarkers can be uniquely helpful in understanding the mechanisms of aging and developing models to accurately predict the age. Prior studies have employed gene expression and DNA methylation data aiming at accurate prediction of age. In this line, we propose a new framework for human age estimation using information from human dermal fibroblast gene expression data. First, we propose a new spatial representation as well as a data augmentation approach for gene expression data. Next in order to predict the age, we design an architecture of neural network and apply it to this new representation of the original and augmented data, as an ensemble classification approach. Our experimental results suggest the superiority of the proposed framework over state-of-the-art age estimation methods using DNA methylation and gene expression data., Comment: 8 pages, 5 figures, This paper is accepted to 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

Plant species identification in the wild is a difficult problem in part due to the high variability of the input data, but also because of complications induced by the long-tail effects of the datasets distribution. Inspired by the most recent fine-grained visual classification approaches which are based on attention to mitigate the effects of data variability, we explore the idea of using object detection as a form of attention. We introduce a bottom-up approach based on detecting plant organs and fusing the predictions of a variable number of organ-based species classifiers. We also curate a new dataset with a long-tail distribution for evaluating plant organ detection and organ-based species identification, which is publicly available., Comment: 6 pages, 4 figures. Accepted to the CVPR 2021 FGVC Workshop. Models, testing code, and link to dataset can be found at https://github.com/wvuvl/DARMA

We study the flat-band ferromagnetic phase of the Haldane-Hubbard model on a honeycomb lattice within a bosonization scheme for flat-band Chern insulators, focusing on the calculation of the spin-wave excitation spectrum. We consider the Haldane-Hubbard model with the noninteracting lower bands in a nearly-flat band limit, previously determined for the spinless model, and at 1/4-filling of its corresponding noninteracting limit. Within the bosonization scheme, the Haldane-Hubbard model is mapped into an effective interacting boson model, whose quadratic term allows us to determine the spin-wave spectrum at the harmonic approximation. We show that the excitation spectrum has two branches with a Goldstone mode and Dirac points at center and at the K and K' points of the first Brillouin zone, respectively. We also consider the effects on the spin-wave spectrum due to an energy offset in the on-site Hubbard repulsion energies and due to the presence of an staggered on-site energy term, both quantities associated with the two triangular sublattices. In both cases, we find that an energy gap opens at the K and K' points. Moreover, we also find some evidences for an instability of the flat-band ferromagnetic phase in the presence of the staggered on-site energy term. We provide some additional results for the square lattice topological Hubbard model previous studied within the bosonization formalism and comment on the differences between the bosonization scheme implementation for the correlated Chern insulators on both square and honeycomb lattices., Comment: 17 pages, 11 figures, final version

Autoencoder networks are unsupervised approaches aiming at combining generative and representational properties by learning simultaneously an encoder-generator map. Although studied extensively, the issues of whether they have the same generative power of GANs, or learn disentangled representations, have not been fully addressed. We introduce an autoencoder that tackles these issues jointly, which we call Adversarial Latent Autoencoder (ALAE). It is a general architecture that can leverage recent improvements on GAN training procedures. We designed two autoencoders: one based on a MLP encoder, and another based on a StyleGAN generator, which we call StyleALAE. We verify the disentanglement properties of both architectures. We show that StyleALAE can not only generate 1024x1024 face images with comparable quality of StyleGAN, but at the same resolution can also produce face reconstructions and manipulations based on real images. This makes ALAE the first autoencoder able to compare with, and go beyond the capabilities of a generator-only type of architecture.

We study a system of interacting triplons (the elementary excitations of a valence-bond solid) described by an effective interacting boson model derived within the bond-operator formalism. In particular, we consider the square lattice spin-1/2 $J_1$-$J_2$ antiferromagnetic Heisenberg model, focus on the intermediate parameter region, where a quantum paramagnetic phase sets in, and consider the columnar valence-bond solid phase. Within the bond-operator theory, the Heisenberg model is mapped into an effective boson model in terms of triplet operators $t$. The effective boson model is studied at the harmonic approximation and the energy of the triplons and the expansion of the triplon operators $b$ in terms of the triplet operators $t$ are determined. Such an expansion allows us to performed a second mapping, and therefore, determine an effective interacting boson model in terms of the triplon operators $b$. We then consider systems with a fixed number of triplons and determined the ground-state energy and the spectrum of elementary excitations within a mean-field approximation. We show that many-triplon states are stable, the lowest-energy ones are constituted by a small number of triplons, and the excitation gaps are finite. For $J_2=0.48 J_1$ and $J_2=0.52 J_1$, we also calculate spin-spin and dimer-dimer correlation functions, dimer order parameters, and the bipartite von Neumann entanglement entropy within our mean-field formalism in order to determine the properties of the many-triplon state as a function of the triplon number. We find that the spin and the dimer correlations decay exponentially and that the entanglement entropy obeys an area law, regardless the triplon number. Moreover, only for $J_2=0.48 J_1$, the spin correlations indicate that the many-triplon states with large triplon number might display a more homogeneous singlet pattern than the columnar valence-bond solid., Comment: 21 pages, 11 figures, final version

We calculate the bipartite von Neumann and second R\'enyi entanglement entropies of the ground states of spin-1/2 dimerized Heisenberg antiferromagnets on a square lattice. Two distinct dimerization patterns are considered: columnar and staggered. In both cases, we concentrate on the valence bond solid (VBS) phase and describe such a phase with the bond-operator representation. Within this formalism, the original spin Hamiltonian is mapped into an effective interacting boson model for the triplet excitations. We study the effective Hamiltonian at the harmonic approximation and determine the spectrum of the elementary triplet excitations. We then follow an analytical procedure, which is based on a modified spin-wave theory for finite systems and was originally employed to calculate the entanglement entropies of magnetic ordered phases, and calculate the entanglement entropies of the VBS ground states. In particular, we consider one-dimensional (line) subsystems within the square lattice, a choice that allows us to consider line subsystems with sizes up to $L' = 1000$. We combine such a procedure with the results of the bond-operator formalism at the harmonic level and show that, for both dimerized Heisenberg models, the entanglement entropies of the corresponding VBS ground states obey an area law as expected for gapped phases. For both columnar-dimer and staggered-dimer models, we also show that the entanglement entropies increase but do not diverge as the dimerization decreases and the system approaches the N\'eel--VBS quantum phase transition. Finally, the entanglement spectra associated with the VBS ground states are presented., Comment: 14 pages, 9 figures

Novelty detection is the problem of identifying whether a new data point is considered to be an inlier or an outlier. We assume that training data is available to describe only the inlier distribution. Recent approaches primarily leverage deep encoder-decoder network architectures to compute a reconstruction error that is used to either compute a novelty score or to train a one-class classifier. While we too leverage a novel network of that kind, we take a probabilistic approach and effectively compute how likely is that a sample was generated by the inlier distribution. We achieve this with two main contributions. First, we make the computation of the novelty probability feasible because we linearize the parameterized manifold capturing the underlying structure of the inlier distribution, and show how the probability factorizes and can be computed with respect to local coordinates of the manifold tangent space. Second, we improved the training of the autoencoder network. An extensive set of results show that the approach achieves state-of-the-art results on several benchmark datasets.

The quest for deeper understanding of biological systems has driven the acquisition of increasingly larger multidimensional image datasets. Inspecting and manipulating data of this complexity is very challenging in traditional visualization systems. We developed syGlass, a software package capable of visualizing large scale volumetric data with inexpensive virtual reality head-mounted display technology. This allows leveraging stereoscopic vision to significantly improve perception of complex 3D structures, and provides immersive interaction with data directly in 3D. We accomplished this by developing highly optimized data flow and volume rendering pipelines, tested on datasets up to 16TB in size, as well as tools available in a virtual reality GUI to support advanced data exploration, annotation, and cataloguing.

This work provides a framework for addressing the problem of supervised domain adaptation with deep models. The main idea is to exploit adversarial learning to learn an embedded subspace that simultaneously maximizes the confusion between two domains while semantically aligning their embedding. The supervised setting becomes attractive especially when there are only a few target data samples that need to be labeled. In this few-shot learning scenario, alignment and separation of semantic probability distributions is difficult because of the lack of data. We found that by carefully designing a training scheme whereby the typical binary adversarial discriminator is augmented to distinguish between four different classes, it is possible to effectively address the supervised adaptation problem. In addition, the approach has a high speed of adaptation, i.e. it requires an extremely low number of labeled target training samples, even one per category can be effective. We then extensively compare this approach to the state of the art in domain adaptation in two experiments: one using datasets for handwritten digit recognition, and one using datasets for visual object recognition., Comment: Accepted to NIPS 2017. arXiv admin note: text overlap with arXiv:1709.10190

This work provides a unified framework for addressing the problem of visual supervised domain adaptation and generalization with deep models. The main idea is to exploit the Siamese architecture to learn an embedding subspace that is discriminative, and where mapped visual domains are semantically aligned and yet maximally separated. The supervised setting becomes attractive especially when only few target data samples need to be labeled. In this scenario, alignment and separation of semantic probability distributions is difficult because of the lack of data. We found that by reverting to point-wise surrogates of distribution distances and similarities provides an effective solution. In addition, the approach has a high speed of adaptation, which requires an extremely low number of labeled target training samples, even one per category can be effective. The approach is extended to domain generalization. For both applications the experiments show very promising results., Comment: International Conference on Computer Vision ICCV 2017

We study the flat-band ferromagnetic phase of a topological Hubbard model within a bosonization formalism and, in particular, determine the spin-wave excitation spectrum. We consider a square lattice Hubbard model at 1/4-filling whose free-electron term is the \pi-flux model with topologically nontrivial and nearly flat energy bands. The electron spin is introduced such that the model either explicitly breaks time-reversal symmetry (correlated flat-band Chern insulator) or is invariant under time-reversal symmetry (correlated flat-band $Z_2$ topological insulator). We generalize for flat-band Chern and topological insulators the bosonization formalism [Phys. Rev. B 71, 045339 (2005)] previously developed for the two-dimensional electron gas in a uniform and perpendicular magnetic field at filling factor \nu=1. We show that, within the bosonization scheme, the topological Hubbard model is mapped into an effective interacting boson model. We consider the boson model at the harmonic approximation and show that, for the correlated Chern insulator, the spin-wave excitation spectrum is gapless while, for the correlated topological insulator, gapped. We briefly comment on the possible effects of the boson-boson (spin-wave--spin-wave) coupling., Comment: 16 pages, 5 figures

We study the plaquette valence-bond solid phase of the spin-1/2 J_1-J_2 antiferromagnet Heisenberg model on the square lattice within the bond-operator theory. We start by considering four S = 1/2 spins on a single plaquette and determine the bond operator representation for the spin operators in terms of singlet, triplet, and quintet boson operators. The formalism is then applied to the J_1-J_2 model and an effective interacting boson model in terms of singlets and triplets is derived. The effective model is analyzed within the harmonic approximation and the previous results of Zhitomirsky and Ueda [Phys. Rev. B 54, 9007 (1996)] are recovered. By perturbatively including cubic (triplet-triplet-triplet and singlet-triplet-triplet) and quartic interactions, we find that the plaquette valence-bond solid phase is stable within the parameter region 0.34 < J_2/J_1 < 0.59, which is narrower than the harmonic one. Differently from the harmonic approximation, the excitation gap vanishes at both critical couplings J_2 = 0.34 J_1 and J_2 = 0.59 J_1. Interestingly, for J_2 < 0.48 J_1, the excitation gap corresponds to a singlet-triplet excitation at the $\Gamma$ point while, for J_2 > 0.48 J_1, it is related to a singlet-singlet excitation at the X = (pi/2,0) point of the tetramerized Brillouin zone., Comment: 17 pages, 12 figures, effects quartic interactions included, final version

We study the bilayer quantum Hall system at total filling factor \nu_T = 1 within a bosonization formalism which allows us to approximately treat the magnetic exciton as a boson. We show that in the region where the distance between the two layers is comparable to the magnetic length, the ground state of the system can be seen as a finite-momentum condensate of magnetic excitons provided that the excitation spectrum is gapped. We analyze the stability of such a phase within the Bogoliubov approximation firstly assuming that only one momentum Q0 is macroscopically occupied and later we consider the same situation for two modes \pm Q0. We find strong evidences that a first-order quantum phase transition at small interlayer separation takes place from a zero-momentum condensate phase, which corresponds to Halperin 111 state, to a finite-momentum condensate of magnetic excitons., Comment: 18 pages, 11 figures, final version

Motivated by experiments on non-magnetic triangular-lattice Mott insulators, we study one candidate paramagnetic phase, the columnar dimer (or valence-bond) phase. We apply variants of the bond-operator theory to a dimerized and spatially anisotropic spin-1/2 Heisenberg model and determine its zero-temperature phase diagram and the spectrum of elementary triplet excitations (triplons). Depending on model parameters, we find that the minimum of the triplon energy is located at either a commensurate or an incommensurate wavevector. Condensation of triplons at this commensurate-incommensurate transition defines a quantum Lifshitz point, with effective dimensional reduction which possibly leads to non-trivial paramagnetic (e.g. spin-liquid) states near the closing of the triplet gap. We also discuss the two-particle decay of high-energy triplons, and we comment on the relevance of our results for the organic Mott insulator EtMe3P[Pd(dmit)2]2., Comment: 14 pages, 12 figures, new figures added, final version

In certain Mott-insulating dimerized antiferromagnets, triplet excitations of the paramagnetic phase can decay into the two-particle continuum. When such a magnet undergoes a quantum phase transition into a magnetically ordered state, this coupling becomes part of the critical theory provided that the lattice ordering wavevector is zero. One microscopic example is the staggered-dimer antiferromagnet on the square lattice, for which deviations from O(3) universality have been reported in numerical studies. Using both symmetry arguments and microscopic calculations, we show that a non-trivial cubic term arises in the relevant order-parameter quantum field theory, and assess its consequences using a combination of analytical and numerical methods. We also present finite-temperature quantum Monte Carlo data for the staggered-dimer antiferromagnet which complement recently published results. The data can be consistently interpreted in terms of critical exponents identical to that of the standard O(3) universality class, but with anomalously large corrections to scaling. We argue that the two-particle decay of critical triplons, although irrelevant in two spatial dimensions, is responsible for the leading corrections to scaling due to its small scaling dimension., Comment: 14 pages, 7 figs

In magnets with strong quantum fluctuations, paramagnetic ground states with or without confinement of spinon excitations can be realized. Here we discuss the physics of the confined phase in the vicinity of a confinement--deconfinement transition: This regime, likely relevant to a multitude of frustrated spin systems, is characterized by multiple length scales: In addition to the magnetic correlation length, a confinement length can be defined, which can be probed, e.g., by local static measurements near non-magnetic impurities. We illustrate the ideas by explicit calculations for dimerized spin chains, but our qualitative results remain valid in higher dimensions as well. In particular, we study crossover from weak to strong confinement visible in the antiferromagnetic polarization cloud around a non-magnetic impurity. We also discuss the effective magnetic interaction between impurities, relevant for impurity-induced magnetic order, and consequences for nuclear magnetic resonance and neutron scattering experiments., Comment: 16 pages, 15 figures, references added, final version

We study the quantum Hall effect in graphene at filling factors \nu = 0 and \nu = \pm, concentrating on the quantum Hall ferromagnetic regime, within a non-perturbative bosonization formalism. We start by developing a bosonization scheme for electrons with two discrete degrees of freedom (spin-1/2 and pseudospin-1/2) restricted to the lowest Landau level. Three distinct phases are considered, namely the so-called spin-pseudospin, spin, and pseudospin phases. The first corresponds to a quarter-filled (\nu =-1) while the others to a half-filled (\nu = 0) lowest Landau level. In each case, we show that the elementary neutral excitations can be treated approximately as a set of n-independent kinds of boson excitations. The boson representation of the projected electron density, the spin, pseudospin, and mixed spin-pseudospin density operators are derived. We then apply the developed formalism to the effective continuous model, which includes SU(4) symmetry breaking terms, recently proposed by Alicea and Fisher. For each quantum Hall state, an effective interacting boson model is derived and the dispersion relations of the elementary excitations are analytically calculated. We propose that the charged excitations (quantum Hall skyrmions) can be described as a coherent state of bosons. We calculate the semiclassical limit of the boson model derived from the SU(4) invariant part of the original fermionic Hamiltonian and show that it agrees with the results of Arovas and co-workers for SU(N) quantum Hall skyrmions. We briefly discuss the influence of the SU(4) symmetry breaking terms in the skyrmion energy., Comment: 16 pages, 4 figures, final version, extended discussion about the boson-boson interaction and its relation with quantum Hall skyrmions

We develop a non-perturbative bosonization approach for bilayer quantum Hall systems at \nu_T = 1, which allows us to systematically study the existence of an exciton condensate in these systems. An effective boson model is derived and the excitation spectrum is calculated both in the Bogoliubov and in the Popov approximations. In the latter case, we show that the ground state of the system is an exciton condensate only when the distance between the layers is very small compared to the magnetic length, indicating that the system possibly undergoes another phase transition before the incompressible-compressible one. The effect of a finite electron interlayer tunnelling is included and a quantitative phase diagram is proposed., Comment: 4 pages, 3 figures, the effect of a finite electron interlayer tunnelling is included

The spin-excitations of a fractional quantum Hall system are evaluated within a bosonization approach. In a first step, we generalize Murthy and Shankar's Hamiltonian theory of the fractional quantum Hall effect to the case of composite fermions with an extra discrete degree of freedom. Here, we mainly investigate the spin degrees of freedom, but the proposed formalism may be useful also in the study of bilayer quantum-Hall systems, where the layer index may formally be treated as an isospin. In a second step, we apply a bosonization scheme, recently developed for the study of the two-dimensional electron gas, to the interacting composite-fermion Hamiltonian. The dispersion of the bosons, which represent quasiparticle-quasihole excitations, is analytically evaluated for fractional quantum Hall systems at \nu = 1/3 and \nu = 1/5. The finite width of the two-dimensional electron gas is also taken into account explicitly. In addition, we consider the interacting bosonic model and calculate the lowest-energy state for two bosons. Besides a continuum describing scattering states, we find a bound-state of two bosons. This state is interpreted as a pair excitation, which consists of a skyrmion of composite fermions and an antiskyrmion of composite fermions. The dispersion relation of the two-boson state is evaluated for \nu = 1/3 and \nu = 1/5. Finally, we show that our theory provides the microscopic basis for a phenomenological non-linear sigma-model for studying the skyrmion of composite fermions., Comment: Revised version, 14 pages, 4 figures, accepted to Phys. Rev. B

We report on the theoretical photoluminescence spectrum of the interacting two-dimensional electron gas at filling factor one (\nu=1). We considered a model similar to the one adopted to study the X-ray spectra of metals and solved it analytically using the bosonization method previously developed for the two-dimensional electron gas at \nu=1. We calculated the emission spectra of the right and the left circularly polarized radiations for the situations where the distance between the two-dimensional electron gas and the valence band hole are smaller and greater than the magnetic length. For the former, we showed that the polarized photoluminescence spectra can be understood as the recombination of the so-called excitonic state with the valence band hole whereas, for the latter, the observed emission spectra can be related to the recombination of a state formed by a spin down electron bound to n spin waves. This state seems to be a good description for the quantum Hall skyrmion., Comment: Revised version, 10 pages, 5 figures, accepted to Phys. Rev. B

The non-monotonic behavior of the NMR signal linewidth in the 2D quantum Hall system is explained in terms of the interplay between skyrmions localization, due to the influence of disorder, and the non-trivial temperature dependent skyrmion dynamics., Comment: 5 pages, 2 figures

We develop a bosonization scheme for the two-dimensional electron gas in the presence of an uniform magnetic field perpendicular to the two-dimensional system when the filling factor \nu = 1. We show that the elementary neutral excitations of this system, known as magnetic excitons, can be treated approximately as bosons and we apply the method to the interacting system. We show that the Hamiltonian of the fermionic system is mapped into an interacting bosonic Hamiltonian, where the dispersion relation of the bosons agrees with previous RPA calculations. The interaction term accounts for the formation of two-boson bound states. We discuss a possible relation between these excitations and the skyrmion-antiskyrmion pair, in analogy with the ferromagnetic Heisenberg model. Finally, we analyze the semiclassical limit of the interacting bosonic Hamiltonian and show that the results are in agreement with those derived from the model of Sondhi {\it et al.} for the quantum Hall skyrmion., Comment: Revised version, new figures