Search

Cutaneous squamous cell cancer (cSCC) is the second most common skin cancer in the US. It is diagnosed by manual multi-class tumor grading using a tissue whole slide image (WSI), which is subjective and suffers from inter-pathologist variability. We propose an automated weakly-supervised grading approach for cSCC WSIs that is trained using WSI-level grade and does not require fine-grained tumor annotations. The proposed model, RACR-MIL, transforms each WSI into a bag of tiled patches and leverages attention-based multiple-instance learning to assign a WSI-level grade. We propose three key innovations to address general as well as cSCC-specific challenges in tumor grading. First, we leverage spatial and semantic proximity to define a WSI graph that encodes both local and non-local dependencies between tumor regions and leverage graph attention convolution to derive contextual patch features. Second, we introduce a novel ordinal ranking constraint on the patch attention network to ensure that higher-grade tumor regions are assigned higher attention. Third, we use tumor depth as an auxiliary task to improve grade classification in a multitask learning framework. RACR-MIL achieves 2-9% improvement in grade classification over existing weakly-supervised approaches on a dataset of 718 cSCC tissue images and localizes the tumor better. The model achieves 5-20% higher accuracy in difficult-to-classify high-risk grade classes and is robust to class imbalance., Comment: 7 pages main text, 2 page references, 3 page appendix; submitted to AAAI

Pre-trained foundation models, due to their enormous capacity and exposure to vast amounts of data during pre-training, are known to have learned plenty of real-world concepts. An important step in making these pre-trained models effective on downstream tasks is to fine-tune them on related datasets. While various fine-tuning methods have been devised and have been shown to be highly effective, we observe that a fine-tuned model's ability to recognize concepts on tasks $\textit{different}$ from the downstream one is reduced significantly compared to its pre-trained counterpart. This is an undesirable effect of fine-tuning as a substantial amount of resources was used to learn these pre-trained concepts in the first place. We call this phenomenon ''concept forgetting'' and via experiments show that most end-to-end fine-tuning approaches suffer heavily from this side effect. To this end, we propose a simple fix to this problem by designing a new fine-tuning method called $\textit{LDIFS}$ (short for $\ell_2$ distance in feature space) that, while learning new concepts related to the downstream task, allows a model to preserve its pre-trained knowledge as well. Through extensive experiments on 10 fine-tuning tasks we show that $\textit{LDIFS}$ significantly reduces concept forgetting. Additionally, we show that LDIFS is highly effective in performing continual fine-tuning on a sequence of tasks as well, in comparison with both fine-tuning as well as continual learning baselines., Comment: Published in TMLR: https://openreview.net/forum?id=kfhoeZCeW7

Excitons -- bound electron-hole pairs -- play a central role in light-matter interaction phenomena, and are crucial for wide-ranging applications from light harvesting and generation to quantum information processing. A long-standing challenge in solid-state optics has been to achieve precise and scalable control over the quantum mechanical state of excitons in semiconductor heterostructures. Here, we demonstrate a technique for creating tailored and tunable potential landscapes for optically active excitons in 2D semiconductors that enables in-situ wavefunction shaping at the nanoscopic lengthscale. Using nanostructured gate electrodes, we create localized electrostatic traps for excitons in diverse geometries such as quantum dots and rings, and arrays thereof. We show independent spectral tuning of multiple spatially separated quantum dots, which allows us to bring them to degeneracy despite material disorder. Owing to the strong light-matter coupling of excitons in 2D semiconductors, we observe unambiguous signatures of confined exciton wavefunctions in optical reflection and photoluminescence measurements. Our work introduces a new approach to engineering exciton dynamics and interactions at the nanometer scale, with implications for novel optoelectronic devices, topological photonics, and many-body quantum nonlinear optics.

Food classification is an important task in health care. In this work, we propose a multimodal classification framework that uses the modified version of EfficientNet with the Mish activation function for image classification, and the traditional BERT transformer-based network is used for text classification. The proposed network and the other state-of-the-art methods are evaluated on a large open-source dataset, UPMC Food-101. The experimental results show that the proposed network outperforms the other methods, a significant difference of 11.57% and 6.34% in accuracy is observed for image and text classification, respectively, when compared with the second-best performing method. We also compared the performance in terms of accuracy, precision, and recall for text classification using both machine learning and deep learning-based models. The comparative analysis from the prediction results of both images and text demonstrated the efficiency and robustness of the proposed approach., Comment: Updated results and discussions to be posted and some sections needed to be expanded

Query auto-completion (QAC) aims to suggest plausible completions for a given query prefix. Traditionally, QAC systems have leveraged tries curated from historical query logs to suggest most popular completions. In this context, there are two specific scenarios that are difficult to handle for any QAC system: short prefixes (which are inherently ambiguous) and unseen prefixes. Recently, personalized Natural Language Generation (NLG) models have been proposed to leverage previous session queries as context for addressing these two challenges. However, such NLG models suffer from two drawbacks: (1) some of the previous session queries could be noisy and irrelevant to the user intent for the current prefix, and (2) NLG models cannot directly incorporate historical query popularity. This motivates us to propose a novel NLG model for QAC, Trie-NLG, which jointly leverages popularity signals from trie and personalization signals from previous session queries. We train the Trie-NLG model by augmenting the prefix with rich context comprising of recent session queries and top trie completions. This simple modeling approach overcomes the limitations of trie-based and NLG-based approaches and leads to state-of-the-art performance. We evaluate the Trie-NLG model using two large QAC datasets. On average, our model achieves huge ~57% and ~14% boost in MRR over the popular trie-based lookup and the strong BART-based baseline methods, respectively. We make our code publicly available., Comment: ECML-PKDD 2023 (Journal Track)

The autocorrelation function in many complex systems shows a crossover in the form of its decay: from stretched exponential relaxation (SER) at short times to power law at long times. Studies of the mechanisms leading to such multiple relaxation patterns are rare. Additionally, the inherent complexity of these systems makes it hard to understand the underlying mechanism leading to the crossover. Here we develop a simple one-dimensional spin model, which we call a Domain Wall (DW) to Doublon model, that shows such a crossover as the nature of the excitations governing the relaxation dynamics changes with temperature and time. The relevant excitations are DWs and bound pairs of DWs, which we term `doublons'. The diffusive motion of the DWs govern the relaxation at short times, whereas the diffusive motion of the doublons yields the long time decay. This change of excitations and their relaxation leads to a crossover from SER to power law in the decay pattern of the autocorrelation function. We augment our numerical results with simple physical arguments and analytic derivations.

The current approach for testing the robustness of object detectors suffers from serious deficiencies such as improper methods of performing out-of-distribution detection and using calibration metrics which do not consider both localisation and classification quality. In this work, we address these issues, and introduce the Self-Aware Object Detection (SAOD) task, a unified testing framework which respects and adheres to the challenges that object detectors face in safety-critical environments such as autonomous driving. Specifically, the SAOD task requires an object detector to be: robust to domain shift; obtain reliable uncertainty estimates for the entire scene; and provide calibrated confidence scores for the detections. We extensively use our framework, which introduces novel metrics and large scale test datasets, to test numerous object detectors in two different use-cases, allowing us to highlight critical insights into their robustness performance. Finally, we introduce a simple baseline for the SAOD task, enabling researchers to benchmark future proposed methods and move towards robust object detectors which are fit for purpose. Code is available at https://github.com/fiveai/saod, Comment: CVPR 2023

This paper aims to demonstrate the importance and feasibility of fusing multimodal information for emotion recognition. It introduces a multimodal framework for emotion understanding by fusing the information from visual facial features and rPPG signals extracted from the input videos. An interpretability technique based on permutation feature importance analysis has also been implemented to compute the contributions of rPPG and visual modalities toward classifying a given input video into a particular emotion class. The experiments on IEMOCAP dataset demonstrate that the emotion classification performance improves by combining the complementary information from multiple modalities., Comment: Accepted for Oral Presentation in DAI 2023 (https://rbcdsai.iitm.ac.in/DAI-2023/program.html)

Empirical evidence on the effectiveness of performance-based rewards for teachers is primarily based on the evaluation of monetary reward schemes. We present results from a randomized evaluation of a teacher and principal incentive programme in India that offered a non-pecuniary recognition reward based on students' test scores on standardized assessments. We find a positive (0.16 SD) yet statistically insignificant effect on student performance when both teachers and principals are incentivized. In schools where only teachers are incentivized, the estimates remain statistically insignificant but are also much smaller in magnitude (0.012 SD). Our findings provide suggestive evidence that recognition rewards may have the potential as a low-cost tool to improve student achievement when both teachers and principals are incentivized. However, further research is required to substantiate the findings and investigate the mechanisms at play. [This is the online version of an article published in "Applied Economic Letters."]

Despite the enormous theoretical and application interests, a fundamental understanding of the glassy dynamics remains elusive. The static properties of glassy and ordinary liquids are similar, but their dynamics are dramatically different. What leads to this difference is the central puzzle of the field. Even the primary defining glassy characteristics, their implications, and if they are related to a single mechanism remain unclear. This lack of clarity is a severe hindrance to theoretical progress. Here, we combine analytical arguments and simulations of various systems in different dimensions and address these questions. Our results suggest that the myriad of glassy features are manifestations of two distinct mechanisms. Particle caging controls the mean, and coexisting slow- and fast-moving regions govern the distribution of particle displacements. All the other glassy characteristics are manifestations of these two mechanisms; thus, the Fickian yet non-Gaussian nature of glassy liquids is not surprising. Our results have crucial implications on how the glassy dynamics data are analyzed, challenge some recent suggestions on the mechanisms governing gassy dynamics, and impose strict constraints that a correct theory of glasses must have.

Transformers for graph data are increasingly widely studied and successful in numerous learning tasks. Graph inductive biases are crucial for Graph Transformers, and previous works incorporate them using message-passing modules and/or positional encodings. However, Graph Transformers that use message-passing inherit known issues of message-passing, and differ significantly from Transformers used in other domains, thus making transfer of research advances more difficult. On the other hand, Graph Transformers without message-passing often perform poorly on smaller datasets, where inductive biases are more crucial. To bridge this gap, we propose the Graph Inductive bias Transformer (GRIT) -- a new Graph Transformer that incorporates graph inductive biases without using message passing. GRIT is based on several architectural changes that are each theoretically and empirically justified, including: learned relative positional encodings initialized with random walk probabilities, a flexible attention mechanism that updates node and node-pair representations, and injection of degree information in each layer. We prove that GRIT is expressive -- it can express shortest path distances and various graph propagation matrices. GRIT achieves state-of-the-art empirical performance across a variety of graph datasets, thus showing the power that Graph Transformers without message-passing can deliver., Comment: Published as a conference paper at ICML 2023; 17 pages

Traditional online continual learning (OCL) research has primarily focused on mitigating catastrophic forgetting with fixed and limited storage allocation throughout an agent's lifetime. However, a broad range of real-world applications are primarily constrained by computational costs rather than storage limitations. In this paper, we target such applications, investigating the online continual learning problem under relaxed storage constraints and limited computational budgets. We contribute a simple algorithm, which updates a kNN classifier continually along with a fixed, pretrained feature extractor. We selected this algorithm due to its exceptional suitability for online continual learning. It can adapt to rapidly changing streams, has zero stability gap, operates within tiny computational budgets, has low storage requirements by only storing features, and has a consistency property: It never forgets previously seen data. These attributes yield significant improvements, allowing our proposed algorithm to outperform existing methods by over 20% in accuracy on two large-scale OCL datasets: Continual LOCalization (CLOC) with 39M images and 712 classes and Continual Google Landmarks V2 (CGLM) with 580K images and 10,788 classes, even when existing methods retain all previously seen images. Furthermore, we achieve this superior performance with considerably reduced computational and storage expenses. We provide code to reproduce our results at github.com/drimpossible/ACM., Comment: Tech Report [Additional Experiments and Improved ACM]

Approximate computing methods have shown great potential for deep learning. Due to the reduced hardware costs, these methods are especially suitable for inference tasks on battery-operated devices that are constrained by their power budget. However, approximate computing hasn't reached its full potential due to the lack of work on training methods. In this work, we discuss training methods for approximate hardware. We demonstrate how training needs to be specialized for approximate hardware, and propose methods to speed up the training process by up to 18X.

Pulmonary Embolisms (PE) represent a leading cause of cardiovascular death. While medical imaging, through computed tomographic pulmonary angiography (CTPA), represents the gold standard for PE diagnosis, it is still susceptible to misdiagnosis or significant diagnosis delays, which may be fatal for critical cases. Despite the recently demonstrated power of deep learning to bring a significant boost in performance in a wide range of medical imaging tasks, there are still very few published researches on automatic pulmonary embolism detection. Herein we introduce a deep learning based approach, which efficiently combines computer vision and deep neural networks for pulmonary embolism detection in CTPA. Our method features novel improvements along three orthogonal axes: 1) automatic detection of anatomical structures; 2) anatomical aware pretraining, and 3) a dual-hop deep neural net for PE detection. We obtain state-of-the-art results on the publicly available multicenter large-scale RSNA dataset., Comment: Accepted to Computers in Biology and Medicine journal

Continual Learning (CL) aims to sequentially train models on streams of incoming data that vary in distribution by preserving previous knowledge while adapting to new data. Current CL literature focuses on restricted access to previously seen data, while imposing no constraints on the computational budget for training. This is unreasonable for applications in-the-wild, where systems are primarily constrained by computational and time budgets, not storage. We revisit this problem with a large-scale benchmark and analyze the performance of traditional CL approaches in a compute-constrained setting, where effective memory samples used in training can be implicitly restricted as a consequence of limited computation. We conduct experiments evaluating various CL sampling strategies, distillation losses, and partial fine-tuning on two large-scale datasets, namely ImageNet2K and Continual Google Landmarks V2 in data incremental, class incremental, and time incremental settings. Through extensive experiments amounting to a total of over 1500 GPU-hours, we find that, under compute-constrained setting, traditional CL approaches, with no exception, fail to outperform a simple minimal baseline that samples uniformly from memory. Our conclusions are consistent in a different number of stream time steps, e.g., 20 to 200, and under several computational budgets. This suggests that most existing CL methods are particularly too computationally expensive for realistic budgeted deployment. Code for this project is available at: https://github.com/drimpossible/BudgetCL., Comment: CVPR 2023

Achieving strong interactions in fermionic many-body systems is a major theme of research in condensed matter physics. It is well-known that interactions between fermions can be mediated through a bosonic medium, such as a phonon bath or Bose-Einstein condensate (BEC). Here, we show that such induced attraction can be resonantly enhanced when the bosonic medium is a two-component spinor BEC. The strongest interaction is achieved by tuning the boson-boson scattering to the quantum critical spinodal point of the BEC where the sound velocity vanishes. The fermion pairing gap and the superconducting critical temperature can thus be dramatically enhanced. We propose two experimental realizations of this scenario, with exciton-polariton systems in two-dimensional semiconductors and ultracold atomic Bose-Fermi mixtures., Comment: 8 pages, 3 figures