Search

As various property studies continue to emerge on high entropy and entropy-stabilized ceramics, we seek further understanding of property changes across the phase boundary between \enquote{high-entropy} and \enquote{entropy-stabilized}. The thermal and mechanical properties of bulk ceramic entropy stabilized oxide composition Mg$_{0.2}$Co$_{0.2}$Ni$_{0.2}$Cu$_{0.2}$Zn$_{0.2}$O are investigated across this critical transition temperature via the transient plane-source method, temperature-dependent X-ray diffraction, and nano-indentation. Thermal conductivity remains constant within uncertainty across the multi-to-single phase transition at a value of ~2.5 W/mK, while the linear coefficient of thermal expansion increases nearly 24 % from 10.8 to 14.1 x 10$^{-6}$ K$^{-1}$. Mechanical softening is also observed across the transition., Comment: 14 pages, 4 figures, to be published in APL Materials

Most video-and-language representation learning approaches employ contrastive learning, e.g., CLIP, to project the video and text features into a common latent space according to the semantic similarities of text-video pairs. However, such learned shared latent spaces are not often optimal, and the modality gap between visual and textual representation can not be fully eliminated. In this paper, we propose Expectation-Maximization Contrastive Learning (EMCL) to learn compact video-and-language representations. Specifically, we use the Expectation-Maximization algorithm to find a compact set of bases for the latent space, where the features could be concisely represented as the linear combinations of these bases. Such feature decomposition of video-and-language representations reduces the rank of the latent space, resulting in increased representing power for the semantics. Extensive experiments on three benchmark text-video retrieval datasets prove that our EMCL can learn more discriminative video-and-language representations than previous methods, and significantly outperform previous state-of-the-art methods across all metrics. More encouragingly, the proposed method can be applied to boost the performance of existing approaches either as a jointly training layer or an out-of-the-box inference module with no extra training, making it easy to be incorporated into any existing methods., Comment: Accepted to NeurIPS 2022

Artificial Intelligence (AI) has the potential to significantly benefit or harm humanity. At present, a few for-profit companies largely control the development and use of this technology, and therefore determine its outcomes. In an effort to diversify and democratize work on AI, various groups are building open AI systems, investigating their risks, and discussing their ethics. In this paper, we demonstrate how blockchain technology can facilitate and formalize these efforts. Concretely, we analyze multiple use-cases for blockchain in AI research and development, including decentralized governance, the creation of immutable audit trails, and access to more diverse and representative datasets. We argue that decentralizing AI can help mitigate AI risks and ethical concerns, while also introducing new issues that should be considered in future work.

Language models (LMs), including large language models (such as ChatGPT), have the potential to assist clinicians in generating various clinical notes. However, LMs are prone to produce ``hallucinations'', i.e., generated content that is not aligned with facts and knowledge. In this paper, we propose the Re$^3$Writer method with retrieval-augmented generation and knowledge-grounded reasoning to enable LMs to generate faithful clinical texts. We demonstrate the effectiveness of our method in generating patient discharge instructions. It requires the LMs not to only understand the patients' long clinical documents, i.e., the health records during hospitalization, but also to generate critical instructional information provided both to carers and to the patient at the time of discharge. The proposed Re$^3$Writer imitates the working patterns of physicians to first \textbf{re}trieve related working experience from historical instructions written by physicians, then \textbf{re}ason related medical knowledge. Finally, it \textbf{re}fines the retrieved working experience and reasoned medical knowledge to extract useful information, which is used to generate the discharge instructions for previously-unseen patients. Our experiments show that, using our method, the performance of five representative LMs can be substantially boosted across all metrics. Meanwhile, we show results from human evaluations to measure the effectiveness in terms of fluency, faithfulness, and comprehensiveness.

Observational studies have recently received significant attention from the machine learning community due to the increasingly available non-experimental observational data and the limitations of the experimental studies, such as considerable cost, impracticality, small and less representative sample sizes, etc. In observational studies, de-confounding is a fundamental problem of individualised treatment effects (ITE) estimation. This paper proposes disentangled representations with adversarial training to selectively balance the confounders in the binary treatment setting for the ITE estimation. The adversarial training of treatment policy selectively encourages treatment-agnostic balanced representations for the confounders and helps to estimate the ITE in the observational studies via counterfactual inference. Empirical results on synthetic and real-world datasets, with varying degrees of confounding, prove that our proposed approach improves the state-of-the-art methods in achieving lower error in the ITE estimation., Comment: accepted to AISTATS 2023

Processing information locked within clinical health records is a challenging task that remains an active area of research in biomedical NLP. In this work, we evaluate a broad set of machine learning techniques ranging from simple RNNs to specialised transformers such as BioBERT on a dataset containing clinical notes along with a set of annotations indicating whether a sample is cancer-related or not. Furthermore, we specifically employ efficient fine-tuning methods from NLP, namely, bottleneck adapters and prompt tuning, to adapt the models to our specialised task. Our evaluations suggest that fine-tuning a frozen BERT model pre-trained on natural language and with bottleneck adapters outperforms all other strategies, including full fine-tuning of the specialised BioBERT model. Based on our findings, we suggest that using bottleneck adapters in low-resource situations with limited access to labelled data or processing capacity could be a viable strategy in biomedical text mining. The code used in the experiments are going to be made available at https://github.com/omidrohanian/bottleneck-adapters.

Pre-trained Language Models (LMs) have become an integral part of Natural Language Processing (NLP) in recent years, due to their superior performance in downstream applications. In spite of this resounding success, the usability of LMs is constrained by computational and time complexity, along with their increasing size; an issue that has been referred to as `overparameterisation'. Different strategies have been proposed in the literature to alleviate these problems, with the aim to create effective compact models that nearly match the performance of their bloated counterparts with negligible performance losses. One of the most popular techniques in this area of research is model distillation. Another potent but underutilised technique is cross-layer parameter sharing. In this work, we combine these two strategies and present MiniALBERT, a technique for converting the knowledge of fully parameterised LMs (such as BERT) into a compact recursive student. In addition, we investigate the application of bottleneck adapters for layer-wise adaptation of our recursive student, and also explore the efficacy of adapter tuning for fine-tuning of compact models. We test our proposed models on a number of general and biomedical NLP tasks to demonstrate their viability and compare them with the state-of-the-art and other existing compact models. All the codes used in the experiments are available at https://github.com/nlpie-research/MiniALBERT. Our pre-trained compact models can be accessed from https://huggingface.co/nlpie.

The Reconstruction Conjecture due to Kelly and Ulam states that every graph with at least 3 vertices is uniquely determined by its multiset of subgraphs $\{G-v: v\in V(G)\}$. Let $diam(G)$ and $\kappa(G)$ denote the diameter and the connectivity of a graph $G$, respectively, and let $\mathcal{G}_2:=\{G: \textrm{diam}(G)=2\}$ and $\mathcal{G}_3:=\{G:\textrm{diam}(G)=\textrm{diam}(\overline{G})=3\}$. It is known that the Reconstruction Conjecture is true if and only if it is true for every 2-connected graph in $\mathcal{G}_2\cup \mathcal{G}_3$. Balakumar and Monikandan showed that the Reconstruction Conjecture holds for every triangle-free graph $G$ in $\mathcal{G}_2\cup \mathcal{G}_3$ with $\kappa(G)=2$. Moreover, they asked whether the result still holds if $\kappa(G)\ge 3$. (If yes, the class of graphs critical for solving the Reconstruction Conjecture is restricted to 2-connected graphs in $\mathcal{G}_2\cup\mathcal{G}_3$ which contain triangles.) In this paper, we give a partial solution to their question by showing that the Reconstruction Conjecture holds for every triangle-free graph $G$ in $\mathcal{G}_3$ and every triangle-free graph $G$ in $\mathcal{G}_2$ with $\kappa(G)=3$. We also prove similar results about the Edge Reconstruction Conjecture., Comment: 11 pages, 3 figures

Purpose: This study aimed to use deep learning-based dose prediction to assess head and neck (HN) plan quality and identify suboptimal plans. Methods: A total of 245 VMAT HN plans were created using RapidPlan knowledge-based planning (KBP). A subset of 112 high-quality plans was selected under the supervision of an HN radiation oncologist. We trained a 3D Dense Dilated U-Net architecture to predict 3-dimensional dose distributions using 3-fold cross-validation on 90 plans. Model inputs included CT images, target prescriptions, and contours for targets and organs at risk (OARs). The model's performance was assessed on the remaining 22 test plans. We then tested the application of the dose prediction model for automated review of plan quality. Dose distributions were predicted on 14 clinical plans. The predicted versus clinical OAR dose metrics were compared to flag OARs with suboptimal normal tissue sparing using a 2 Gy dose difference or 3% dose-volume threshold. OAR flags were compared to manual flags by 3 HN radiation oncologists. Results: The predicted dose distributions were of comparable quality to the KBP plans. The differences between the predicted and KBP-planned D1%, D95%, and D99% across the targets were within -2.53%(SD=1.34%), -0.42%(SD=1.27%), and -0.12%(SD=1.97%), respectively, and the OAR mean and maximum doses were within -0.33Gy(SD=1.40Gy) and -0.96Gy(SD=2.08Gy). For the plan quality assessment study, radiation oncologists flagged 47 OARs for possible plan improvement. There was high interphysician variability; 83% of physician-flagged OARs were flagged by only one of 3 physicians. The comparative dose prediction model flagged 63 OARs, including 30 of 47 physician-flagged OARs. Conclusion: Deep learning can predict high-quality dose distributions, which can be used as comparative dose distributions for automated, individualized assessment of HN plan quality., Comment: updated to reflect the published peer-reviewed article

Recent studies on contrastive learning have achieved remarkable performance solely by leveraging few labels in the context of medical image segmentation. Existing methods mainly focus on instance discrimination and invariant mapping. However, they face three common pitfalls: (1) tailness: medical image data usually follows an implicit long-tail class distribution. Blindly leveraging all pixels in training hence can lead to the data imbalance issues, and cause deteriorated performance; (2) consistency: it remains unclear whether a segmentation model has learned meaningful and yet consistent anatomical features due to the intra-class variations between different anatomical features; and (3) diversity: the intra-slice correlations within the entire dataset have received significantly less attention. This motivates us to seek a principled approach for strategically making use of the dataset itself to discover similar yet distinct samples from different anatomical views. In this paper, we introduce a novel semi-supervised 2D medical image segmentation framework termed Mine yOur owN Anatomy (MONA), and make three contributions. First, prior work argues that every pixel equally matters to the model training; we observe empirically that this alone is unlikely to define meaningful anatomical features, mainly due to lacking the supervision signal. We show two simple solutions towards learning invariances - through the use of stronger data augmentations and nearest neighbors. Second, we construct a set of objectives that encourage the model to be capable of decomposing medical images into a collection of anatomical features in an unsupervised manner. Lastly, we both empirically and theoretically, demonstrate the efficacy of our MONA on three benchmark datasets with different labeled settings, achieving new state-of-the-art under different labeled semi-supervised settings., Comment: Accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence (IEEE-TPAMI)

In this paper we describe the Portuguese-language podcast dataset we have released for academic research purposes. We give an overview of how the data was sampled, descriptive statistics over the collection, as well as information about the distribution over Brazilian and Portuguese dialects. We give results from experiments on multi-lingual summarization, showing that summarizing podcast transcripts can be performed well by a system supporting both English and Portuguese. We also show experiments on Portuguese podcast genre classification using text metadata. Combining this collection with previously released English-language collection opens up the potential for multi-modal, multi-lingual and multi-dialect podcast information access research., Comment: 12 pages, 1 figure