Search

The widespread adoption of Large Language Models (LLMs) has facilitated numerous benefits. However, hallucination is a significant concern. In response, Retrieval Augmented Generation (RAG) has emerged as a highly promising paradigm to improve LLM outputs by grounding them in factual information. RAG relies on textual entailment (TE) or similar methods to check if the text produced by LLMs is supported or contradicted, compared to retrieved documents. This paper argues that conventional TE methods are inadequate for spotting hallucinations in content generated by LLMs. For instance, consider a prompt about the 'USA's stance on the Ukraine war''. The AI-generated text states, ...U.S. President Barack Obama says the U.S. will not put troops in Ukraine...'' However, during the war the U.S. president is Joe Biden which contradicts factual reality. Moreover, current TE systems are unable to accurately annotate the given text and identify the exact portion that is contradicted. To address this, we introduces a new type of TE called ``Factual Entailment (FE).'', aims to detect factual inaccuracies in content generated by LLMs while also highlighting the specific text segment that contradicts reality. We present FACTOID (FACTual enTAILment for hallucInation Detection), a benchmark dataset for FE. We propose a multi-task learning (MTL) framework for FE, incorporating state-of-the-art (SoTA) long text embeddings such as e5-mistral-7b-instruct, along with GPT-3, SpanBERT, and RoFormer. The proposed MTL architecture for FE achieves an avg. 40\% improvement in accuracy on the FACTOID benchmark compared to SoTA TE methods. As FE automatically detects hallucinations, we assessed 15 modern LLMs and ranked them using our proposed Auto Hallucination Vulnerability Index (HVI_auto). This index quantifies and offers a comparative scale to evaluate and rank LLMs according to their hallucinations.

The troubling rise of hallucination presents perhaps the most significant impediment to the advancement of responsible AI. In recent times, considerable research has focused on detecting and mitigating hallucination in Large Language Models (LLMs). However, it's worth noting that hallucination is also quite prevalent in Vision-Language models (VLMs). In this paper, we offer a fine-grained discourse on profiling VLM hallucination based on two tasks: i) image captioning, and ii) Visual Question Answering (VQA). We delineate eight fine-grained orientations of visual hallucination: i) Contextual Guessing, ii) Identity Incongruity, iii) Geographical Erratum, iv) Visual Illusion, v) Gender Anomaly, vi) VLM as Classifier, vii) Wrong Reading, and viii) Numeric Discrepancy. We curate Visual HallucInation eLiciTation (VHILT), a publicly available dataset comprising 2,000 samples generated using eight VLMs across two tasks of captioning and VQA along with human annotations for the categories as mentioned earlier.

Electrical current rectification is an interesting electronic feature, popularly known as a diode. Achieving a high rectification ratio in a molecular junction has been a long-standing goal in molecular electronics. The present work describes mimicking electrical current rectification with pi-stacked rhenium(I) compound sandwiched between two electrical contacts. Among the two mononuclear rhenium compounds studied here, [Re(CO)4(PPh3){(N)-saccharinate}] (1) and [Re(CO)3(phen){(N)-saccharinate}] (2), the latter show strong pi-pi interactions-induced high rectification ratio of ~ 4000 at 2.0 V at room temperature. Alternating current (AC)-based electrical measurements ensuring AC to DC electrical signal conversion at a frequency f of 1 KHz showing 2 can act as an excellent half-wave rectifier. Asymmetric charge injection barrier height at the electrode/Re(I) interfaces of the devices with a stacking configuration of p++-Si/Re compound31nm(2)/ITO originates the flow of electrical current unidirectionally. The charge transport mechanism governed by thermally activated hopping phenomena, and charge carrier propagation is explained through an energy profile considering the Fermi levels of two electrodes, and the energy of frontier molecular orbitals, HOMO, and LUMO, confirming rectification is of a molecular origin. The present work paves the way to combine different organometallic compounds as circuit elements in nanoelectronic devices to achieve numerous exciting electronic features., Comment: 16 pages, 5 figures