Search

Large language models (LLMs) are highly effective in various natural language processing (NLP) tasks. However, they are susceptible to producing unreliable conjectures in ambiguous contexts called hallucination. This paper presents a new method for evaluating LLM hallucination in Question Answering (QA) based on the unanswerable math word problem (MWP). To support this approach, we innovatively develop a dataset called Unanswerable Math Word Problem (UMWP) which comprises 5200 questions across five categories. We developed an evaluation methodology combining text similarity and mathematical expression detection to determine whether LLM considers the question unanswerable. The results of extensive experiments conducted on 31 LLMs, including GPT-3, InstructGPT, LLaMA, and Claude, demonstrate that in-context learning and reinforcement learning with human feedback (RLHF) training significantly enhance the model's ability to avoid hallucination. We show that utilizing MWP is a reliable and effective approach to assess hallucination. Our code and data are available at https://github.com/Yuki-Asuuna/UMWP., Comment: 11 pages, 8 figures, accepted by LREC-Coling 2024

Wastewater treatment plant (WWTP) effluent discharge could induce the resistome enrichment in the receiving water environments. However, because of the general lack of a robust antibiotic-resistant bacteria (ARB) identification method, the driving mechanism for resistome accumulation in receiving environment is unclear. Here, we took advantage of the enhanced ARBs recognition by nanopore long reads to distinguish the indigenous ARBs and the accumulation of WWTP-borne ARBs in the receiving water body of a domestic WWTP. A bioinformatic framework (named ARGpore2: https://github.com/sustc-xylab/ARGpore2) was constructed and evaluate to facilitate antibiotic resistance genes (ARGs) and ARBs identification in nanopore reads. ARGs identification by ARGpore2 showed comparable precision and recall to that of the commonly adopt BLASTP-based method, whereas the spectrum of ARBs doubled that of the assembled Illumina dataset. Totally, we identified 33 ARBs genera carrying 65 ARG subtypes in the receiving seawater, whose concentration was in general 10 times higher than clean seawater's. Notably we report a primary resistome intrusion caused by the revival of residual microbes survived from disinfection treatment. These WWTP-borne ARBs, including several animal/human enteric pathogens, contributed up to 85% of the receiving water resistome. Plasmids and class 1 integrons were reckoned as major vehicles facilitating the persistence and dissemination of ARGs. Moreover, our work demonstrated the importance of extensive carrier identification in determining the driving force of multifactor coupled resistome booming in complicated environmental conditions, thereby paving the way for establishing priority for effective ARGs mitigation strategies.

This paper introduces a new metamodel-based knowledge representation that significantly improves autonomous learning and adaptation. While interest in hybrid machine learning / symbolic AI systems leveraging, for example, reasoning and knowledge graphs, is gaining popularity, we find there remains a need for both a clear definition of knowledge and a metamodel to guide the creation and manipulation of knowledge. Some of the benefits of the metamodel we introduce in this paper include a solution to the symbol grounding problem, cumulative learning, and federated learning. We have applied the metamodel to problems ranging from time series analysis, computer vision, and natural language understanding and have found that the metamodel enables a wide variety of learning mechanisms ranging from machine learning, to graph network analysis and learning by reasoning engines to interoperate in a highly synergistic way. Our metamodel-based projects have consistently exhibited unprecedented accuracy, performance, and ability to generalize. This paper is inspired by the state-of-the-art approaches to AGI, recent AGI-aspiring work, the granular computing community, as well as Alfred Korzybski's general semantics. One surprising consequence of the metamodel is that it not only enables a new level of autonomous learning and optimal functioning for machine intelligences, but may also shed light on a path to better understanding how to improve human cognition., Comment: arXiv admin note: text overlap with arXiv:2008.12879