Search

The Artificial Intelligence (AI) research community has used ad-hoc benchmarks to measure the “intelligence” level of Large Language Models (LLMs). In humans, intelligence is closely linked to the functional integrity of the prefrontal lobes, which are essential for higher-order cognitive processes. Previous research has found that LLMs struggle with cognitive tasks that rely on these prefrontal functions, highlighting a significant challenge in replicating human-like intelligence. In December 2022, OpenAI released ChatGPT, a new chatbot based on the GPT-3.5 model that quickly gained popularity for its impressive ability to understand and respond to human instructions, suggesting a significant step towards intelligent behaviour in AI. Therefore, to rigorously investigate LLMs' level of “intelligence,” we evaluated the GPT-3.5 and GPT-4 versions through a neuropsychological assessment using tests in the Italian language routinely employed to assess prefrontal functioning in humans. The same tests were also administered to Claude2 and Llama2 to verify whether similar language models perform similarly in prefrontal tests. When using human performance as a reference, GPT-3.5 showed inhomogeneous results on prefrontal tests, with some tests well above average, others in the lower range, and others frankly impaired. Specifically, we have identified poor planning abilities and difficulty in recognising semantic absurdities and understanding others' intentions and mental states. Claude2 exhibited a similar pattern to GPT-3.5, while Llama2 performed poorly in almost all tests. These inconsistent profiles highlight how LLMs' emergent abilities do not yet mimic human cognitive functioning. The sole exception was GPT-4, which performed within the normative range for all the tasks except planning. Furthermore, we showed how standardised neuropsychological batteries developed to assess human cognitive functions may be suitable for challenging LLMs’ performance.

Abstract Human accuracy in detecting deception with intuitive judgments has been proven to not go above the chance level. Therefore, several automatized verbal lie detection techniques employing Machine Learning and Transformer models have been developed to reach higher levels of accuracy. This study is the first to explore the performance of a Large Language Model, FLAN-T5 (small and base sizes), in a lie-detection classification task in three English-language datasets encompassing personal opinions, autobiographical memories, and future intentions. After performing stylometric analysis to describe linguistic differences in the three datasets, we tested the small- and base-sized FLAN-T5 in three Scenarios using 10-fold cross-validation: one with train and test set coming from the same single dataset, one with train set coming from two datasets and the test set coming from the third remaining dataset, one with train and test set coming from all the three datasets. We reached state-of-the-art results in Scenarios 1 and 3, outperforming previous benchmarks. The results revealed also that model performance depended on model size, with larger models exhibiting higher performance. Furthermore, stylometric analysis was performed to carry out explainability analysis, finding that linguistic features associated with the Cognitive Load framework may influence the model’s predictions.

The accurate recognition of others' facial expressions is a core skill for social interactions. The left Dorsolateral Prefrontal Cortex (L-DLPFC) represents a key node in the network for facial emotion recognition. However, its specific role is still under debate. As such, the aim of the current neuromodulation study was to assess the causal role of the L-DLPFC in humans' rating of facial expressions of emotions and implicit attitudes toward other races. In this sham-controlled single-blind between-subject experiment, we offline administered L-DLPFC transcranial direct current stimulation (tDCS) to 69 healthy participants who were divided into three groups of 23 (each receiving anodal 1 mA tDCS, anodal 2 mA tDCS, or Sham), before completing an “Emotion Rating task and two Implicit Association Tests (IATs). The former required the intensity rating of 192 faces (half black and half white) displaying happiness, sadness, anger, or fear. The IATs were designed to assess participants' automatic associations of positive or negative attributes with racial contents. Results on the Emotion Rating task showed participants' gender-specific effect of tDCS. Specifically, a gender bias, with only males showing a tendency to underestimate negative emotions was found in Sham, and absent in the tDCS groups. When considering the race of the stimuli, females but not males in Sham exhibited a racial bias, that is, the tendency to overestimate negative emotions of other-race faces. Again, the bias disappeared in the tDCS groups. Concerning the IATs, no significant effects emerged. We conclude that the L-DLPFC plays a critical role in humans’ rating of facial expressions, and for variability in other-race emotional judgements. These results shed light on the neural bases of the human emotional system and its gender-related differences, and have potential implications for interventional settings.