Search

The release of the generative pre-trained transformer (GPT) series has brought artificial general intelligence (AGI) to the forefront of the artificial intelligence (AI) field once again. However, the questions of how to define and evaluate AGI remain unclear. This perspective article proposes that the evaluation of AGI should be rooted in dynamic embodied physical and social interactions (DEPSI). More specifically, we propose five critical characteristics to be considered as AGI benchmarks and suggest the Tong test as an AGI evaluation system. The Tong test describes a value- and ability-oriented testing system that delineates five levels of AGI milestones through a virtual environment with DEPSI, allowing for infinite task generation. We contrast the Tong test with classical AI testing systems in terms of various aspects and propose a systematic evaluation system to promote standardized, quantitative, and objective benchmarks and evaluation of AGI.

AI is transforming many fields, including higher education. The pandemic has shown how AI can improve learning and teaching in higher education. This review examines how AI affects education quality, learning assessment, and higher education jobs (HE). The study employs a systematic qualitative method to review the academic literature on AI and higher education between 1900 and 2021. The data was gathered from various sources, including ERIC, Scopus, and the Web of Science, using specific exclusion and inclusion criteria centred on publication date, language, reported outcomes, setting, and publication type. From there on, the articles were analysed by Rayyan Software and categorised in Excel according to a scale that included aspects such as the quality of learning and teaching, assessment, and potential ethical future careers. The research also produced two bibliometric figures using VOSviewer to investigate co-authorship and the frequency of keyword occurrences in academic journals published in AI and HE. The analysis was done to ensure the study's validity in the scientific community. The study found that AI can improve education quality, provide practical learning and teaching methods, and improve assessments to better prepare students for careers. The study also emphasises the potential of AI to shape future employment opportunities and the need for higher education institutions to adopt AI to meet market demands. The study calls for more research on AI's effects on assessment, integrity, and higher education careers. [ABSTRACT FROM AUTHOR]

In search of greater resource savings and efficiencies, companies are turning to new technologies in the interview process, such as artificial intelligence evaluation (AIE). However, little is known about candidate reactions to this new tool. We identified outstanding questions regarding reactions to AIE arising from justice and signaling theories and conducted interviews with 33 professionals to understand their perceptions of AIE use in selection. Participants raised issues related to all four types of justice and indicated that there is a signaling effect of AIE use. Despite acknowledging the superior objectivity of AIE, participants expressed a desire for the maintenance of human elements in the evaluation process, seemingly preferring 'the devil they know' (human biases and intuition) rather than the one they do not (AIE algorithm). This result is explored through the lens of uncertainty reduction, discussing theoretical implications for justice and signaling theories, and providing implications for the implementation of AIE in the selection process. [ABSTRACT FROM AUTHOR]

AI systems are usually evaluated on a range of problem instances and compared to other AI systems that use different strategies. These instances are rarely independent. Machine learning, and supervised learning in particular, is a very good example of this. Given a machine learning model, its behaviour for a single instance cannot be understood in isolation but rather in relation to the rest of the data distribution or dataset. In a dual way, the results of one machine learning model for an instance can be analysed in comparison to other models. While this analysis is relative to a population or distribution of models, it can give much more insight than an isolated analysis. Item response theory (IRT) combines this duality between items and respondents to extract latent variables of the items (such as discrimination or difficulty) and the respondents (such as ability). IRT can be adapted to the analysis of machine learning experiments (and by extension to any other artificial intelligence experiments). In this paper, we see that IRT suits classification tasks perfectly, where instances correspond to items and classifiers correspond to respondents. We perform a series of experiments with a range of datasets and classification methods to fully understand what the IRT parameters such as discrimination, difficulty and guessing mean for classification instances (and their relation to instance hardness measures) and how the estimated classifier ability can be used to compare classifier performance in a different way through classifier characteristic curves. [ABSTRACT FROM AUTHOR]

The progression in several cognitive tests for the same subjects at different ages provides valuable information about their cognitive development. One question that has caught recent interest is whether the same approach can be used to assess the cognitive development of artificial systems. In particular, can we assess whether the ‘fluid’ or ‘crystallised’ intelligence of an artificial cognitive system is changing during its cognitive development as a result of acquiring more concepts? In this paper, we address several IQ tests problems (odd-one-out problems, Raven’s Progressive Matrices and Thurstone’s letter series) with a general learning system that is not particularly designed on purpose to solve intelligence tests. The goal is to better understand the role of the basic cognitive operational constructs (such as identity, difference, order, counting, logic, etc.) that are needed to solve these intelligence test problems and serve as a proof-of-concept for evaluation in other developmental problems. From here, we gain some insights into the characteristics and usefulness of these tests and how careful we need to be when applying human test problems to assess the abilities and cognitive development of robots and other artificial cognitive systems. [ABSTRACT FROM AUTHOR]