Search

Given the prevalence of depression worldwide and its major impact on society, several studies employed artificial intelligence modelling to automatically detect and assess depression. However, interpretation of these models and cues are rarely discussed in detail in the AI community, but have received increased attention lately. In this study, we aim to analyse the commonly selected features using a proposed framework of several feature selection methods and their effect on the classification results, which will provide an interpretation of the depression detection model. The developed framework aggregates and selects the most promising features for modelling depression detection from 38 feature selection algorithms of different categories. Using three real-world depression datasets, 902 behavioural cues were extracted from speech behaviour, speech prosody, eye movement and head pose. To verify the generalisability of the proposed framework, we applied the entire process to depression datasets individually and when combined. The results from the proposed framework showed that speech behaviour features (e.g. pauses) are the most distinctive features of the depression detection model. From the speech prosody modality, the strongest feature groups were F0, HNR, formants, and MFCC, while for the eye activity modality they were left-right eye movement and gaze direction, and for the head modality it was yaw head movement. Modelling depression detection using the selected features (even though there are only 9 features) outperformed using all features in all the individual and combined datasets. Our feature selection framework did not only provide an interpretation of the model, but was also able to produce a higher accuracy of depression detection with a small number of features in varied datasets. This could help to reduce the processing time needed to extract features and creating the model.

Information visualizations are an efficient means to support the users in understanding large amounts of complex, interconnected data; user comprehension, however, depends on individual factors such as their cognitive abilities. The research literature provides evidence that user-adaptive information visualizations positively impact the users’ performance in visualization tasks. This study attempts to contribute toward the development of a computational model to predict the users’ success in visual search tasks from eye gaze data and thereby drive such user-adaptive systems. State-of-the-art deep learning models for time series classification have been trained on sequential eye gaze data obtained from 40 study participants’ interaction with a circular and an organizational graph. The results suggest that such models yield higher accuracy than a baseline classifier and previously used models for this purpose. In particular, a Multivariate Long Short Term Memory Fully Convolutional Network shows encouraging performance for its use in online user-adaptive systems. Given this finding, such a computational model can infer the users’ need for support during interaction with a graph and trigger appropriate interventions in user-adaptive information visualization systems. This facilitates the design of such systems since further interaction data like mouse clicks is not required.

The brain-computer interface (BCI) has become one of the most important biomedical research fields and has many useful applications. An important component of BCI, electroencephalography (EEG) is in general sensitive to noise and rich in all kinds of information from our brain. In this paper, we study the feature fusion problem in electroencephalography (EEG). We introduce (1) a discriminative feature extractor which can classify multi-labels from short-term EEG signals, and (2) a new strategy to filter out unwanted features from EEG signals based on our feature extractor. Filtering out signals relating to one property of the EEG signal while retaining another is similar to the way we can listen to just one voice during a party, which is known as the cocktail party problem in the machine learning area. Built based on the success of short-term EEG discriminative model, the feature filter is an end-to-end framework which is trained to map EEG signals with unwanted features directly to EEG signals without those features. Our experimental results on an alcoholism dataset show that our novel model can filter out over 90% of alcoholism information on average from EEG signals, with an average of only 4.2% useful feature accuracy lost, showing effectiveness for our proposed task.

Future affective computing research could be enhanced by enabling the computer to recognise a displayer's mental state from an observer's reaction (measured by physiological signals), using this information to improve recognition algorithms, and eventually to computer systems which are more responsive to human emotions. In this paper, an observer's physiological signals are analysed to distinguish displayers’ genuine from fake smiles. Overall, thirty smile videos were collected from four benchmark database and classified as showing genuine or fake smiles. Overall, forty observers viewed videos. We generally recorded four physiological signals: pupillary response (PR), electrocardiogram (ECG), galvanic skin response (GSR), and blood volume pulse (BVP). A number of temporal features were extracted after a few processing steps, and minimally correlated features between genuine and fake smiles were selected using the NCCA (canonical correlation analysis with neural network) system. Finally, classification accuracy was found to be as high as 98.8 percent from PR features using a leave-one-observer-out process. In comparison, the best current image processing technique [1] on the same video data was 95 percent correct. Observers were 59 percent (on average) to 90 percent (by voting) correct by their conscious choices. Our results demonstrate that humans can non-consciously (or emotionally) recognise the quality of smiles 4 percent better than current image processing techniques and 9 percent better than the conscious choices of groups.

The ecology of images of which Susan Sontag spoke in 1978 is urgently needed now if we are to establish a framework within which the daily onslaught of uncertain imagery can be understood, and a reliable photographic record of our current and future real world can be preserved. Many technologies (ubiquitous digital photography, sophisticated camera sensors, image editing, ‘deep fakes,’ GAN imaging) have brought to fruition the potential for photographic images to expand into a wide range of ‘ecological’ niches. The contexts that apply to image distribution (digital platforms, social engineering, governmental regulation) are daily news.Drawing together a range of investigations into aspects of the human-image nexus using eye gaze signals, neural networks, online surveys, educational workshops and research into digital platforms, we outline the main technological and social features of a potential ‘ecology of images’ that may help shine light on the panoply of images of our times.

In this paper, we explore the effects of individual pressure level and time constraint on searchers' behaviors and their assessment of search experience within the framework of interactive information retrieval. A user experiment was conducted in which 40 participants individually searched for information in a laboratory setting under two conditions: with time constraint (TC) and with no time constraint (NTC). Participants filled in a Perceived Stress Scale questionnaire to measure their chronic pressure value (subjective stress), and their pressure value was recorded as their individual characteristic. The results showed that the more chronic pressure the searcher has, the more search efforts they devote, including more time in searching and more time to complete the search tasks, especially when there was no time constraint. Time constraint and searchers’ pressure value had a significant effect on users’ numbers of scrolling actions per minute. The results indicate that when given a time constraint, searchers with higher-pressure values tend to lower their reading or scanning speed, while searchers with lower-pressure values tend to accelerate their reading or scanning speed. The results suggested different people would react to the time condition change in different ways, especially people with higher pressure. Therefore, it is necessary to examine users’ search behaviors in person-in-situation frameworks to analyze the effects of contextual factors on users. This study contributes to our knowledge of how contextual factors and individual characteristics affect searchers’ behaviors and have implications for the design of IIR systems.

We investigated a method to detect genuine smiles from observers’ physiological states. We recorded two physiological measures from people observing videos of smiles: pupillary response (PR) and galvanic skin response (GSR). Smile videos were from two benchmark databases (MAHNOB and AFEW). MAHNOB videos were classified as showing genuine or real smiles and AFEW videos were classified as not showing real smiles, based on their process of elicitation. A leave-one-observer-out procedure was employed to investigate classification performance using k-nearest neighbor (KNN), support vector machine (SVM), simple neural network (NN), and ensemble classifiers. Different noise removal techniques and a feature selection method — canonical correlation analysis with neural network (NCCA) — were applied to find minimally correlated features for the classes. Using these methods, the highest classification accuracy of 97.8% for PR and 96.6% for GSR signals were found via the ensemble classifier. In comparison, the observers (n = 20) correctly judged smiles as real only 58.9% of the time (on average) to 68.4% (by voting), which is similar to the literature, showing our data is similar in quality. Overall, our results demonstrate that user-independent analyses of physiological measures can substantially outperform individual self-reports for detecting real smiles.

Now, mobile devices extend learning to any location and time, where students adopt different preferences to learn effectively and efficiently through mobile devices. However, podcasting learning materials to mobile learners involves considerable challenge because of ongoing changes in the behaviour of learners in different contexts. This paper explores mobile learning (m-learning) preferences in tertiary education in order to suggest the best approaches to deliver digital learning materials (podcasts) in different contexts which are physical spaces (e.g. quiet, busy, or walking), and social spaces (e.g. alone, family, or with classmates). A total of 345 students completed a survey study to identify the role of mobile learners’ characteristics (gender, age, material status, nationality: Australians and Saudis, and prior m-learning experience) and their impact on podcast preferences in different physical and social spaces. Based on the survey results, in this paper, we shall present the impact characteristics on m-learning preferences for podcast length and types. We shall also summarise the affected contexts and their causes.

Smiles play a vital role in the understanding of social interactions within different communities, and reveal the physical state of mind of people in both real and deceptive ways. Several methods have been proposed to recognize spontaneous and posed smiles. All follow a feature-engineering based pipeline requiring costly pre-processing steps such as manual annotation of face landmarks, tracking, segmentation of smile phases, and hand-crafted features. The resulting computation is expensive, and strongly dependent on pre-processing steps. We investigate an end-to-end deep learning model to address these problems, the first end-to-end model for spontaneous and posed smile recognition. Our fully automated model is fast and learns the feature extraction processes by training a series of convolution and ConvLSTM layer from scratch. Our experiments on four datasets demonstrate the robustness and generalization of the proposed model by achieving state-of-the-art performances.

Invertible networks have various benefits for image de-noising since they are lightweight, information-lossless, and memory-saving during back-propagation. However, applying invertible models to remove noise is challenging because the input is noisy, and the reversed output is clean, following two different distributions. We propose an invertible denoising network, InvDN, to address this challenge. InvDN transforms the noisy input into a low-resolution clean image and a latent representation containing noise. To discard noise and restore the clean image, InvDN replaces the noisy latent representation with another one sampled from a prior distribution during reversion. The de-noising performance of InvDN is better than all the existing competitive models, achieving a new state-of-the-art result for the SIDD dataset while enjoying less run time. Moreover, the size of InvDN is far smaller, only having 4.2% of the number of parameters compared to the most recently proposed DANet. Further, via manipulating the noisy latent representation, InvDN is also able to generate noise more similar to the original one. Our code is available at: https://github.com/Yang-Liu1082/InvDN.git.

Deep learning (DL) models are widely used to provide a more convenient and smarter life. However, biased algorithms will negatively influence us. For instance, groups targeted by biased algorithms will feel unfairly treated and even fearful of negative consequences of these biases. This work targets biased generative models' behaviors, identifying the cause of the biases and eliminating them. We can (as expected) conclude that biased data causes biased predictions of face frontalization models. Varying the proportions of male and female faces in the training data can have a substantial effect on behavior on the test data: we found that the seemingly obvious choice of 50:50 proportions was not the best for this dataset to reduce biased behavior on female faces, which was 71% unbiased as compared to our top unbiased rate of 84%. Failure in generation and generating incorrect gender faces are two behaviors of these models. In addition, only some layers in face frontalization models are vulnerable to biased datasets. Optimizing the skip-connections of the generator in face frontalization models can make models less biased. We conclude that it is likely to be impossible to eliminate all training bias without an unlimited size dataset, and our experiments show that the bias can be reduced and quantified. We believe the next best to a perfect unbiased predictor is one that has minimized the remaining known bias., Comment: 8 pages, 11 figures; accepted by IJCNN2021

Although no consensus has been reached on the conditions under which modularity emerges, we lind the idea of specialisation driving modularity highly plausible. Current abstractions have demonstrated emergence of modularity in scenarios evolving Gene Regulatory Networks to show simple targeted behaviours, but these methods are less successful in more complex, real-world-level scenarios. We aim to study methods that are more efficient and effective in these simple scenarios, aiming to extend them to higher complexity. We engineer two modifications to the system that advance this aim. Progressive selection allows more than an order of magnitude reduction in computational cost without damaging the emergence of modularity, while dynamic specialisation produces significant improvements in modularity while also modestly reducing computational costs. We also provide some insight into the first of the two evolutionary stages in this approach, providing some explanation of why dynamic specialisation works.

This paper introduces the Eighth Emotion Recognition in the Wild (EmotiW) challenge. EmotiW is a benchmarking effort run as a grand challenge of the 22nd ACM International Conference on Multimodal Interaction 2020. It comprises of four tasks related to automatic human behavior analysis: a) driver gaze prediction; b) audio-visual group-level emotion recognition; c) engagement prediction in the wild; and d) physiological signal based emotion recognition. The motivation of EmotiW is to bring researchers in affective computing, computer vision, speech processing and machine learning to a common platform for evaluating techniques on a test data. We discuss the challenge protocols, databases and their associated baselines.

In this paper we explore the correlation between Convolutional Neural Network (CNN) architectures with random weights in the convolutional layers to the same architectures with trained weights. We show that this correlation extends to deep CNN architectures of up to 10 or even 12 layers to the extent that untrained model accuracy could be a useful proxy for trained model accuracy. We also find that for models with fewer layers much of this relationship comes from the strong correlation between the number of features output from the final CNN layer and final accuracy. With 10 and 12 layers there is a moderate correlation even when the size of the fully connected layer is held constant. We anticipate our findings in extending these correlations to deeper networks will be useful in designing faster Neural Architecture Search (NAS) models. Analytically solving for the weights of the final prediction layer is orders of magnitude faster than training the weights via backpropagation

Recently, researchers in the field of affective neuroscience have taken a keen interest in identifying patterns in brain activities that correspond to specific emotions. The relationship between music stimuli and brain waves has been of particular interest due to music’s disputed effects on brain activity. While music can have an anticonvulsant effect on the brain and act as a therapeutic stimulus, it can also have proconvulsant effects such as triggering epileptic seizures. In this paper, we take a computational approach to understand the effects of different types of music on the human brain; we analyse the effects of 3 different genres of music in participants electroencephalograms (EEGs). Brain activity was recorded using a 14-channel headset from 24 participants while they listened to different music stimuli. Statistical features were extracted from the signals and useful features and channels were identified using various feature selecting techniques. Using these features we built classification models based on K-nearest Neighbour (KNN), Support Vector Machine (SVM) and Neural Network (NN). Our analysis shows that NN, along with Genetic Algorithm (GA) feature selection, can reach the highest accuracy of 97.5% in classifying the 3 music genres. The model also reaches 98.6% accuracy in classifying music based on participants’ subjective rating of emotion. Additionally, the recorded brain waves identify different gamma wave levels, which are crucial in detecting epileptic seizures. Our results show that these computational techniques are effective in distinguishing music genres based on their effects on human brains.

Facial thermal imaging is a non-contact technology which can be useful for ubiquitous deceptive anxiety recognition. To date, studies investigating this technology have produced equivocal results in classification accuracy and finding the most correlated regions on the face. This study was conducted using our dataset with 41 subjects using two different protocols and three modalities (thermal, GSR and PPG). We selected and tracked five regions of interest (ROI) on each facial thermal imprint including periorbital, forehead, cheek, perinasal and chin that were mostly used in previous papers. By employing six statistical features, four feature reduction techniques and three classifiers, we attempted to identify the ROIs which are mostly associated with activation of the sympathetic nervous system to increase the final classification accuracy rate. The results of linear classification models show significant improvement of classification accuracy by using ROC feature selection method. We achieved 90.1% and 74.7% accuracy rate for thermal features in mock crime and best friend scenarios, respectively. Our experimental results show that perinasal and cheek areas have greater discriminatory power in comparison with other ROIs on the face.

People can manipulate emotion expressions when interacting with others. For example, acted anger can be expressed when the stimulus is not genuinely angry with an aim to manipulate the observer. In this paper, we aim to examine if the veracity of anger can be recognized from observers’ pupillary data with computational approaches. We use Genetic-based Feature Selection (GFS) methods to select time-series pupillary features of observers who see acted and genuine anger as video stimuli. We then use the selected features to train a simple fully connected neural network and a two-stream neural network. Our results show that the two-stream architecture is able to achieve a promising recognition result with an accuracy of 93.6% when the pupillary responses from both eyes are available. It also shows that genetic algorithm based feature selection method can effectively improve the classification accuracy by 3.1%. We hope our work could help current research such as human machine interaction and psychology studies that require emotion recognition.

This study addresses the question whether galvanic skin response (GSR) and blood volume pulse (BVP) of untrained and unaided observers can be used to identify real and posed smiles from different sets of smile videos or smile images. Observers were shown smile face videos/images, either singly or paired, with the intention to recognise each viewed as real or posed smiles. We created four experimental situations, namely single images (SI), single videos (SV), paired images (PI), and paired videos (PV). The GSR and BVP signals were recorded and processed. Our machine learning classifiers reached the highest accuracy of 93.3%, 87.6%, 92.0%, 91.7% for PV, PI, SV, and SI respectively. Finally, PV and SI were found to be the easiest and hardest way to identify real and posed smiles respectively. Overall, we demonstrated that observers’ subconscious physiological signals (GSR and BVP) are able to identify real and posed smiles at a good accuracy.

Ever since the advent of AlexNet, designing novel deep neural architectures for different tasks has consistently been a productive research direction. Despite the exceptional performance of various architectures in practice, we study a theoretical question: what is the condition for deep neural architectures to preserve all the information of the input data? Identifying the information lossless condition for deep neural architectures is important, because tasks such as image restoration require keep the detailed information of the input data as much as possible. Using the definition of mutual information, we show that: a deep neural architecture can preserve maximum details about the given data if and only if the architecture is invertible. We verify the advantages of our Invertible Restoring Autoencoder (IRAE) network by comparing it with competitive models on three perturbed image restoration tasks: image denoising, JPEG image decompression and image inpainting. Experimental results show that IRAE consistently outperforms non-invertible ones. Our model even contains far fewer parameters. Thus, it may be worthwhile to try replacing standard components of deep neural architectures with their invertible counterparts. We believe our work provides a unique perspective and direction for future deep learning research.

Transfer learning has been widely used as a deep learning technique to solve computer vision related problems, especially when the problem is image classification employing Convolutional Neural Networks (CNN). In this paper, a novel transfer learning approach that can adaptively integrate multiple models with different fine-tuning settings is proposed, which is denoted as MultiTune. To evaluate the performance of MultiTune, we compare it to SpotTune, a state-of-the-art transfer learning technique. Two image datasets from the Visual Decathlon Challenge are used to evaluate the performance of MultiTune. The FGVC-Aircraft dataset is a fine-grained task and the CIFAR100 dataset is a more general task. Results obtained in this paper show that MultiTune outperforms SpotTune on both tasks. We also evaluate MultiTune on a range of target datasets with smaller numbers of images per class. MultiTune outperforms SpotTune on most of these smaller-sized datasets as well. MultiTune is also less computational than SpotTune and requires less time for training for each dataset used in this paper.

This paper uses a graphic engine to simulate a large amount of training data with free annotations. Between synthetic and real data, there is a two-level domain gap, i.e., content level and appearance level. While the latter has been widely studied, we focus on reducing the content gap in attributes like illumination and viewpoint. To reduce the problem complexity, we choose a smaller and more controllable application, vehicle re-identification (re-ID). We introduce a large-scale synthetic dataset VehicleX. Created in Unity, it contains 1,362 vehicles of various 3D models with fully editable attributes. We propose an attribute descent approach to let VehicleX approximate the attributes in real-world datasets. Specifically, we manipulate each attribute in VehicleX, aiming to minimize the discrepancy between VehicleX and real data in terms of the Frechet Inception Distance (FID). This attribute descent algorithm allows content domain adaptation (DA) orthogonal to existing appearance DA methods. We mix the optimized VehicleX data with real-world vehicle re-ID datasets, and observe consistent improvement. With the augmented datasets, we report competitive accuracy. We make the dataset, engine and our codes available at https://github.com/yorkeyao/VehicleX.

Three-dimensional face reconstruction is one of the popular applications in computer vision. However, even state-of-the-art models still require frontal face as inputs, restricting its usage scenarios in the wild. A similar dilemma also happens in face recognition. New research designed to recover the frontal face from a single side-pose facial image has emerged. The state-of-the-art in this area is the Face-Transformation generative adversarial network, which is based on the CycleGAN. This inspired our researchwhich explores two models’ performance from pixel transformation in frontal facial synthesis, Pix2Pix and CycleGAN. We conducted the experiments on five different loss functions on Pix2Pix to improve its performance, then followed by proposing a new network Pairwise-GAN in frontal facial synthesis. Pairwise-GAN uses two parallel U-Nets as the generator and PatchGAN as the discriminator. The detailed hyper-parameters are also discussed. Based on the quantitative measurement by face similarity comparison, our results showed that Pix2Pix with L1 loss, gradient difference loss, and identity loss results in 2.72\(\%\) of improvement at average similarity compared to the default Pix2Pix model. Additionally, the performance of Pairwise-GAN is 5.4\(\%\) better than the CycleGAN, 9.1\(\%\) than the Pix2Pix, and 14.22\(\%\) than the CR-GAN at the average similarity. More experiment results and codes were released at https://github.com/XuyangSHEN/Pairwise-GAN.

There is a need to protect the personal identity information in public EEG datasets. However, it is challenging to remove such information that has infinite classes (open set). We propose an approach to disguise the identity information in EEG signals with dummy identities, while preserving the key features. The dummy identities are obtained by applying grand average on EEG spectrums across the subjects within a group that have common attributes. The personal identity information in original EEGs are transformed into disguised ones with a CycleGAN-based EEG disguising model. With the constraints added to the model, the features of interest in EEG signals can be preserved. We evaluate the model by performing classification tasks on both the original and the disguised EEG and compare the results. For evaluation, we also experiment with ResNet classifiers, which perform well especially on the identity recognition task with an accuracy of 98.4%. The results show that our EEG disguising model can hide about 90% of personal identity information and can preserve most of the other key features. Our code is available at https://github.com/ShiyaLiu/EEG-feature-filter-and-disguising.

Research on driving simulation has increasingly been concerned with the user’s experience of immersion and realism in mixed reality environments. One of the key issues is to determine whether people perceive and respond differently in these environments. Physiological signals provide objective indicators of people’s cognitive load, mental stress, and emotional state. Such data can be used to develop effective computational models and improve future systems. This study was designed to investigate the relationship between the verisimilitude of simple driving simulators and people’s physiological signals, specifically GSR (galvanic skin response), BVP (blood volume pulse) and PR (pupillary response). A within-subject design user experiment with 24 participants for five different driving simulation environments was conducted. Our results reveal that there is a significant difference in the mean of GSR among the conditions of different configurations of simple driving simulators, but this is not the case for BVP and PR. The individual differences of gender, whether people wear glasses and previous experiences of driving a car or using a driving simulator are correlated with some physiological signals. The data is classified using a hybrid GA-SVM (genetic algorithm-support vector machine) and GA-ANN (artificial neural network) approach. The evaluation of the classification performance using 10-fold cross-validation shows that the choice of the feature subset has minor impact on the classification performance, while the choice of the classifier can improve the accuracy for some classification tasks. The results further indicate that the SVM is more sensitive to the selection of training and test data than the ANN. Our findings inform about the verisimilitude of simple driving simulators on the driver’s perceived fidelity and physiological responses. Implications for the design of driving simulators in support of training are discussed.

The use of simulator technology has become popular in providing training, investigating driving activity and performing research as it is a suitable alternative to actual field study. The transferability of the achieved result from driving simulators to the real world is a critical issue considering later real-world risks, and important to the ethics of experiments. Moreover, researchers have to trade-off between simulator sophistication and the cost it incurs to achieve a given level of realism. This study will be the first step towards investigating the plausibility of different driving simulator configurations of varying verisimilitude, from drivers' galvanic skin response (GSR) signals. GSR is the widely used indicator of behavioural response. By analyzing GSR signals in a simulation environment, our results are aimed to support or contradict the use of simple low-level driving simulators. We investigate GSR signals of 23 participants doing virtual driving tasks in 5 different configurations of simulation environments. A number of features are extracted from the GSR signals after data preprocessing. With a simple neural network classifier, the prediction accuracy of different simulator configurations reaches up to 90% during driving. Our results suggest that participants are more engaged when realistic controls are used in normal driving, and are less affected by visible context during driving in emergency situations. The implications for future research are that for emergency situations realistic controls are important and research can be conducted with simple simulators in lab settings, whereas for normal driving the research should be conducted with full context in a real driving setting.

Visual Keyword Spotting (KWS), as a newly proposed task deriving from visual speech recognition, has plenty of room for improvements. This paper details our Visual Keyword Spotting system used in the first Mandarin Audio-Visual Speech Recognition Challenge (MAVSR 2019). With the assumption that the vocabularies of target dataset are a subset of the vocabulary of the training set, we proposed a simple and scalable classification based strategy that achieves 19.0% mean average precision (mAP) on this challenge. Our method is based on the idea of using sliding windows to bridge between the word-level dataset and the sentence-level dataset, showing that a strong word level classifier can be directly used in building sentence embedding, thereby making it possible to build a KWS system.

Controlled vocabularies such as Medical Subject Headings (MeSH) have been extensively used to organise information resources in the biomedical domain. However, the usefulness of these terms for information access has not been rigorously evaluated in interactive search environments. The objective of this study was to gain an understanding of domain experts’ interactions with novel search interfaces within the context of biomedical information search, with a goal of better interface design of information retrieval systems. An eye tracking study of biomedical domain experts’ interactions with novel search interfaces was conducted, considering user’s individual differences and gaze behaviour. The findings suggest that types of search interfaces have significant effects on gaze behaviour in terms of fixation-based measures of areas of interest, i.e. visual attention to the elements of title, author, abstract and MeSH terms in document surrogates. Significant interaction effects between cognitive style ...

The revolution in mobile devices and wireless networks has transferred E-learning to mobile learning (M-learning). Delivering learning materials (podcasting) to mobile learners is a challenging task due to the variability of mobile learner contexts. This paper presents a novel method of considering mobile learners’ preferences of podcast types in different contexts, utilizing learning preferences and personality. The survey has been collected from 345 students from Australia and Saudi Arabia. Unlike previous studies, this paper deals with all podcast types. Two dimensions have been included in context namely: physical space and social space. This has been followed by determining big five personality traits and learning preferences to determine how to personalize mobile learners’ podcast usage preferences.

The confluence of significant computational power and inexpensive sensors provides the opportunity to reliably collect large volumes of information from the world and extract humanly useful information resources. This paper reviews a coherent body of work over the last 20+ years focused on development of advanced bio-inspired computing techniques, and their applications primarily for human related data in behaviour and human centered computing. We close with a synthesis proposing an experiment analysis methodology combining these tools.

The presentation of learning materials affects how we learn. In this paper, we use eye tracking to investigate how different sequences of text and test questions affect performance outcomes, eye movements, and reading behavior for first (L1) English language and second (L2) English language readers. We show that different presentation sequences induce different performance outcomes, eye movements, and reading behavior. The sequence can affect how a participant reads the text as well as their perceptions of how well they understood what they read. For instance, if questions and text are not shown together, this improves participants’ ability to accurately perceive their comprehension and promotes thorough reading. Alternatively, showing questions before the text promotes skimming behavior. Importantly, the presentation sequence affects both L1 and L2 readers in the same way. We observe that L2 reader take longer to read text but have the same comprehension levels as L1 readers; this difference comes primarily from longer fixation durations. The results from this paper can be used to design learning materials in eLearning environments to influence how students interact with the learning environment as well as how they learn. The purpose of this investigation is to make informative decisions about designing adaptive eLearning environments.

Recent advances in the generative adversarial network (GAN) based image translation have shown its potential of being an image style transformer. Similarly, defined as a style transformer for physiological signals, a feature filter is used to filter privacy-related features while still keeping useful features. However, existing feature filter techniques have three problems: (1) the privacy-related features cannot be filtered out to the extent we need through a simple Conv-Deconv generator structure, and (2) the generator cannot control the semantics (maintain desired features) of given physiological signals. To address these problems, we utilize deeper neural networks and adopt techniques from domain adaptation. This includes semantic loss and a GAN based model structure with two generators, two discriminators and a classifier to form a game of five. Our results on the UCI EEG dataset demonstrate that our model can simultaneously (1) achieve the state-of-the-art accuracy removal for the privacy-related feature, (2) reduce the desired feature removal accuracy drop, and (3) make the filtered signals can be interpreted or visually checked.

Revealing the correspondences and relationships between physiological signals is attractive for bioinformatics and human-computer interaction. Time alignment is a straightforward way to figure out correspondences between time sequential data. However, alignment between multimodal physiological signals is hard to achieve because the similarity metrics are difficult to define if the two physiological signals being investigated are non-linearly correlated, misaligned or quite different in morphology. In this paper, we propose a generalized time alignment method for multimodal physiological signals which (i) learns the feature extractions on physiological signals in a generalized way, and (ii) enables learned features to be in a coordinated space where the similarity between sub-components from two signals can be defined. Furthermore, we applied our alignment based multimodal feature fusion on an evaluation model to perform emotion recognition tasks on the DEAP multimodal physiological signal dataset. The experimental results show that the alignment based feature fusion outperforms the non-aligned feature fusion in most cases.

Recognizing micro-expressions underpins significant and critical research and significant application. We speculate that this problem requires the understanding of the subtle face movement, integration of face structures and a solution of limited training data. In this paper, we build an effective micro-expression recognition system that leverages techniques stemming from these speculations. First, we introduce an optical flow method based on the onset frame and the apex frame to encode the subtle face motion. This has already been validated by prior research. Second, to obtain discriminative representations from the rigid face structures, part-based average pooling is proposed to inject structure priors to the network. Finally, because the system suffers from small training sets, we propose to transfer domain knowledge from macro-expression recognition tasks to micro-expression recognition. Specifically, we adopt two domain adaptation techniques including adversarial training and expression magnification and reduction (EMR). Through experiment, we show that the proposed system achieves very competitive results on the 2nd Micro-Expression Grand Challenge (MEGC).

With the rapid development of artificial intelligence in past decades, great attention has been drawn to the field of face detection and recognition. Humans show a high degree of variability in their expressions, poses and appearance. Thus, limitations such as disguised and occluded faces, make it hard to implement high-accuracy face detection in real life. Although several algorithms have been proposed to handle recognition of disguised faces, the interpretation of the possible features that may have impact on the performance of models is hardly mentioned. In this paper, we explored possible features that could distinguish disguised faces compared to original faces, including skin region, luminosity, textures and edges. A Deep Neural Network model was utilised for the comparison between different features using the Disguised Faces in the Wild dataset. Our results show that colour on skin region, luminosity and texture in images could greatly contribute to the performance in detection of disguised faces with a CNN architecture. The results from fusing the individual features significantly outperformed the results when using the whole image, performing 72%, which is considered the state-of-the-art in subject-independent disguised face detection.

We extend work on the transferability of features in deep neural networks to explore the interaction between training hyperparameters, optimal number of layers to transfer and the size of a target dataset. We show that using the commonly adopted transfer learning protocols results in increased overfitting and significantly decreased accuracy compared to optimal protocols, particularly for very small target datasets. We demonstrate that there is a relationship between fine-tuning hyperparameters used and the optimal number of layers to transfer. Our research shows that if this relationship is not taken into account, the optimal number of layers to transfer to the target dataset will likely be estimated incorrectly. Best practice transfer learning protocols cannot be predicted from existing research that has analysed transfer learning under very specific conditions that are not universally applicable. Extrapolating transfer learning training settings from previous findings can in fact be counterintuitive, particularly in the case of smaller datasets. We present optimal transfer learning protocols for various target dataset sizes from very small to large when source and target datasets and tasks are similar. Our results show that using these settings results in a large increase in accuracy when compared to commonly used transfer learning protocols. These results are most significant with very small target datasets. We observed an increase in accuracy of 47.8% on our smallest dataset which comprised of only 10 training examples per class. These findings are important as they are likely to improve outcomes from past, current and future research in transfer learning. We expect that researchers will want to re-examine their experiments to incorporate our findings and to check the robustness of their existing results.

Student forums are important for student engagement and learning in university courses but require high staff resources to moderate and answer questions. In introductory courses, the content can remain almost unchanged each year, so the questions asked in the course forums do not see a lot of variety over different iterations, which provides an opportunity for automation. This paper compiles a dataset of forum threads and meta-information of the participants from the Web Design and Development course at the Australian National University for the purposes of duplicate question detection in educational forums. A state of the art neural network model is trained on the dataset to measure its usefulness. An accuracy of 91.8% is achieved, which is on par with what is achieved on other datasets with similar features. A high performing neural network for this dataset could potentially be used to create a live system that detects and reuses answers for duplicate questions on course forums.

Objective We introduced a novel framework to identify the dynamic pattern of blood flow changes in the cutaneous superficial blood vessels of the face for 'fight or flight' responses through facial thermal imaging. Approach For this purpose, a thermal dataset was collected from 41 subjects in a mock crime scenario. Five facial areas including periorbital, forehead, perinasal, cheek and chin were selected on the face. Due to the cause and effect movement of blood in the facial cutaneous vasculature, the effective connectivity approach and graph analysis were used to extract causality features. The effective connectivity was quantified using a modified version of the multivariate Granger causality (GC) method among each pair of facial region of interests. Main results Validation was performed using statistical analysis, and the results demonstrated that the proposed method was statistically significant in detecting the physiological pattern of deceptive anxiety on the face. Moreover, the obtained graph is visualized by different schemes to show these interactions more effectively. We used machine learning techniques to classify our data based on the GC values, which result in a greater than 87% accuracy rate in discriminating between deceptive and truthful subjects.

This paper investigates observers' pupillary responses while they viewed two graphical visualizations (circular and organizational). The graphical visualizations are snapshots of the kind of data used in checking the degree of compliance with corporate governance best practice. Six very similar questions were asked from 24 observers for each visualization. In particular, we developed a neural network based classification model to understand these two visualizations from temporal features of observers' pupillary responses. We predicted that whether each observer is more accurate in understanding the two visualizations from their unconscious pupillary responses or conscious verbal responses, by answering relevant questions. We found that observers were physiologically 96.5% and 95.1% accurate, and verbally 80.6% and 81.3% accurate, for the circular and organizational visualizations, respectively.

This paper details the sixth Emotion Recognition in the Wild (EmotiW) challenge. EmotiW 2018 is a grand challenge in the ACM International Conference on Multimodal Interaction 2018, Colarado, USA. The challenge aims at providing a common platform to researchers working in the affective computing community to benchmark their algorithms on 'in the wild' data. This year EmotiW contains three sub-challenges: a) Audio-video based emotion recognition; b) Student engagement prediction; and c) Group-level emotion recognition. The databases, protocols and baselines are discussed in detail.

Compared to the early versions of smart phones, recent mobile devices have bigger screens that can present more web search results. Several previous studies have reported differences in user interaction between conventional desktop computer and mobile device-based web searches, so it is imperative to consider the differences in user behavior for web search engine interface design on mobile devices. However, it is still unknown how the diversification of screen sizes on hand-held devices affects how users search. In this article, we investigate search performance and behavior on three different small screen sizes: early smart phones, recent smart phones, and phablets. We found no significant difference with respect to the efficiency of carrying out tasks, however participants exhibited different search behaviors: less eye movement within top links on the larger screen, fast reading with some hesitation before choosing a link on the medium, and frequent use of scrolling on the small screen. This result suggests that the presentation of web search results for each screen needs to take into account differences in search behavior. We suggest several ideas for presentation design for each screen size.

The emergence of a "post-fact" world has seen knowledge being misinterpreted and manipulated to suit diverse purposes. Information from credible sources is often drowned out by ‘facts’ with dubious provenance. It is not enough to simply establish knowledge provenance if this meta-data can be changed afterwards. This paper presents a possible solution using blockchains to manage knowledge provenance. The solution is modelled using a concrete example of knowledge institutions. Variations and extensions to suit other domains are suggested. Particularly, a graph based variation of the blockchain is outlined. We also discuss the impact of our model beyond technology.

Wagner's modularity inducing problem domain is a key contribution to the study of the evolution of modularity, including both evolutionary theory and evolutionary computation. We study its behavior under classical genetic algorithms. Unlike what we seem to observe in nature, the emergence of modularity is highly conditional and dependent, for example, on the eagerness of search. In nature, modular solutions generally dominate populations, whereas in this domain, modularity, when it emerges, is a relatively rare variant. Emergence of modularity depends heavily on random fluctuations in the fitness function; with a randomly varied but unchanging fitness function, modularity evolved far more rarely. Interestingly, high-fitness non-modular solutions could frequently be converted into even-higher-fitness modular solutions by manually removing all inter-module edges. Despite careful exploration, we do not yet have a full explanation of why the genetic algorithm was unable to find these better solutions.

Previous research has revealed that people are generally poor at distinguishing genuine and acted anger facial expressions, with a mere 65% accuracy of verbal answers. We aim to investigate whether a group of feedforward neural networks can perform better using raw pupillary dilation signals from individuals. Our results show that a single neural network cannot accurately discern the veracity of an emotion based on raw physiological signals, with an accuracy of 50.5%. Nonetheless, distinct neural networks using pupillary dilation signals from different individuals display a variety of genuineness for discerning the anger emotion, from 27.8% to 83.3%. By leveraging these differences, our novel Misaka neural networks can compose predictions using different individuals’ pupillary dilation signals to give a more accurate overall prediction than even from the highest performing single individual, reaching an accuracy of 88.9%. Further research will involve the investigation of the correlation between two groups of high-performing predictors using verbal answers and pupillary dilation signals.

In this era of deep learning and big data, the transformation of biomedical big data into recognizable patterns is an important research focus and a great challenge in bioinformatics. An important form of biomedical data is electroencephalography (EEG) signals, which are generally strongly affected by noise and there exists notable individual, environmental and device differences. In this paper, we focus on learning discriminative features from short time EEG signals. Inspired by traditional image compression techniques to learn a robust representation of an image, we introduce and compare two strategies for learning features from EEG using two specifically designed autoencoders. Channel-wise autoencoders focus on features in each channel, while Image-wise autoencoders instead learn features from the whole trial. Our results on a UCI EEG dataset show that using both Channel-wise and Image-wise autoencoders achieve good performance for a classification problem with state of art accuracy in both within-subject and cross-subject tests. A further experiment using shared weights shows that the shared weights technique only slightly influenced learning but it reduced training time significantly.

Training neural network and other classifiers on physiological signals has challenges beyond more traditional datasets, as the training data includes data points which are not independent. Most obviously, more than one sample can come from a particular human subject. Standard cross-validation as implemented in many AI tools gives artificially high results as the common human subject is not considered. This is handled by some papers in the literature, by using leave-one-subject-out cross-validation. We argue that this is not sufficient, and introduce our independent approach, which is leave-one-subject-and-one-stimulus-out cross-validation. We demonstrate our approach using KNN, SVM and NN classifiers and their ensemble, using an extended example of physiological recordings from subjects observing genuine versus posed smiles, which are the two kinds of the nicest smiles and hard for people to differentiate reliably. We use three physiological signals, 20 video stimuli and 24 observers/participants, achieving 96.1% correct results, in a truly robust fashion.

Crowd predictions have demonstrated powerful performance in predicting future events. We aim to understand crowd prediction efficacy in ascertaining the veracity of human emotional expressions. We discover that collective discernment can increase the accuracy of detecting emotion veracity from 63%, which is the average individual performance, to 80%. Constraining data to best-performers can further increase the result up to 92%. Neural networks can achieve an accuracy of 99.69% by aggregating participants’ answers. That is, assigning positive and negative weights to high and low human predictors, respectively. Furthermore, neural networks that are trained with one emotion data can also produce high accuracies on discerning the veracity of other emotion types: our crowdsourced transfer of emotion learning is novel. We find that our neural networks do not require a large number of participants, particularly, 30 randomly selected, to achieve high accuracy predictions, better than any individual participant. Our proposed method of assembling peoples’ predictions with neural networks can provide insights for applications such as fake news prevention and lie detection.

In the big data era, while correlation detection is relatively straightforward and successfully addressed by many techniques, causality detection does not have a generally-used solution. Causality provides valuable insights into data and guides further studies. With the overall assumption that causal influence can only be from prior history events, time plays an essential part in causality analysis, and this important feature means the data with strict temporal structure needs to be modelled. Traditionally, temporal point processes are employed to model data containing temporal structure information. The heuristic parameterization property of such models makes the task difficult. Domain related knowledge are needed to design proper parameterization. Recently, Recurrent Neural Networks (RNNs) have been used for time-related data modelling. RNN’s trainable parameterization considerably reduces the dependency on domain-related knowledge. In this work, we show that combining neural network techniques with Granger causality framework has great potential by presenting an RNN model integrated with a Granger causality framework. The experimental results show that the same network structure can be applied to a variety of datasets and causalities are detected successfully.

Relevance is the strength of the relationship between a user's perceived information need and an information object. Systems designed to help users identify relevant information can often rely on high quality labelled datasets. However, the subjective and personal nature of relevance means that establishing ground truth labels is difficult. In this work, we conduct a user study on text documents to crowdsource relevance assessments against four topics. Workers' self-reported measures and task completion speed are used to calculate a range of ground truth measures against which classification performance can be assessed. Our results indicate that average subjective relevance and confidence-weighted measures are on par with the annotations from an expert panel. Further work is planned to expand these findings.