Search

Falls are a major issue for those over the age of 65 years worldwide. Objective assessment of fall risk is rare in clinical practice. The most common methods of assessment are time-consuming observational tests (clinical tests). Computer-aided diagnosis could be a great help. A popular clinical test for fall risk is the five times sit-to-stand. The time taken to complete the test is the most commonly used metric to identify the most at-risk patients. However, tracking the movement of skeletal joints can provide much richer insights. We use markerless motion capture, allied with a representational model, to identify those at risk of falls. Our method uses an LSTM autoencoder to derive a distance measure. Using this measure, we introduce a new scoring system, allowing individuals with differing falls risks to be placed on a continuous scale. Evaluating our method on the KINECAL dataset, we achieved an accuracy of 0.84 in identifying those at elevated falls risk. In addition to identifying potential fallers, our method could find applications in rehabilitation. This aligns with the goals of the KINECAL Dataset. KINECAL contains the recordings of 90 individuals undertaking 11 movements used in clinical assessments. KINECAL is labelled to disambiguate age-related decline and falls risk.

Abstract The field of human action recognition has made great strides in recent years, much helped by the availability of a wide variety of datasets that use Kinect to record human movement. Conversely, progress towards the use of Kinect in clinical practice has been hampered by the lack of appropriate data. In particular, datasets that contain clinically significant movements and appropriate metadata. This paper proposes a dataset to address this issue, namely KINECAL. It contains the recordings of 90 individuals carrying out 11 movements, commonly used in the clinical assessment of balance. The dataset contains relevant metadata, including clinical labelling, falls history labelling and postural sway metrics. KINECAL should be of interest to researchers interested in the clinical use of motion capture and motion analysis.

In this article, we investigate the spontaneity issue in facial expression sequence generation. Current leading methods in the field are commonly reliant on manually adjusted conditional variables to direct the model to generate a specific class of expression. We propose a neural network-based method which uses Gaussian noise to model spontaneity in the generation process, removing the need for manual control of conditional generation variables. Our model takes two sequential images as input, with additive noise, and produces the next image in the sequence. We trained two types of models: single-expression, and mixed-expression. With single-expression, unique facial movements of certain emotion class can be generated; with mixed expressions, fully spontaneous expression sequence generation can be achieved. We compared our method to current leading generation methods on a variety of publicly available datasets. Initial qualitative results show our method produces visually more realistic expressions and facial action unit (AU) trajectories; initial quantitative results using image quality metrics (SSIM and NIQE) show the quality of our generated images is higher. Our approach and results are novel in the field of facial expression generation, with potential wider applications to other sequence generation tasks.

Early detection of skin cancer, particularly melanoma, is crucial to enable advanced treatment. Due to the rapid growth in the number of skin cancers, there is a growing need of computerised analysis for skin lesions. The state-of-the-art public available datasets for skin lesions are often accompanied with a very limited amount of segmentation ground truth labeling. Also, the available segmentation datasets consist of noisy expert annotations reflecting the fact that precise annotations to represent the boundary of skin lesions are laborious and expensive. The lesion boundary segmentation is vital to locate the lesion accurately in dermoscopic images and lesion diagnosis of different skin lesion types. In this work, we propose the fully automated deep learning ensemble methods to achieve high sensitivity and high specificity in lesion boundary segmentation. We trained the ensemble methods based on Mask R-CNN and DeeplabV3+ methods on ISIC-2017 segmentation training set and evaluate the performance of the ensemble networks on ISIC-2017 testing set and PH2 dataset. Our results showed that the proposed ensemble methods segmented the skin lesions with Sensitivity of 89.93% and Specificity of 97.94% for the ISIC-2017 testing set. The proposed ensemble method Ensemble-A outperformed FrCN, FCNs, U-Net, and SegNet in Sensitivity by 4.4%, 8.8%, 22.7%, and 9.8% respectively. Furthermore, the proposed ensemble method Ensemble-S achieved a specificity score of 97.98% for clinically benign cases, 97.30% for the melanoma cases, and 98.58% for the seborrhoeic keratosis cases on ISIC-2017 testing set, exhibiting better performance than FrCN, FCNs, U-Net, and SegNet.

We propose the Edge-Embedded Multi-Dropout (EEMD) framework for real-time face alignment. The EEMD framework extracts facial edge features and explores multiple dropout architecture for locating facial landmarks. It consists of two major component networks, namely the Contour Detection Network (CDN) and the Multi-Dropout Network (MDN); and two supplementary networks, one for face detection and the other for pose regression. When a face is detected by the face detector, its pose will be classified by the pose classifier, then the associated facial edges be detected by the CDN, and then the landmarks be located by the MDN. The embedding of the CDN into the EEMD framework describes the observation that most landmarks are located on the contours/edges of the facial components and of the whole face. We revise a state-of-the-art edge detector as part of the base network for the CDN. The MDN is proposed to better design the regression architecture with appropriate dropout settings for better preventing overfitting and enhancing regression accuracy. Unlike most of the 2D approaches unable to locate landmarks in extreme poses, the proposed framework can detect landmarks on profile faces, i.e., ±90° in yaw, in real time. Evaluated on benchmark databases, the EEMD demonstrates a competitive performance to other state-of-the-art approaches with a satisfying runtime speed.

We present a novel silhouette extraction algorithm designed for the binary segmentation of swimmers underwater. The intended use of this algorithm is within a 2D-to-3D pipeline for the markerless motion capture of swimmers, a task which has not been achieved satisfactorily, partly due to the absence of silhouette extraction methods that work well on images of swimmers. Our algorithm, FISHnet, was trained on the novel Scylla dataset, which contains 3,100 images (and corresponding hand-traced silhouettes) of swimmers underwater, and achieved a dice score of 0.9712 on its test data. Our algorithm uses a U-Net-like architecture and VGG16 as a backbone. It introduces two novel modules: a modified version of the Semantic Embedding Branch module from ExFuse, which increases the complexity of the features learned by the layers of the encoder; and the Spatial Resolution Enhancer module, which increases the spatial resolution of the features of the decoder before they are skip connected with the features of the encoder. The contribution of these two modules to the performance of our network was marginal, and we attribute this result to the lack of data on which our network was trained. Nevertheless, our model outperformed state-of-the-art silhouette extraction algorithms (namely DeepLabv3+) on Scylla, and it is the first algorithm developed specifically for the task of accurately segmenting the silhouettes of swimmers.

Understanding the processes by which the mammalian embryo implants in the maternal uterus is a long-standing challenge in embryology. New insights into this morphogenetic event could be of great importance in helping, for example, to reduce human infertility. During implantation the blastocyst, composed of epiblast, trophectoderm and primitive endoderm, undergoes significant remodelling from an oval ball to an egg cylinder. A main feature of this transformation is symmetry breaking and reshaping of the epiblast into a "cup". Based on previous studies, we hypothesise that this event is the result of mechanical constraints originating from the trophectoderm, which is also significantly transformed during this process. In order to investigate this hypothesis we propose MG# (MechanoGenetic Sharp), an original computational model of biomechanics able to reproduce key cell shape changes and tissue level behaviours in silico. With this model, we simulate epiblast and trophectoderm morphogenesis during implantation. First, our results uphold experimental findings that repulsion at the apical surface of the epiblast is essential to drive lumenogenesis. Then, we provide new theoretical evidence that trophectoderm morphogenesis indeed can dictate the cup shape of the epiblast and fosters its movement towards the uterine tissue. Our results offer novel mechanical insights into mouse peri-implantation and highlight the usefulness of agent-based modelling methods in the study of embryogenesis.

Posturography provides quantitative, objective measurements of human balance and postural control for research and clinical use. However, it usually requires access to specialist equipment to measure ground reaction forces, which are not widely available in practice, due to their size or cost. In this study, we propose an alternative approach to posturography. It uses the skeletal output of an inexpensive Kinect depth camera to localise the Centre of Mass (CoM) of an upright individual. We demonstrate a pipeline which is able to measure postural sway directly from CoM trajectories, obtained from tracking the relative position of three key joints. In addition, we present the results of a pilot study that compares this method of measuring postural sway to the output of a NeuroCom SMART Balance Master. 15 healthy individuals (age: 42.3 ± 20.4 yrs, height: 172 ± 11 cm, weight: 75.1 ± 14.2 kg, male = 11), completed 25 Sensory Organisation Test (SOT) on a NeuroCom SMART Balance Master. Simultaneously, the sessions were recorded using custom software developed for this study (CoM path recorder). Postural sway was calculated from the output of both methods and the level of agreement determined, using Bland-Altman plots. Good agreement was found for eyes open tasks with a firm support, the agreement decreased as the SOT tasks became more challenging. The reasons for this discrepancy may lie in the different approaches that each method takes to calculate CoM. This discrepancy warrants further study with a larger cohort, including fall-prone individuals, cross-referenced with a marker-based system. However, this pilot study lays the foundation for the development of a portable device, which could be used to assess postural control, more cost-effectively than existing equipment.

Ultrasound imaging systems produce images that are affected by speckle. This is mainly due to the interference of the returning wave at the transducer aperture that degrades images quality. As the filtering stage affects the quality of segmentation for breast ultrasound images, significant research recently concentrated on eliminating the speckle using different mathematical methods. In this paper, we propose a new approach for segmentation of breast ultrasound images called Adjusted Quick Shift Phase Preserving Dynamic Range Compression (AQS-APPDRC). AQS-APPDRC consists of three steps: a preprocessing step by applying APPDRC Filter and Frost Filter, followed by proposed Adjusted Quick Shift segmentation for superpixel extraction, and a post processing step of Binary Thresholding for blob selection. The results of our proposed AQS-APPDRC segmentation is compared with two other conventional segmentation methods namely: QS-FR, and QS-PPDRC. In addition, this study considers two state-of-the-art Convolutional Neural Networks (CNNs), i.e. U-Net and FCNs (FCN-AlexNet, FCN-32s and FCN-16s) for comparison. The segmentation results are evaluated on two small breast ultrasound datasets, where Dataset A has 306 images and Dataset B has 163 images. The proposed AQS-APPDRC approach achieved the best performance amongst two conventional methods and the CNNs in terms of Dice, Specificity, and MCC, when evaluated on Dataset A. For Dataset B, FCN-16s showed the best Dice, Specificity, and MCC, but the proposed AQS-APPDRC achieved comparable results. For Sensitivity, FCN-32s showed the best result for both datasets. Our results revealed that, for CNNs, the size of dataset is always the key indicator for its performance. The conventional methods produce comparable results on small datasets.

Breast ultrasound images have a complicated structure, which is difficult to be segmented due to the fact that it has low signal and affected by noise ratio. Recent research concentrated on the Region of Interest (ROI) labeling and ROI segmentation. In order to reduce chances of human error, stages of processing in breast ultrasound images might be different from one another. This research proposes a new image filtering method for breast ultrasound, namely Altered Phase Preserving Dynamic Range Compression (APPDRC). In addition, this paper compares the performance of filtering algorithms, in combination with standard thresholding segmentation. Focusing on the filtering stage, a comparison between the proposed method APPDRC Filter and previous approaches is validated on a dataset of 306 images, namely Inverted Median filter, Multifractal Filter, Hybrid Filter, SRAD filter, and PPDRC. Further, a summary of the work to date on the effect of filtering on lesion segmentation in ultrasound breast images is reported. Jaccard Similarity Index (JSI) is used for evaluation, in which the automated segmentation result is compared with the experienced radiologist's manual delineation. Our results revealed that making the choice of filtering algorithm affects the final segmentation results. Considering Mean JSI, Dice and MCC metrics, the proposed APPDRC Filter achieved the best performance, and outperformed the five evaluated filtering methods. Keywords: Ultrasound images, Jaccard similarity index, Filtering, Region-of-interest (ROI), Breast lesion, 2017 MSC: 00-01, 99-00

Over the past few decades, different clinical diagnostic algorithms have been proposed to diagnose malignant melanoma in its early stages. Furthermore, the detection of skin moles driven by current deep learning based approaches yields impressive results in the classification of malignant melanoma. However, in all these approaches, the researchers do not take into account the origin of the skin lesion. It has been observed that the specific criteria for in situ and early invasive melanoma highly depend on the anatomic site of the body. To address this problem, we propose a deep learning architecture based framework to classify skin lesions into the three most important anatomic sites, including the face, trunk and extremities, and acral lesions. In this study, we take advantage of pretrained networks, including VGG19, ResNet50, Xception, DenseNet121, and EfficientNetB0, to calculate the features with an adjusted and densely connected classifier. Furthermore, we perform in depth analysis on database, architecture, and result regarding the effectiveness of the proposed framework. Experiments confirm the ability of the developed algorithms to classify skin lesions into the most important anatomical sites with 91.45% overall accuracy for the EfficientNetB0 architecture, which is a state-of-the-art result in this domain.

Long video datasets of facial macro- and micro-expressions remains in strong demand with the current dominance of data-hungry deep learning methods. There are limited methods of generating long videos which contain micro-expressions. Moreover, there is a lack of performance metrics to quantify the generated data. To address the research gaps, we introduce a new approach to generate synthetic long videos and recommend assessment methods to inspect dataset quality. For synthetic long video generation, we use the state-of-the-art generative adversarial network style transfer method—StarGANv2. Using StarGANv2 pre-trained on the CelebA dataset, we transfer the style of a reference image from SAMM long videos (a facial micro- and macro-expression long video dataset) onto a source image of the FFHQ dataset to generate a synthetic dataset (SAMM-SYNTH). We evaluate SAMM-SYNTH by conducting an analysis based on the facial action units detected by OpenFace. For quantitative measurement, our findings show high correlation on two Action Units (AUs), i.e., AU12 and AU6, of the original and synthetic data with a Pearson’s correlation of 0.74 and 0.72, respectively. This is further supported by evaluation method proposed by OpenFace on those AUs, which also have high scores of 0.85 and 0.59. Additionally, optical flow is used to visually compare the original facial movements and the transferred facial movements. With this article, we publish our dataset to enable future research and to increase the data pool of micro-expressions research, especially in the spotting task.

Facial wrinkles (considered to be natural features) appear as people get older. Wrinkle detection is an important aspect of applications that depend on facial skin changes, such as face age estimation and soft biometrics. While existing wrinkle detection algorithms focus on forehead horizontal lines, it is necessary to develop new methods to detect all wrinkles (vertical and horizontal) on whole face. Therefore, we evaluated the performance of wrinkle detection algorithms on the whole face and proposed an enhancement technique to improve the performance. More specifically, we used 45 images of the Face Recognition Technology dataset (FERET) and 25 images of the Sudanese dataset. For ground truth annotations, the selected images were manually annotated by the researcher. The experiments showed that the method with enhancement performed better at detecting facial wrinkles when compared to the state-of-the-art methods. When evaluated on FERET, the average Jaccard similarity indices were 56.17%, 31.69% and 15.87% for the enhancement method, Hybrid Hessian Filter and Gabor Filter, respectively.

Wrinkles play an important role in the face-based analysis. They have been widely used in applications, such as facial retouching, facial expression recognition, and face age estimation. Although a few techniques for a wrinkle analysis have been explored in the literature, poor detection limits the accuracy and reliability of wrinkle segmentation. Therefore, an automated wrinkle detection method is crucial to maintain consistency and reduce human error. In this paper, we propose Hessian line tracking (HLT) to overcome the detection problem. HLT is composed of Hessian seeding and directional line tracking. It is an extension of a Hessian filter; however, it significantly increases the accuracy of wrinkle localization when compared with existing methods. In the experimental phase, three coders were instructed to annotate wrinkles manually. To assess the manual annotation, both intrareliability and interreliability were measured, with an accuracy of 94% or above. The experimental results show that the proposed method is capable of tracking hidden pixels; thus, it increases connectivity of detection between wrinkles, allowing some fine wrinkles to be detected. In comparison to the state-of-the-art methods such as the Cula Method, Frangi Filter, and Hybrid Hessian Filter, the proposed HLT yields better results, with an accuracy of 84%. This paper demonstrates that the HLT is a remarkably strong detector of forehead wrinkles in 2-D images.

Micro-expressions are brief spontaneous facial expressions that appear on a face when a person conceals an emotion, making them different to normal facial expressions in subtlety and duration. Currently, emotion classes within the CASME II dataset (Chinese Academy of Sciences Micro-expression II) are based on Action Units and self-reports, creating conflicts during machine learning training. We will show that classifying expressions using Action Units, instead of predicted emotion, removes the potential bias of human reporting. The proposed classes are tested using LBP-TOP (Local Binary Patterns from Three Orthogonal Planes), HOOF (Histograms of Oriented Optical Flow) and HOG 3D (3D Histogram of Oriented Gradient) feature descriptors. The experiments are evaluated on two benchmark FACS (Facial Action Coding System) coded datasets: CASME II and SAMM (A Spontaneous Micro-Facial Movement). The best result achieves 86.35% accuracy when classifying the proposed 5 classes on CASME II using HOG 3D, outperforming the result of the state-of-the-art 5-class emotional-based classification in CASME II. Results indicate that classification based on Action Units provides an objective method to improve micro-expression recognition.

Rodents are key to generating translational data for healthcare research. Behavioural analyses, in particular, are integral to the non-invasive monitoring of rodent health and welfare. Finding quantitative behavioural measures mitigates stress, allowing for the animal behave freely while also enabling the same animal to be studied over the time-course of its life. Locomotion and whisking are both such quantitative behavioural measures, and have been found to be significantly impacted in rodent models of neurodegenerative disease. While automatic trackers of whiskers and locomotion exist, a manual tracker is required to validate these approaches, and also to annotate complex videos where these automatic versions fail. Manually annotating whiskers for research purposes is a long and tedious task and current software does little to provide an intuitive and simple interface to carry out this task. This led to the creation of the Manual Whisker Annotator (MWA). MWA is an open source, portable whisker annotation tool developed for the Windows platform. Not only does MWA make the process much quicker, it also provides added statistical tools to analyse the data. MWA was developed in C# and WPF using the .NET framework, and could be used in any situation where annotating or tracking multiple targets is desired.

Modern facial motion capture systems employ a two-pronged approach for capturing and rendering facial motion. Visual data (2D) is used for tracking the facial features and predicting facial expression, whereas Depth (3D) data is used to build a series of expressions on 3D face models. An issue with modern research approaches is the use of a single data stream that provides little indication of the 3D facial structure. We compare and analyse the performance of Convolutional Neural Networks (CNN) using visual, Depth and merged data to identify facial features in real-time using a Depth sensor. First, we review the facial landmarking algorithms and its datasets for Depth data. We address the limitation of the current datasets by introducing the Kinect One Expression Dataset (KOED). Then, we propose the use of CNNs for the single data stream and merged data streams for facial landmark detection. We contribute to existing work by performing a full evaluation on which streams are the most effective for the field of facial landmarking. Furthermore, we improve upon the existing work by extending neural networks to predict into 3D landmarks in real-time with additional observations on the impact of using 2D landmarks as auxiliary information. We evaluate the performance by using Mean Square Error (MSE) and Mean Average Error (MAE). We observe that the single data stream predicts accurate facial landmarks on Depth data when auxiliary information is used to train the network. The codes and dataset used in this paper will be made available.

The aim of this study was to compare the performance between young adults (n = 15), healthy old people (n = 10), and masters athletes (n = 15) using a depth sensor and automated digital assessment framework. Participants were asked to complete a clinically validated assessment of the sit-to-stand technique (five repetitions), which was recorded using a depth sensor. A feature encoding and evaluation framework to assess balance, core, and limb performance using time- and speed-related measurements was applied to markerless motion capture data. The associations between the measurements and participant groups were examined and used to evaluate the assessment framework suitability. The proposed framework could identify phases of sit-to-stand, stability, transition style, and performance between participant groups with a high degree of accuracy. In summary, we found that a depth sensor coupled with the proposed framework could identify performance subtleties between groups.