Search

Bias and fairness of biometric algorithms have been key topics of research in recent years, mainly due to the societal, legal and ethical implications of potentially unfair decisions made by automated decision-making models. A considerable amount of work has been done on this topic across different biometric modalities, aiming at better understanding the main sources of algorithmic bias or devising mitigation measures. In this work, we contribute to these efforts and present the first study investigating bias and fairness of sclera segmentation models. Although sclera segmentation techniques represent a key component of sclera-based biometric systems with a considerable impact on the overall recognition performance, the presence of different types of biases in sclera segmentation methods is still underexplored. To address this limitation, we describe the results of a group evaluation effort (involving seven research groups), organized to explore the performance of recent sclera segmentation models within a common experimental framework and study performance differences (and bias), originating from various demographic as well as environmental factors. Using five diverse datasets, we analyze seven independently developed sclera segmentation models in different experimental configurations. The results of our experiments suggest that there are significant differences in the overall segmentation performance across the seven models and that among the considered factors, ethnicity appears to be the biggest cause of bias. Additionally, we observe that training with representative and balanced data does not necessarily lead to less biased results. Finally, we find that in general there appears to be a negative correlation between the amount of bias observed (due to eye color, ethnicity and acquisition device) and the overall segmentation performance, suggesting that advances in the field of semantic segmentation may also help with mitigating bias.

Altering handwritten documents is receiving special attention from researchers because it is useful in several sensitive crime applications such as the identification of suicide notes, fraudulent certificates, fake answer scripts, bank and property documents etc. This paper aims at developing a robust method for detecting altered text in handwritten document images in noisy and blurry environments. This study considers ten classes of handwritten text affected by multiple forgery operations with noise and blur as follows: (i) Normal-original text, (ii) Copy-paste forgery, (iii) Insertion forgery, (iv) Copy-paste and insertion forgery, (v) Noisy text, (vi) Blurred text, (vii) Copy-paste forgery with noise, (viii) Copy-paste forgery with blur, (ix) Insertion forgery with noise and (x) Insertion with blur. For ten-class classification, the proposed work explores the combination of statistical, gradient and texture features with a Bayesian classifier. The proposed approach works based on the premise that altered content in noisy and blurry handwritten documents exhibits inconsistent patterns of pixel arrangements while the original text exhibits a regular pattern of pixel arrangements. Comprehensive experiments on our dataset of 10-class and three standard datasets, namely, a dataset of forged handwritten text, a dataset of altered receipt images, and a dataset of forged IMEI number images are conducted to show effectiveness and robustness of the proposed approach compared to the state-of-the-art methods.

Handwritten image-based writer age estimation is a challenging task due to the various writing styles of different individuals, use of different scripts, varying alignment, etc. Unlike age estimation using face recognition in biometrics, handwriting-based age classification is reliable and inexpensive because of the plain backgrounds of documents. This paper presents a novel model for deriving the phase spectrum based on the Harmonic Wavelet Transform (HWT) for age classification on handwritten images from 11 to 65 years. This includes 11 classes with an interval of 5 years. In contrast to the Fourier transform, which provides a noisy phase spectrum due to loss of time variations, the proposed HWT-based phase spectrum retains time variations of phase and magnitude. As a result, the proposed HWT-based phase spectrum preserves vital information of changes in handwritten images. In order to extract such information, we propose new phase statistics-based features for age classification based on the understanding that as age changes, writing style also changes. The features and the input images are fed to a VGG-16 model for age classification. The proposed method is tested on our own dataset and three standard datasets, namely, IAM-2, KHATT and that of Basavaraja et al. to demonstrate the effectiveness of the proposed model compared to the existing methods in terms of classification rate. The results of the proposed and existing methods on different datasets show that the proposed method outperforms the existing methods in terms of classification rate.

With the evolution of electronic devices, such as 3D cameras, addressing the challenges of text localization in 3D video (e.g., for indexing) is increasingly drawing the attention of the multimedia and video processing community. Existing methods focus on 2D video and their performance in the presence of the challenges in 3D video, such as shadow areas associated with text and irregularly sized and shaped text, degrades. This paper proposes the first approach that successfully addresses the challenges of 3D video in addition to those of 2D. It employs a number of innovations, among which, the first is the Generalized Gradient Vector Flow (GGVF) for dominant points detection. The second is the Wavefront concept for text candidate point detection from those dominant points. In addition, an Adaptive B-Spline Polygon Curve Network (ABS-Net) is proposed for accurate text localization in 3D videos by constructing tight fitting bounding polygons using text candidate points. Extensive experiments on custom (3D video) and standard datasets (2D video and scene text) show that the proposed method is practical and useful, and overall outperforms existing state-of-the-art methods.

Aesthetic image analysis is essential for improving the performance of multimedia image retrieval systems, especially from a repository of social media and multimedia content stored on mobile devices. This paper presents a novel method for classifying aesthetic natural scene images by studying the naturalness of image content using statistical features, and reading text in the images using semantic features. Unlike existing methods that focus only on image quality with human information, the proposed approach focuses on image features as well as text-based semantic features without human intervention to reduce the gap between subjectivity and objectivity in the classification. The aesthetic classes considered in this work are (i) Very Pleasant, (ii) Pleasant, (iii) Normal and (iv) Unpleasant. The naturalness is represented by features of focus, defocus, perceived brightness, perceived contrast, blurriness and noisiness, while semantics are represented by text recognition, description of the images and labels of images, profile pictures, and banner images. Furthermore, a deep learning model is proposed in a novel way to fuse statistical and semantic features for the classification of aesthetic natural scene images. Experiments on our own dataset and the standard datasets demonstrate that the proposed approach achieves 92.74%, 88.67% and 83.22% average classification rates on our own dataset, AVA dataset and CUHKPQ dataset, respectively. Furthermore, a comparative study of the proposed model with the existing methods shows that the proposed method is effective for the classification of aesthetic social media images.

Due to the proliferation of sports-related multimedia content on the WWW, effective visual search and retrieval present interesting research challenges. These are caused by poor image quality, a wide range of possible camera points of view, pose variations on the part of athletes engaged in playing a sport, deformations of text appearing on sports person’s clothing and uniforms in motion, occlusions caused by other objects, etc. To address these challenges, this paper presents a new method for detecting text on human bodies in sports images. Unlike most existing methods, which attempt to exploit locations of a player’s torso, face, and skin, we propose an end-to-end episodic learning approach that employs inductive learning criteria for detecting clothing regions in an image, which are, in turn, then used for text detection. Our method integrates a Residual Network (ResNet) and Pyramidal Pooling Module (PPM) for generating a spatial attention map. The Progressive Scalable Expansion Algorithm (PSE) is adapted for text detection from these regions. Experimental results on our own dataset as well as several benchmarks (like RBNR and MMM which contain images of runners in marathons, and Re-ID which is a person re-identification dataset) demonstrate that the proposed method outperforms existing methods in terms of precision and F1-score. We also present results for sports images chosen from natural scene text detection datasets such as CTW1500 and MS-COCO to show the proposed method is effective and reliable across a range of inputs.

When the images are captured by drones, the effect of oblique angles, distance variations and open environment are the main challenges for successful palm tree detection. This paper presents a method towards palm tree counting in Drone images using a novel idea of detecting dominant points by exploring Generalized Gradient Vector Flow, which defines symmetry based on gradient direction of the pixels. For each dominant point, we use angle information for classifying diagonal dominant points. It is intuition that the direction of the branches of tree converges at center of tree irrespective of the type of tree and plants. This observation motivated us to expand the direction of diagonal dominant points until it finds intersection point with another diagonal dominant point and this results in candidate points. For each candidate point, the proposed method constructs the ring by considering the distance between the intersection point and nearest neighbor candidate point as radius. This outputs region of interest and it includes center of each tree in the image. To ease the effect of complex background, we explore YOLOv5 architecture to remove false region of interests. This step results in counting oil palm trees in the mages irrespective of tree type of palm family. Experimental results on our dataset of the images captured by drones and standard dataset of coconut images captured by unmanned aerial vehicle of different trees show that the proposed method is effective and performs better than SOTA methods.