Search

The interaction between elephants and their environment has profound implications for both ecology and conservation strategies. This study presents an analytical approach to decipher the intricate patterns of elephant movement in Sub-Saharan Africa, concentrating on key ecological drivers such as seasonal variations and rainfall patterns. Despite the complexities surrounding these influential factors, our analysis provides a holistic view of elephant migratory behavior in the context of the dynamic African landscape. Our comprehensive approach enables us to predict the potential impact of these ecological determinants on elephant migration, a critical step in establishing informed conservation strategies. This projection is particularly crucial given the impacts of global climate change on seasonal and rainfall patterns, which could substantially influence elephant movements in the future. The findings of our work aim to not only advance the understanding of movement ecology but also foster a sustainable coexistence of humans and elephants in Sub-Saharan Africa. By predicting potential elephant routes, our work can inform strategies to minimize human-elephant conflict, effectively manage land use, and enhance anti-poaching efforts. This research underscores the importance of integrating movement ecology and climatic variables for effective wildlife management and conservation planning., 11 pages, 17 figures, Accepted in ACM SIGCAS SIGCHI Conference on Computing and Sustainable Societies (COMPASS 2023)

In this paper, we lay out a vision for analysing semantic trajectory traces and generating synthetic semantic trajectory data (SSTs) using generative language model. Leveraging the advancements in deep learning, as evident by progress in the field of natural language processing (NLP), computer vision, etc. we intend to create intelligent models that can study the semantic trajectories in various contexts, predicting future trends, increasing machine understanding of the movement of animals, humans, goods, etc. enhancing human-computer interactions, and contributing to an array of applications ranging from urban-planning to personalized recommendation engines and business strategy., 8 pages, 1 figure, Submitted for peer review

Automatic group emotion recognition plays an important role in understanding complex human-human interaction. This paper introduces, Emolysis, a standalone open-source toolkit for real-time multimodal group emotion recognition and visualization. Given any input video, Emolysis processes nearly real-time synchronized multimodal input and maps it to group level emotion, valence and arousal. Additionally, the toolkit supports major mobile and desktop platforms (Android, iOS, Windows). The Emolysis platform also comes with an intuitive graphical user interface that allows users to select different modalities and target persons for more fine grained emotion analysis. Emolysis is freely available for academic research, and encourages application developers to extend it to application specific environments on top of the existing system. We believe that the extension mechanism is quite straightforward. Our code and models are available at https://github.com/ControlNet/emolysis., Both Shreya Ghosh and Zhixi Cai contributed equally to this research

This paper proposes a feedback mechanism to 'break bad habits' using the Pavlok device. Pavlok utilises beeps, vibration and shocks as a mode of aversion technique to help individuals with behaviour modification. While the device can be useful in certain periodic daily life situations, like alarms and exercise notifications, the device relies on manual operations that limit its usage. To this end, we design a user interface to generate an automatic feedback mechanism that integrates Pavlok and a deep learning based model to detect certain behaviours via an integrated user interface i.e. mobile or desktop application. Our proposed solution is implemented and verified in the context of snoring, which first detects audio from the environment following a prediction of whether the audio content is a snore or not. Based on the prediction of the deep learning model, we use Pavlok to alert users for preventive measures. We believe that this simple solution can help people to change their atomic habits, which may lead to long-term benefits., Comment: Shreya Ghosh, Rakibul Hasan and Pradyumna Agrawal contributed equally to this research

We introduce NightPulse, an interactive tool for Night-time light (NTL) data visualization and analytics, which enables researchers and stakeholders to explore and analyze NTL data with a user-friendly platform. Powered by efficient system architecture, NightPulse supports image segmentation, clustering, and change pattern detection to identify urban development and sprawl patterns. It captures temporal trends of NTL and semantics of cities, answering questions about demographic factors, city boundaries, and unusual differences., Comment: 5 pages, 3 figures. Accepted in ECML PKDD Demo track

Most deepfake detection methods focus on detecting spatial and/or spatio-temporal changes in facial attributes and are centered around the binary classification task of detecting whether a video is real or fake. This is because available benchmark datasets contain mostly visual-only modifications present in the entirety of the video. However, a sophisticated deepfake may include small segments of audio or audio-visual manipulations that can completely change the meaning of the video content. To addresses this gap, we propose and benchmark a new dataset, Localized Audio Visual DeepFake (LAV-DF), consisting of strategic content-driven audio, visual and audio-visual manipulations. The proposed baseline method, Boundary Aware Temporal Forgery Detection (BA-TFD), is a 3D Convolutional Neural Network-based architecture which effectively captures multimodal manipulations. We further improve (i.e. BA-TFD+) the baseline method by replacing the backbone with a Multiscale Vision Transformer and guide the training process with contrastive, frame classification, boundary matching and multimodal boundary matching loss functions. The quantitative analysis demonstrates the superiority of BA-TFD+ on temporal forgery localization and deepfake detection tasks using several benchmark datasets including our newly proposed dataset. The dataset, models and code are available at https://github.com/ControlNet/LAV-DF., Comment: The paper is under consideration/review at Computer Vision and Image Understanding Journal

Automatic eye gaze estimation is an important problem in vision based assistive technology with use cases in different emerging topics such as augmented reality, virtual reality and human-computer interaction. Over the past few years, there has been an increasing interest in unsupervised and self-supervised learning paradigms as it overcomes the requirement of large scale annotated data. In this paper, we propose RAZE, a Region guided self-supervised gAZE representation learning framework which leverage from non-annotated facial image data. RAZE learns gaze representation via auxiliary supervision i.e. pseudo-gaze zone classification where the objective is to classify visual field into different gaze zones (i.e. left, right and center) by leveraging the relative position of pupil-centers. Thus, we automatically annotate pseudo gaze zone labels of 154K web-crawled images and learn feature representations via `Ize-Net' framework. `Ize-Net' is a capsule layer based CNN architecture which can efficiently capture rich eye representation. The discriminative behaviour of the feature representation is evaluated on four benchmark datasets: CAVE, TabletGaze, MPII and RT-GENE. Additionally, we evaluate the generalizability of the proposed network on two other downstream task (i.e. driver gaze estimation and visual attention estimation) which demonstrate the effectiveness of the learnt eye gaze representation., arXiv admin note: substantial text overlap with arXiv:1904.02459

Over the past few years, there has been an increasing interest to interpret gaze direction in an unconstrained environment with limited supervision. Owing to data curation and annotation issues, replicating gaze estimation method to other platforms, such as unconstrained outdoor or AR/VR, might lead to significant drop in performance due to insufficient availability of accurately annotated data for model training. In this paper, we explore an interesting yet challenging problem of gaze estimation method with a limited amount of labelled data. The proposed method distills knowledge from the labelled subset with visual features; including identity-specific appearance, gaze trajectory consistency and motion features. Given a gaze trajectory, the method utilizes label information of only the start and the end frames of a gaze sequence. An extension of the proposed method further reduces the requirement of labelled frames to only the start frame with a minor drop in the generated label's quality. We evaluate the proposed method on four benchmark datasets (CAVE, TabletGaze, MPII and Gaze360) as well as web-crawled YouTube videos. Our proposed method reduces the annotation effort to as low as 2.67%, with minimal impact on performance; indicating the potential of our model enabling gaze estimation 'in-the-wild' setup.

In challenging real-life conditions such as extreme head-pose, occlusions, and low-resolution images where the visual information fails to estimate visual attention/gaze direction, audio signals could provide important and complementary information. In this paper, we explore if audio-guided coarse head-pose can further enhance visual attention estimation performance for non-prolific faces. Since it is difficult to annotate audio signals for estimating the head-pose of the speaker, we use off-the-shelf state-of-the-art models to facilitate cross-modal weak-supervision. During the training phase, the framework learns complementary information from synchronized audio-visual modality. Our model can utilize any of the available modalities i.e. audio, visual or audio-visual for task-specific inference. It is interesting to note that, when AV-Gaze is tested on benchmark datasets with these specific modalities, it achieves competitive results on multiple datasets, while being highly adaptive toward challenging scenarios.

Visual perception is an essential cognitive process. Recent research has focused on understanding visual perception through the development of intelligent systems. Designing these systems require accurate estimation of gaze direction. Even with significant progress in the past few years, automatic gaze estimation still remains challenging especially due to the constrained environment based data acquisition setup. To overcome the limitation, this research attempts to develop automatic methods that advance the algorithm for gaze estimation in an unconstrained environment with limited supervision. The proposed methods arising from this thesis have potential applications in gaze based user interfaces and behavioural studies.

This paper proposes a self-supervised approach to learn universal facial representations from videos, that can transfer across a variety of facial analysis tasks such as Facial Attribute Recognition (FAR), Facial Expression Recognition (FER), DeepFake Detection (DFD), and Lip Synchronization (LS). Our proposed framework, named MARLIN, is a facial video masked autoencoder, that learns highly robust and generic facial embeddings from abundantly available non-annotated web crawled facial videos. As a challenging auxiliary task, MARLIN reconstructs the spatio-temporal details of the face from the densely masked facial regions which mainly include eyes, nose, mouth, lips, and skin to capture local and global aspects that in turn help in encoding generic and transferable features. Through a variety of experiments on diverse downstream tasks, we demonstrate MARLIN to be an excellent facial video encoder as well as feature extractor, that performs consistently well across a variety of downstream tasks including FAR (1.13% gain over supervised benchmark), FER (2.64% gain over unsupervised benchmark), DFD (1.86% gain over unsupervised benchmark), LS (29.36% gain for Frechet Inception Distance), and even in low data regime. Our code and models are available at https://github.com/ControlNet/MARLIN ., Comment: CVPR 2023

Supplementary information

Eye gaze analysis is an important research problem in the field of Computer Vision and Human-Computer Interaction. Even with notable progress in the last 10 years, automatic gaze analysis still remains challenging due to the uniqueness of eye appearance, eye-head interplay, occlusion, image quality, and illumination conditions. There are several open questions, including what are the important cues to interpret gaze direction in an unconstrained environment without prior knowledge and how to encode them in real-time. We review the progress across a range of gaze analysis tasks and applications to elucidate these fundamental questions, identify effective methods in gaze analysis, and provide possible future directions. We analyze recent gaze estimation and segmentation methods, especially in the unsupervised and weakly supervised domain, based on their advantages and reported evaluation metrics. Our analysis shows that the development of a robust and generic gaze analysis method still needs to address real-world challenges such as unconstrained setup and learning with less supervision. We conclude by discussing future research directions for designing a real-world gaze analysis system that can propagate to other domains including Computer Vision, Augmented Reality (AR), Virtual Reality (VR), and Human Computer Interaction (HCI). Project Page: https://github.com/i-am-shreya/EyeGazeSurvey}{https://github.com/i-am-shreya/EyeGazeSurvey

Statistics reveal a huge increase in cyberattacks making technology businesses more susceptible to data loss. With increasing application of machine learning in different domains, studies have been focused on building cognitive models for traffic anomaly detection in a communication network. These studies have led to generation of datasets containing network traffic data packets, usually captured using softwares like Wireshark. These datasets contain high dimensional data corresponding to benign data packets and attack data packets of known attacks. Recent research has mainly focused on developing machine learning architectures that are able to extract useful information from high dimensional datasets to detect attack data packets in a network. In addition, machine learning algorithms are currently trained to detect only documented attacks with available training data. However, with the proliferation of new cyberattacks and zero-day attacks with little to no training data available, current employed algorithms have little to no success in detecting new attacks. In this thesis, we focus on detecting rare attacks using transfer learning from a dataset containing information pertaining to known attacks.In the literature, there is proof of concept for both classical machine learning and deep learning approaches for anomaly detection. We show that a deep learning approach outperforms explicit statistical modeling based approaches by at least 21% for the used dataset. We perform a preliminary survey of candidate deep learning architectures before testing for transferability and propose a Convolutional Neural Network architecture that is 99.65% accurate in classifying attack data packets.To test for transferability, we train this proposed CNN architecture with a known attack and test it's performance on attacks that are unknown to the network. For this model to extract adequate information for transferability, the model requires a higher representation of attack data in the training dataset with the current attack data comprising only 20% of the dataset. To overcome the problem of small training sets, several techniques to boost the number of attack data packets are employed like a novel synthetic dataset based training and bootstrapped dataset training.Our study results in identification of training-testing attack pairs that show high learning transferability. Most of the strong and consistent correlations are observed among Denial of Service(DoS) training-testing attack pairs. Furthermore, we propose hypotheses for model generalization. Our results are validated by a study of dataset features and attack characteristics using the Recursive Feature Elimination(RFE) algorithm.

With the pervasiveness of IoT devices, smart-phones and improvement of location-tracking technologies huge volume of heterogeneous geo-tagged (location specific) data is generated which facilitates several location-aware services. The analytics with this spatio-temporal (having location and time dimensions) datasets provide varied important services such as, smart transportation, emergency services (health-care, national defence or urban planning). While cloud paradigm is suitable for the capability of storage and computation, the major bottleneck is network connectivity loss. In time-critical application, where real-time response is required for emergency service-provisioning, such connectivity issues increases the latency and thus affects the overall quality of system (QoS). To overcome the issue, fog/ edge topology has emerged, where partial computation is carried out in the edge of the network to reduce the delay in communication. Such fog/ edge based system complements the cloud technology and extends the features of the system. This chapter discusses cloud-fog-edge based hierarchical collaborative framework, where several components are deployed to improve the QoS. On the other side. mobility is another critical factor to enhance the efficacy of such location-aware service provisioning. Therefore, this chapter discusses the concerns and challenges associated with mobility-driven cloud-fog-edge based framework to provide several location-aware services to the end-users efficiently., 17 pages, 4 figures, Submitted as a book-chapter in Mobile Edge Computing, Springer

The first part of this work explores DNA aptamer based optical sensors suitable for biomedical applications. These are highly selective towards a wide variety of biomolecules. Such sensors (also called molecular beacons) operate on the principle of Fluorescent Resonance Energy Transfer. Sensors detecting Glycated Albumin, Tumor Necrosis Factor – alpha, C – Reactive Protein and calcium ions, respectively, have been designed and characterized. Ultra low detection capabilities in the picomolar concentration ranges have been observed in sensors detecting C – Reactive Protein and calcium ions. The second part of this work involves the modification of the structure of the molecular beacons. They have been conjugated with a cell penetrating peptide called DSS, which assisted the molecular beacons in crossing the biological cell membrane. This design has been used to characterize the response of the aptamer sensors in an intracellular environment. In another study, the molecular beacons have been anchored on to a graphene oxide substrate for the detection of C – Reactive Protein. The optical properties of these substrate conjugated nanoconstructs have been investigated in the presence of human serum based simulated samples. In both cases of structural modifications, the research opened up a new avenue in theranostics because the DSS peptide as well as the graphene oxide can be easily functionalized with drugs for future drug delivery applications. The third part of this work involves the analysis of the Raman / SERS signatures of several biomolecules. Raman spectra and peak assignments of biomarker proteins, associated to inflammatory diseases such as diabetes mellitus and sepsis, have been studied. Another aspect of this study employs a SERS substrate composed of silver nanorod arrays to obtain the Raman spectra of complex bio fluids like human serum. The main focus of this part of the work is to obtain information on the composition of the biomolecules as well the secondary structure of the biomarker proteins.

Distributed denial of service (DDoS) attacks are a constant threat for services in the Internet. This year, the record for the largest DDoS attack ever observed was set at 1.7 Tbps. Meanwhile, detection and mitigation mechanisms are still lacking behind. Many mitigation systems require the assistance by the victim - or the victim's administrator themself has to become active to mitigate attacks. We introduced a system that can detect attacks, identify attackers, and mitigate the attacks purely within the network infrastructure. With the improved flexibility of software-defined networks, new possibilities to mitigate such attacks can be implemented. In addition to our short paper on the mitigation of reflective DDoS attacks on LCN 2018, we also like to demonstrate our work on mitigating flooding attacks presented at LCN 2017 and our mitigation of slow DDoS attacks. In our demo, we show how these systems can be combined and how they work when faced with such different attacks.

Identity in south Asia was anchored by, on the one end, community, and on the other, an appreciation of sub-continental (geographical and cultural) space. People, historically, drew their identity as part of communities, which in turn existed in continuity to each-other in the seamless regional expanse of south Asia. Imagination as nationals - a post-colonial construct faced contestations, both, from community affiliations and spatial imagination contrary to the territorial - modular form of nation-state. As a response, the state fabricated the idea of ‗patriotic-citizen‘ and used nationalist historiography to create citizens who are taught to believe the nation as prime-marker of self-definition and act like soldiers, guarding national identity against alternative imaginations. Education has become the most potent devise through which this is achieved. The article, on the basis of textbook narratives in Bangladesh, India and Pakistan, would demonstrate (i) how educational practices build a militarist idea of citizenship and, (ii) in doing so co-opts the demands of community by showing the nation as vindication of community-aspirations and on the other hand erasing conceptualisation of a south Asian space from cognitive maps of its subjects. The idea of ‗active‘ citizenship understands ‗active‘ as responsible citizenship, emphasising a right based discourse. On the contrary education in south Asia is used to ‗activate‘ citizenship which is relational in content - based on ideas of ‗us‘ versus ‗them‘ – instead of allowing critical understanding of rights and identities