Search

This paper proposes a feedback mechanism to 'break bad habits' using the Pavlok device. Pavlok utilises beeps, vibration and shocks as a mode of aversion technique to help individuals with behaviour modification. While the device can be useful in certain periodic daily life situations, like alarms and exercise notifications, the device relies on manual operations that limit its usage. To this end, we design a user interface to generate an automatic feedback mechanism that integrates Pavlok and a deep learning based model to detect certain behaviours via an integrated user interface i.e. mobile or desktop application. Our proposed solution is implemented and verified in the context of snoring, which first detects audio from the environment following a prediction of whether the audio content is a snore or not. Based on the prediction of the deep learning model, we use Pavlok to alert users for preventive measures. We believe that this simple solution can help people to change their atomic habits, which may lead to long-term benefits., Comment: Shreya Ghosh, Rakibul Hasan and Pradyumna Agrawal contributed equally to this research

We introduce NightPulse, an interactive tool for Night-time light (NTL) data visualization and analytics, which enables researchers and stakeholders to explore and analyze NTL data with a user-friendly platform. Powered by efficient system architecture, NightPulse supports image segmentation, clustering, and change pattern detection to identify urban development and sprawl patterns. It captures temporal trends of NTL and semantics of cities, answering questions about demographic factors, city boundaries, and unusual differences., Comment: 5 pages, 3 figures. Accepted in ECML PKDD Demo track

Most deepfake detection methods focus on detecting spatial and/or spatio-temporal changes in facial attributes and are centered around the binary classification task of detecting whether a video is real or fake. This is because available benchmark datasets contain mostly visual-only modifications present in the entirety of the video. However, a sophisticated deepfake may include small segments of audio or audio-visual manipulations that can completely change the meaning of the video content. To addresses this gap, we propose and benchmark a new dataset, Localized Audio Visual DeepFake (LAV-DF), consisting of strategic content-driven audio, visual and audio-visual manipulations. The proposed baseline method, Boundary Aware Temporal Forgery Detection (BA-TFD), is a 3D Convolutional Neural Network-based architecture which effectively captures multimodal manipulations. We further improve (i.e. BA-TFD+) the baseline method by replacing the backbone with a Multiscale Vision Transformer and guide the training process with contrastive, frame classification, boundary matching and multimodal boundary matching loss functions. The quantitative analysis demonstrates the superiority of BA-TFD+ on temporal forgery localization and deepfake detection tasks using several benchmark datasets including our newly proposed dataset. The dataset, models and code are available at https://github.com/ControlNet/LAV-DF., Comment: The paper is under consideration/review at Computer Vision and Image Understanding Journal

Visual perception is an essential cognitive process. Recent research has focused on understanding visual perception through the development of intelligent systems. Designing these systems require accurate estimation of gaze direction. Even with significant progress in the past few years, automatic gaze estimation still remains challenging especially due to the constrained environment based data acquisition setup. To overcome the limitation, this research attempts to develop automatic methods that advance the algorithm for gaze estimation in an unconstrained environment with limited supervision. The proposed methods arising from this thesis have potential applications in gaze based user interfaces and behavioural studies.

This paper proposes a self-supervised approach to learn universal facial representations from videos, that can transfer across a variety of facial analysis tasks such as Facial Attribute Recognition (FAR), Facial Expression Recognition (FER), DeepFake Detection (DFD), and Lip Synchronization (LS). Our proposed framework, named MARLIN, is a facial video masked autoencoder, that learns highly robust and generic facial embeddings from abundantly available non-annotated web crawled facial videos. As a challenging auxiliary task, MARLIN reconstructs the spatio-temporal details of the face from the densely masked facial regions which mainly include eyes, nose, mouth, lips, and skin to capture local and global aspects that in turn help in encoding generic and transferable features. Through a variety of experiments on diverse downstream tasks, we demonstrate MARLIN to be an excellent facial video encoder as well as feature extractor, that performs consistently well across a variety of downstream tasks including FAR (1.13% gain over supervised benchmark), FER (2.64% gain over unsupervised benchmark), DFD (1.86% gain over unsupervised benchmark), LS (29.36% gain for Frechet Inception Distance), and even in low data regime. Our code and models are available at https://github.com/ControlNet/MARLIN ., Comment: CVPR 2023

Supplementary information

Eye gaze analysis is an important research problem in the field of Computer Vision and Human-Computer Interaction. Even with notable progress in the last 10 years, automatic gaze analysis still remains challenging due to the uniqueness of eye appearance, eye-head interplay, occlusion, image quality, and illumination conditions. There are several open questions, including what are the important cues to interpret gaze direction in an unconstrained environment without prior knowledge and how to encode them in real-time. We review the progress across a range of gaze analysis tasks and applications to elucidate these fundamental questions, identify effective methods in gaze analysis, and provide possible future directions. We analyze recent gaze estimation and segmentation methods, especially in the unsupervised and weakly supervised domain, based on their advantages and reported evaluation metrics. Our analysis shows that the development of a robust and generic gaze analysis method still needs to address real-world challenges such as unconstrained setup and learning with less supervision. We conclude by discussing future research directions for designing a real-world gaze analysis system that can propagate to other domains including Computer Vision, Augmented Reality (AR), Virtual Reality (VR), and Human Computer Interaction (HCI). Project Page: https://github.com/i-am-shreya/EyeGazeSurvey}{https://github.com/i-am-shreya/EyeGazeSurvey

Statistics reveal a huge increase in cyberattacks making technology businesses more susceptible to data loss. With increasing application of machine learning in different domains, studies have been focused on building cognitive models for traffic anomaly detection in a communication network. These studies have led to generation of datasets containing network traffic data packets, usually captured using softwares like Wireshark. These datasets contain high dimensional data corresponding to benign data packets and attack data packets of known attacks. Recent research has mainly focused on developing machine learning architectures that are able to extract useful information from high dimensional datasets to detect attack data packets in a network. In addition, machine learning algorithms are currently trained to detect only documented attacks with available training data. However, with the proliferation of new cyberattacks and zero-day attacks with little to no training data available, current employed algorithms have little to no success in detecting new attacks. In this thesis, we focus on detecting rare attacks using transfer learning from a dataset containing information pertaining to known attacks.In the literature, there is proof of concept for both classical machine learning and deep learning approaches for anomaly detection. We show that a deep learning approach outperforms explicit statistical modeling based approaches by at least 21% for the used dataset. We perform a preliminary survey of candidate deep learning architectures before testing for transferability and propose a Convolutional Neural Network architecture that is 99.65% accurate in classifying attack data packets.To test for transferability, we train this proposed CNN architecture with a known attack and test it's performance on attacks that are unknown to the network. For this model to extract adequate information for transferability, the model requires a higher representation of attack data in the training dataset with the current attack data comprising only 20% of the dataset. To overcome the problem of small training sets, several techniques to boost the number of attack data packets are employed like a novel synthetic dataset based training and bootstrapped dataset training.Our study results in identification of training-testing attack pairs that show high learning transferability. Most of the strong and consistent correlations are observed among Denial of Service(DoS) training-testing attack pairs. Furthermore, we propose hypotheses for model generalization. Our results are validated by a study of dataset features and attack characteristics using the Recursive Feature Elimination(RFE) algorithm.

The first part of this work explores DNA aptamer based optical sensors suitable for biomedical applications. These are highly selective towards a wide variety of biomolecules. Such sensors (also called molecular beacons) operate on the principle of Fluorescent Resonance Energy Transfer. Sensors detecting Glycated Albumin, Tumor Necrosis Factor – alpha, C – Reactive Protein and calcium ions, respectively, have been designed and characterized. Ultra low detection capabilities in the picomolar concentration ranges have been observed in sensors detecting C – Reactive Protein and calcium ions. The second part of this work involves the modification of the structure of the molecular beacons. They have been conjugated with a cell penetrating peptide called DSS, which assisted the molecular beacons in crossing the biological cell membrane. This design has been used to characterize the response of the aptamer sensors in an intracellular environment. In another study, the molecular beacons have been anchored on to a graphene oxide substrate for the detection of C – Reactive Protein. The optical properties of these substrate conjugated nanoconstructs have been investigated in the presence of human serum based simulated samples. In both cases of structural modifications, the research opened up a new avenue in theranostics because the DSS peptide as well as the graphene oxide can be easily functionalized with drugs for future drug delivery applications. The third part of this work involves the analysis of the Raman / SERS signatures of several biomolecules. Raman spectra and peak assignments of biomarker proteins, associated to inflammatory diseases such as diabetes mellitus and sepsis, have been studied. Another aspect of this study employs a SERS substrate composed of silver nanorod arrays to obtain the Raman spectra of complex bio fluids like human serum. The main focus of this part of the work is to obtain information on the composition of the biomolecules as well the secondary structure of the biomarker proteins.

Distributed denial of service (DDoS) attacks are a constant threat for services in the Internet. This year, the record for the largest DDoS attack ever observed was set at 1.7 Tbps. Meanwhile, detection and mitigation mechanisms are still lacking behind. Many mitigation systems require the assistance by the victim - or the victim's administrator themself has to become active to mitigate attacks. We introduced a system that can detect attacks, identify attackers, and mitigate the attacks purely within the network infrastructure. With the improved flexibility of software-defined networks, new possibilities to mitigate such attacks can be implemented. In addition to our short paper on the mitigation of reflective DDoS attacks on LCN 2018, we also like to demonstrate our work on mitigating flooding attacks presented at LCN 2017 and our mitigation of slow DDoS attacks. In our demo, we show how these systems can be combined and how they work when faced with such different attacks.

Identity in south Asia was anchored by, on the one end, community, and on the other, an appreciation of sub-continental (geographical and cultural) space. People, historically, drew their identity as part of communities, which in turn existed in continuity to each-other in the seamless regional expanse of south Asia. Imagination as nationals - a post-colonial construct faced contestations, both, from community affiliations and spatial imagination contrary to the territorial - modular form of nation-state. As a response, the state fabricated the idea of ‗patriotic-citizen‘ and used nationalist historiography to create citizens who are taught to believe the nation as prime-marker of self-definition and act like soldiers, guarding national identity against alternative imaginations. Education has become the most potent devise through which this is achieved. The article, on the basis of textbook narratives in Bangladesh, India and Pakistan, would demonstrate (i) how educational practices build a militarist idea of citizenship and, (ii) in doing so co-opts the demands of community by showing the nation as vindication of community-aspirations and on the other hand erasing conceptualisation of a south Asian space from cognitive maps of its subjects. The idea of ‗active‘ citizenship understands ‗active‘ as responsible citizenship, emphasising a right based discourse. On the contrary education in south Asia is used to ‗activate‘ citizenship which is relational in content - based on ideas of ‗us‘ versus ‗them‘ – instead of allowing critical understanding of rights and identities