Search

Introduction: Sign language is the only way to communicate for speech-impaired people. But this sign language is not known to normal people so this is the cause of barrier in communicating. This is the problem faced by speech impaired people. In this paper, we have presented our solution which captured hand gestures with Kinect camera and classified the hand gesture into its correct symbol. Method: We used Kinect camera not the ordinary web camera because the ordinary camera does not capture its 3d orientation or depth of an image from camera however Kinect camera can capture 3d image and this will make classification more accurate. Result: Kinect camera will produce a different image for hand gestures for ‘2’ and ‘V’ and similarly for ‘1’ and ‘I’ however, normal web camera will not be able to distinguish between these two. We used hand gesture for Indian sign language and our dataset had 46339, RGB images and 46339 depth images. 80% of the total images were used for training and the remaining 20% for testing. In total 36 hand gestures were considered to capture alphabets and alphabets from A-Z and 10 for numeric, 26 for digits from 0-9 were considered to capture alphabets and Keywords. Conclusion: Along with real-time implementation, we have also shown the comparison of the performance of the various machine learning models in which we have found out the accuracy of CNN on depth- images has given the most accurate performance than other models. All these resulted were obtained on PYNQ Z2 board.

Speech impaired people use hand based gestures to communicate. Unfortunately, the vast majority of the people are not aware of the semantics of these gestures. In a attempt to bridge the same, we propose a real time hand gesture recognition system based on the data captured by the Microsoft Kinect RGB-D camera. Given that there is no one to one mapping between the pixels of the depth and the RGB camera, we used computer vision techniques like 3D contruction and affine transformation. After achieving one to one mapping, segmentation of the hand gestures was done from the background noise. Convolutional Neural Networks (CNNs) were utilised for training 36 static gestures relating to Indian Sign Language (ISL) alphabets and numbers. The model achieved an accuracy of 98.81% on training using 45,000 RGB images and 45,000 depth images. Further Convolutional LSTMs were used for training 10 ISL dynamic word gestures and an accuracy of 99.08% was obtained by training 1080 videos. The model showed accurate real time performance on prediction of ISL static gestures, leaving a scope for further research on sentence formation through gestures. The model also showed competitive adaptability to American Sign Language (ASL) gestures when the ISL models weights were transfer learned to ASL and it resulted in giving 97.71% accuracy.

Deaf or hard-of-hearing people mostly rely on lip-reading to understand speech. They demonstrate the ability of humans to understand speech from visual cues only. Automatic lip reading systems work in a similar fashion - by obtaining speech or text from just the visual information, like a video of a person's face. In this paper, an automatic lip reading system for spoken digit recognition is presented. The system uses simple dynamic features by creating difference images between consecutive frames of the video input. Using this technique, word recognition rates of 83.79 and 65.58 are achieved in speaker-dependent and speaker-independent testing scenarios, respectively. A novel, extended region-of-interest (ROI) which includes lower jaw and neck region is also introduced. Most lip-reading algorithms use only the mouth/lip region for relevant feature extraction. Over simple mouth as the ROI, the proposed ROI improves the performance by 4 in speaker-dependent tests and by 11 in speaker-independent tests. © 2018 Association for Computing Machinery.

Visual Speech Recognition (VSR) deals with the task of extracting speech information from visual cues from a person's face while speaking. Accurate lip segmentation and modeling are essential in any VSR algorithm for good feature extraction. However, lip modeling is a complicated task and is not very robust in natural conditions. This paper describes a novel technique for limited vocabulary visual-only speech recognition that does not use lip modeling. For visual feature extraction, Discrete Cosine Transform (DCT) and Local Binary Pattern (LBP) have been tested. An Error-Correcting Output Codes (ECOC) multi-class model using Support Vector Machine (SVM) binary learners is used for recognition and classification of words.

Person tracking in non-overlapping camera networks is an important problem in many applications. Re-identification plays a significant role for successful tracking. Recently, much progress has been made in re-identification, but the traditional re-identification evaluation is not very suitable for tracking applications. It does not consider specific challenges of re-identification for person tracking such as open-set, dynamic and smaller gallery set and multiple cameras. We propose a new protocol for evaluating re-identification algorithms for person tracking considering issues as mentioned above. We also propose an on-line update scheme for improving the performance of re-identification algorithm. We learn a generic model of the persons off-line, then fine-tune the model to the specific person. We explain the generation of positive and negative samples. We have conducted experiments on six real-world datasets showing the effectiveness of proposed protocol and on-line update.

In this paper we address the problem of tracking a specific person among many in non-overlapping camera networks. It is an important problem with many applications in surveillance and security. Given a person to be tracked by initialising the person in one of the cameras, our objective is to track the person throughout the camera network. We propose a distributed solution where the processing is done in cameras itself and information is shared with neighbours to cooperatively work together. When the person goes out of a field of view (FOV) of a camera, the neighbour cameras go into search mode. We use an on-line fine-tuned re-identification and distributed minimum algorithms to successfully re-identify the person. We performed real-world data experiments showing the performance of the proposed algorithm.

The proposed Brain-Computer Interface system enables Quadriplegic patients, people with severe motor disabilities to send commands to electronic devices and communicate with ease. Interactive sessions are vital for effective knowledge transfer in any learning eco-system. The growth of Brain-Computer Interface (BCI) has led to rapid development in 'Assistive Systems' for the disabled called 'assistive domotics'. Brain-Computer-Interface is capable of reading the brainwaves of an individual and analyse it to obtain some meaningful data. This processed data can be used to assist people having speech disorders and sometimes people with limited locomotion to communicate. In this Project, Emotiv EPOC Headset is used to obtain the electroencephalogram (EEG). The obtained data is processed to communicate pre-defined commands and queries for interactive learning. EEG data can also be used to monitor student's emotional behaviour and provide emotional feedback to the students. Other Vital Information like the heartbeat, blood pressure, ECG and temperature are monitored and uploaded to the server. The Data is processed in Intel Edison, system on chip (SoC). Patient metrics are displayed via Intel IoT Analytics cloud service.

Background/Objectives: Speech emotion recognition (SER) is an important aspect of Human-Computer Interaction systems which is widely used in different sectors like healthcare, robotics, automatic call centres and distance education. Speech emotion recognition involves in depth analysis of the signal and identifying the appropriate emotion based on its trained database using extracted features. Method/Statistical Analysis: This paper aims in devising SER system using linear prediction of the causal part of the autocorrelation sequence (OSALPC) algorithm which has been proven to efficiently reduce noise along with Linear Frequency Cepstral Coefficients (LFCC), Linear Predictive Coding (LPC), MFCC, LPC using cepstrum for feature extraction. After extracting the feature vectors from the voice signal, it is modelled using Gaussian Mixture Models (GMM). The MAP (Maximum a posteriori) rule is used for decision making. Findings: Performance was analysed and our proposed system showed an overall efficiency of 89% when tested on German database (Emo-DB) for 7 emotions. The overall efficiency has proven to increase compared to the studies made up to date on the German Database. The highest emotion recognition rate was for SAD using fused algorithm which was 95.56%. Also results were tabulated and compared using Modified MFCC. A Graphical unit interface of the proposed system is also devised. Application/Improvements: The applications of speech emotion recognition are farfetched. Further scope of this work will be a comparison of the achieved recognition rate using algorithms with recognition rate achieved by humans.

Enhanced Local Ternary Patterns (ELTP) significantly improves performance over other feature descriptor methods including Local Binary Patterns (LBP) and Local Ternary Patterns (LTP). Sequential implementation of ELTP results in poor performance in terms of execution time for real time systems. Speed and accuracy are important characteristics of a real time face recognition system. With the aim of fulfilling both these criteria, this paper presents an implementation of GPU Accelerated Face Recognition System with ELTP using OpenCL framework. As a result of our Optimization techniques, we have achieved highest kernel execution speedup of 374x for ELTP image and histogram generation with 4096 x H 4096 (16MP) image resolution when it is implemented on GPU. Face recognition with ELTP showed higher recognition rates on ORL database. We also implemented LBP and LTP algorithms on GPU and compared their performances with ELTP. Similar Optimization techniques were applied for LBP kernel executions, which resulted in much higher speedups when compared to their previous implementations. Experimental results demonstrated that Parallel implementation with ELTP on GPU (AMD Radeon HD 7650M) outperforms CPU based face recognition system using LBP in terms of speed and accuracy.

Multiple target tracking is an important problem in analysing video data in camera networks. Distributed processing is a promising scheme to deal with huge volume of video data in camera networks. This paper addresses the problem of distributed multiple target tracking in camera networks. Each camera shares measurements with its immediate neighbours and performs inter-camera measurement-to-measurement association in distributed manner. The measurements are assigned to the targets using the nearest neighbourhood principle. To update the target state we use probabilistic data association with sigma point information filters. This filter is integrated with a consensus algorithm to develop distributed multi target tracking algorithm. We evaluated the proposed algorithm on various real world datasets and show that our algorithm outperforms the other related state-of-art distributed algorithms.

Distributed processing is a new paradigm to analyse the huge volume of video data in camera networks. This paper addresses the problem of distributed single target tracking considering false positives and missed detections. Target tracking is modelled as a dynamic state estimation problem with nonlinear process and measurement model. We propose to use the sigma point information filters combined with a consensus algorithm. Sigma point information filters are integrated with probabilistic data association filter to deal with false positives and missed detections. We use a distributed average consensus algorithm which converges in finite time. Unlike other related state of the art technique papers, we report results on real data and show the effectiveness of the proposed algorithm.

This paper presents a real-time face recognition system using kinect sensor. The algorithm is implemented on GPU using opencl and significant speed improvements are observed. We use kinect depth image to increase the robustness and reduce computational cost of conventional LBP based face recognition. The main objective of this paper was to perform robust, high speed fusion based face recognition and tracking. The algorithm is mainly composed of three steps. First step is to detect all faces in the video using viola jones algorithm. The second step is online database generation using a tracking window on the face. A modified LBP feature vector is calculated using fusion information from depth and greyscale image on GPU. This feature vector is used to train a svm classifier. Third step involves recognition of multiple faces based on our modified feature vector.

This paper presents a novel approach for real time face detection using heterogeneous computing. The algorithm uses local binary pattern (LBP) as feature vector for face detection. OpenCL is used to accelerate the code using GPU[1]. Illuminance invariance is achieved using gamma correction and Difference of Gaussian(DOG) to make the algorithm robust against varying lighting conditions. This implementation is compared with previous parallel implementation and is found to perform faster.

Computers have become an essential part of our lives. Many advances have occurred in these machines over past decade in terms of new faster processors, energy efficiency, HD monitors etc. but the basic input devices such as mouse and keyboard remained the same. This project aims at giving a better mode of human computer interaction by allowing computers to respond to human hand gestures directly. The device aims at using the same interface that a mouse uses for interaction with a computer and thus it can be used in any computer without much modification to its software environment.

In this paper, we implement a novel parallelized approach of Local Binary Pattern (LBP) based face recognition algorithm on GPU. High performance rates have been achieved through maximizing the resource exploitation available in the GPU. The launch of GPU programming tools like Open source Computation Language (OpenCL) and (CUDA) have boosted the development of various applications on GPU. In this paper we implement a parallelized LBP algorithm on GPU using OpenCL programming tools. Programs developed under the OpenCL enable us to utilize GPU for general purpose computation with increased performance efficiency in terms of execution time. The experimental results based on the implementation on AMD 6500 GPU processor are observed to increase the computational performance of the system by to 30 folds in case of 1024×1024 images. The relative computational efficiency increases with increase in the size of the Image. This paper addresses several parallelization problems related to memory access and updating, divergent execution paths, understanding and realizing the OpenCL's concurrency and Execution models.

With the advent of Internet, video over IP is gaining popularity. In such an enviroment, scalability and fault tolerance will be the key issues. Existing video on demand (VoD) service systems [1], [2], [3] are usually neither scalable nor tolerant to server faults and hence fail to comply to multi-user, failure-prone networks such as the Internet. Current research areas concerning VoD often focus on increasing the throughput and reliability of single server, but rarely addresses the smooth provision of service during server as well as network failures. Reliable Server Pooling (RSerPool), being capable of providing high availability by using multiple redundant servers as single source point, can be a solution to overcome the above failures. During a possible server failure, the continuity of service is retained by another server. In order to achieve transparent failover, efficient state sharing is an important requirement. In this paper, we present an elegant, simple, efficient and scalable approach which has been developed to facilitate the transfer of state by the client itself, using extended cookie mechanism, which ensures that there is no noticeable change in disruption or the video quality.