Search

The efficiency of human-computer interaction is greatly hindered by the small size of the touchscreens on mobile devices, such as smart phones and watches. This has prompted widespread interest in handwriting recognition systems, which can be divided into active and passive systems. Active systems require additional hardware devices to perceive movements of handwriting or the tracking accuracy is not adequate for hand-writing recognition. Passive methods use the acoustic signal of pen rubbing and are susceptible to environmental noise (above 60dB). This paper presents a novel handwriting recognition system based on vibration signals detected by the built-in accelerometer of smart phones. VibWriter is highly resistant to interference since the normal environmental noise will not cause the vibration of the accelerometer. Extensive experiments demonstrated the efficacy of the system in terms of accuracy in letter recognition (76.15%) and word recognition (88.14%) when dealing with words of various lengths written by various users in a variety of writing positions under a variety of environmental conditions.

Various characteristics of mobile applications (apps) and associated in-app services have been used reveal potentially-sensitive user information; however, privacy concerns have prompted third-party apps to rigorously restrict access to data related to mobile app usage. This paper outlines a novel approach to the extraction of detailed app usage information based on analysis of the electromagnetic (EM) signals emitted from mobile devices when executing app-related tasks. Note that this type of EM leakage becomes high-complex when multiple apps are used simultaneously and is subject to interference from geomagnetic signals generated by device movement. This paper proposes a deep learning-based multi-label classification system to identify apps and in-app services based on magnetometer readings. The proposed MAGTHIEF system uses accelerometer and gyroscope data to cancel out the offset in geomagnetic signals followed by an elaborate deep region convolution neural network (DRCNN) to differentiate among multiple apps and the corresponding inapp services. Experiments on 50 apps demonstrated the efficacy of MAGTHIEF in identifying multiple apps and in-app services, achieving high average macro F1 scores of 0.87 and 0.95, respectively. MAGTHIEF also achieved time duration accuracy of 89.5% in recognizing app trajectory in the real-world scene.

Quick response (QR) codes have been widely used in mobile applications due to its convenience and the pervasive built-in cameras on smartphones. Recently, however, attacks against QR codes have been reported that attackers can capture a QR code of the victim and replay it to achieve a fraudulent transaction or intercept private information, just before the original QR code is scanned. In this study, we enhance the security of a QR code by identifying its authenticity. We propose SCREENID, which embeds a QR code with information of the screen which displays it, thereby the QR code can reveal whether it is reproduced by an adversary or not. In SCREENID, PWM frequency of screens is exploited as the unique screen fingerprint. To improve the estimation accuracy of PWM frequency, SCREENID incorporates a model for the interaction between the camera and screen in the temporal and spatial domains. Extensive experiments demonstrate that SCREENID can differentiate screens of different models, types, and manufacturers, thus improve the security of QR codes.

Understanding the nature of user-device interactions (e.g., who is using the device and what he/she is doing with it) is critical to many applications including time management, user profiles, and privacy protection. However, in scenarios where mobile devices are shared among family members or multiple employees in a company, conventional account-based statistics are not meaningful. This poses an even bigger problem when dealing with sensitive data. Moreover, fingerprint readers and front-facing cameras were not designed to continuously identify users. In this study, we developed MagPrint, a novel approach to fingerprint users based on unique patterns in the electromagnetic (EM) signals associated with the specific use patterns of users. Initial experiments showed that time-varying EM patterns are unique to individual users. They are also temporally and spatially consistent, which makes them suitable for fingerprinting. MagPrint has a number of advantages over existing schemes: i) Non-intrusive fingerprinting, ii) implementation using a small and easy-to-deploy device, and iii) high accuracy thanks to the proposed classification algorithm. In experiments involving 30 users, MagPrint achieves 94.3% accuracy in classifying users from these traces, which represents an 10.9% improvement over the state-of-the-art classification method.