Search

This data paper presents a unique multimodal dataset collected from a comprehensive experiment using a wheelchair training simulator. The dataset consists of quantitative and qualitative data that represents the user's experience and performance. Participants engaged in a series of navigational tasks in a simulated environment under two distinct system configuration conditions: a. a conventional monitor display and b. a virtual reality (VR) headset. The monitor group has a total of 24 participants data while using the simulator with a standard display and then other two groups of 18 and 16 respectively using the VR headset with a different wheelchair's speed profile. It was collected data from total of 58 participants.The dataset includes physiological data - Heart Rate Variability (HRV), Electrodermal Activity (EDA), Acceleration (ACC), Skin Temperature, Heart Rate (HR), and Blood Volume Pulse (BVP) - collected during both experiments. Additionally, for the standard display condition, more detailed data comprising Electroencephalography (EEG) and eye-tracking metrics were recorded to provide insights into cognitive load and visual attention patterns.System metrics captured from the simulator provide an objective performance report, including task completion times, error rates (collision of the virtual wheelchair), number of joystick commands. Also, the navigation efficiency data is complemented by post-experiment questionnaires, which gathered subjective responses on user experience, perceived difficulty, the user immersive levels, arousal, and simulator sickness symptoms.This dataset is valuable for researchers and practitioners in the fields of assistive technology, human-computer interaction, and rehabilitation. It offers metrics to a comprehensive view of how different display technologies influence the user experience in wheelchair simulation training. The data allows for in-depth analysis of physiological responses, cognitive engagement, and subjective perceptions, providing a foundation for future research on effective wheelchair training methodologies and the potential benefits of VR in rehabilitation settings.

The probabilistic Bayesian neural network(BNN) is good at providing trustworthy outcomes that is important, e.g. in intrusion detection. Due to the complex of probabilistic BNN, it is looks like a “black box”. The explanation of its prediction is needed for improving its transparency. However, there is no explanatory method to explain the prediction of probabilistic BNN for the reason of uncertainty. For enhance the explainability of BNN model concerning uncertainty quantification, this paper proposes a Bayesian explanatory model that accounts for uncertainties inherent in Bayesian Autoencoder, encompassing both aleatory and epistemic uncertainties. Through global and local explanations, this Bayesian explanatory model is applied to intrusion detection scenarios. Fidelity and sensitivity analyses showcase that the proposed Bayesian explanatory model, which incorporates external uncertainty, effectively identifies key features and provides robust explanations.

Recent breakthroughs in hierarchical multi-agent deep reinforcement learning (HMADRL) are propelling the development of sophisticated multi-robot systems, particularly in the realm of complex coordination tasks. These advancements hold significant potential for addressing the intricate challenges inherent in fast-evolving sectors such as intelligent manufacturing. In this study, we introduce an innovative simulator tailored for a multi-robot pick-and-place (PnP) operation, built upon the OpenAI Gym framework. Our aim is to demonstrate the efficacy of HMADRL algorithms for multi robot coordination in a manufacturing setting, concentrating on their influence on the gripping rate, a crucial indicator for gauging system performance and operational efficiency.

A digital twin (DT) is a virtual/digital model of any physical object (physical twin), interconnected through data exchange. In the context of Industry 4.0, DTs are integral to intelligent automation driving innovation at scale by providing significant improvements in precision, flexibility, and real-time responsiveness. A critical challenge in developing DTs is achieving a model that reflects real-time conditions with precision and flexibility. This paper focuses on evaluating latency and accuracy, key metrics for assessing the efficacy of a DT, which often hinder scalability and adaptability in robotic applications. This article presents a comprehensive framework for developing DTs using Unity and Robot Operating System (ROS) as the main layers of digitalization and communication. The MoveIt package was used for motion planning and execution for the robotic arm, showcasing the framework’s versatility independent of proprietary constraints. Leveraging the versatility and open-source nature of these tools, the framework ensures interoperability, adaptability, and scalability, crucial for modern smart manufacturing applications. Our approach was validated by conducting extensive accuracy and latency tests. We measured latency by timestamping messages exchanged between the physical and digital twin, achieving a latency of 77.67 ms. Accuracy was assessed by comparing the joint positions of the DT and the physical robotic arm over multiple cycles, resulting in an accuracy rate of 99.99%. The results highlight the potential of DTs in enhancing operational efficiency and decision-making in manufacturing environments.

This research study represents a polydexterous deep reinforcement learning-based pick-and-place framework for industrial clutter scenarios. In the proposed framework, the agent tends to learn the pick-and-place of regularly and irregularly shaped objects in clutter by using the sequential combination of prehensile and non-prehensile robotic manipulations involving different robotic grippers in a completely self-supervised manner. The problem was tackled as a reinforcement learning problem; after the Markov decision process (MDP) was designed, the off-policy model-free Q-learning algorithm was deployed using deep Q-networks as a Q-function approximator. Four distinct robotic manipulations, i.e., grasp from the prehensile manipulation category and inward slide, outward slide, and suction grip from the non-prehensile manipulation category were considered as actions. The Q-function comprised four fully convolutional networks (FCN) corresponding to each action based on memory-efficient DenseNet-121 variants outputting pixel-wise maps of action-values jointly trained via the pixel-wise parametrization technique. Rewards were awarded according to the status of the action performed, and backpropagation was conducted accordingly for the FCN generating the maximum Q-value. The results showed that the agent learned the sequential combination of the polydexterous prehensile and non-prehensile manipulations, where the non-prehensile manipulations increased the possibility of prehensile manipulations. We achieved promising results in comparison to the baselines, differently designed variants, and density-based testing clutter.

Closed-circuit television (CCTV) systems have become pivotal tools in modern urban surveillance and traffic management, contributing significantly to road safety and security. This paper introduces an effective solution that capitalizes on CCTV video analytics and an event-driven framework to provide real-time updates on road traffic events, enhancing road safety. Furthermore, this system minimizes the storage requirements for visual data while retaining crucial details related to road traffic events. To achieve this, a two-step approach is employed: (1) training a Deep Convolutional Neural Network (DCNN) model using synthetic data for the classification of road traffic (accident) events and (2) generating video summaries for the classified events. Privacy laws make it challenging to obtain extensive real-world traffic data from open-source datasets, and this challenge is addressed by creating a customised synthetic visual dataset for training. The evaluation of the synthetically trained DCNN model is conducted on ten real-time videos under varying environmental conditions, yielding an average accuracy of 82.3% for accident classification (ranging from 56.7% to 100%). The test video related to the night scene had the lowest accuracy at 56.7% because there was a lack of synthetic data for night scenes. Furthermore, five experimental videos were summarized through the proposed system, resulting in a notable 23.1% reduction in the duration of the original full-length videos. Overall, this proposed system holds significant promise for event-based training of intelligent vehicles in Intelligent Transport Systems (ITS), facilitating rapid responses to road traffic incidents and the development of advanced context-aware systems.

Deep learning techniques have been widely adopted for cyber defence applications such as malware detection and anomaly detection. The ever-changing nature of cyber threats has made cyber defence a constantly evolving field. Smart manufacturing is critical to the broader thrust towards Industry 4.0 and 5.0. Developing advanced technologies in smart manufacturing requires enabling a paradigm shift in manufacturing, while cyber-attacks significantly threaten smart manufacturing. For example, a cyber attack (e.g., backdoor) occurs during the model’s training process. Cyber attack affects the models and impacts the resultant output to be misled. Therefore, this paper proposes a novel and comprehensive framework for smart cyber defence in deep learning security. The framework collectively incorporates a threat model, data, and model security. The proposed framework encompasses multiple layers, including privacy and protection of data and models. In addition to statistical and intelligent model techniques for maintaining data privacy and confidentiality, the proposed framework covers the structural perspective, i.e., policies and procedures for securing data. The study then offers different methods to make the models robust against attacks coupled with a threat model. Along with the model security, the threat model helps defend the smart systems against attacks by identifying potential or actual vulnerabilities and putting countermeasures and control in place. Moreover, based on our analysis, the study provides a taxonomy of the backdoor attacks and defences. In addition, the study provides a qualitative comparison of the existing backdoor attacks and defences. Finally, the study highlights the future directions for backdoor defences and provides a possible way for further research.

The purpose of smart surveillance systems for automatic detection of road traffic accidents is to quickly respond to minimize human and financial losses in smart cities. However, along with the self-evident benefits of surveillance applications, privacy protection remains crucial under any circumstances. Hence, to ensure the privacy of sensitive data, European General Data Protection Regulation (EU-GDPR) has come into force. EU-GDPR suggests data minimisation and data protection by design for data collection and storage. Therefore, for a privacy-aware surveillance system, this paper targets the identification of two areas of concern: (1) detection of road traffic events (accidents), and (2) privacy preserved video summarization for the detected events in the surveillance videos. The focus of this research is to categorise the traffic events for summarization of the video content, therefore, a state-of-the-art object detection algorithm, i.e., You Only Look Once (YOLOv5), has been employed. YOLOv5 is trained using a customised synthetic dataset of 600 annotated accident and non-accident video frames. Privacy preservation is achieved in two steps, firstly, a synthetic dataset is used for training and validation purposes, while, testing is performed on real-time data with an accuracy from 55% to 85%. Secondly, the real-time summarized videos (reduced video duration to 42.97% on average) are extracted and stored in an encrypted format to avoid un-trusted access to sensitive event-based data. Fernet, a symmetric encryption algorithm is applied to the summarized videos along with Diffie–Hellman (DH) key exchange algorithm and SHA256 hash algorithm. The encryption key is deleted immediately after the encryption process, and the decryption key is generated at the system of authorised stakeholders, which prevents the key from a man-in-the-middle (MITM) attack.

Surveillance video recording is a powerful method of deterring unlawful activities. A robust data protection-by-design solution can be helpful in terms of making a captured video immutable, as such recordings cannot become a piece of evidence until proven to be unaltered. Similarly, video sharing from closed-circuit television video recording or in social media interaction requires self-authentication for responsible and reliable data sharing. This article presents a computationally inexpensive method of preserving a chain-of-evidence in surveillance videos by means of hashing and steganography. The method conforms to the data protection regulations, which are increasingly adopted by governments, and is applicable to network edge storage. Encryption keys are stored in a hardware wallet independently of the video capture device itself, while evidential information is stored steganographically within video frames themselves, independently of the content. Added protection is provided by hiding information within the two least-valued of pixel bitplanes, using a newly introduced technique that randomizes the pixel storage locations on a per video frame and video-capture device basis. Overall, the proposed method has turned out to not only preserve the integrity of stored video data but also results in minimal degradation of the video data resulting from steganography. Despite the inclusion of hidden information, video frames will still be available for common image-processing tasks such as tracking and classification, as their objective video quality is almost unchanged.

Video Surveillance (VS) systems are commonly deployed for real-time abnormal event detection and autonomous video analytics. Video captured by surveillance cameras in real-time often contains identifiable personal information, which must be privacy protected, sometimes along with the locations of the surveillance and other sensitive information. Within the Surveillance System, these videos are processed and stored on a variety of devices. The processing and storage heterogeneity of those devices, together with their network requirements, make real-time surveillance systems complex and challenging. This paper proposes a surveillance system, named as Multi-Level Video Security (MuLViS) for privacy-protected cameras. Firstly, a Smart Surveillance Security Ontology (SSSO) is integrated within the MuLViS, with the aim of autonomously selecting the privacy level matching the operating device's hardware specifications and network capabilities. Overall, along with its device-specific security, the system leads to relatively fast indexing and retrieval of surveillance video. Secondly, information within the videos are protected at the times of capturing, streaming, and storage by means of differing encryption levels. An extensive evaluation of the system, through visual inspection and statistical analysis of experimental video results, such as by the Encryption Space Ratio (ESR), has demonstrated the aptness of the security level assignments. The system is suitable for surveillance footage protection, which can be made General Data Protection Regulation (GDPR) compliant, ensuring that lawful data access respects individuals' privacy rights.

In this study, we develop a framework for an intelligent and self-supervised industrial pick-and-place operation for cluttered environments. Our target is to have the agent learn to perform prehensile and non-prehensile robotic manipulations to improve the efficiency and throughput of the pick-and-place task. To achieve this target, we specify the problem as a Markov decision process (MDP) and deploy a deep reinforcement learning (RL) temporal difference model-free algorithm known as the deep Q-network (DQN). We consider three actions in our MDP; one is ‘grasping’ from the prehensile manipulation category and the other two are ‘left-slide’ and ‘right-slide’ from the non-prehensile manipulation category. Our DQN is composed of three fully convolutional networks (FCN) based on the memory-efficient architecture of DenseNet-121 which are trained together without causing any bottleneck situations. Each FCN corresponds to each discrete action and outputs a pixel-wise map of affordances for the relevant action. Rewards are allocated after every forward pass and backpropagation is carried out for weight tuning in the corresponding FCN. In this manner, non-prehensile manipulations are learnt which can, in turn, lead to possible successful prehensile manipulations in the near future and vice versa, thus increasing the efficiency and throughput of the pick-and-place task. The Results section shows performance comparisons of our approach to a baseline deep learning approach and a ResNet architecture-based approach, along with very promising test results at varying clutter densities across a range of complex scenario test cases.

From an individual's perspective, technological advancement has merits and demerits. Video captured by surveillance cameras while a person goes about their daily life may improve their personal safety but the images collected may also represent an invasion of their privacy. Because of the ease of digital information sharing, there exists a need to protect that visual information from illegal utilization by untrusted parties. The European parliament has ratified the General Data Protection Regulation (GDPR), which has been effective since May 2018 with a view to ensuring the privacy of European Union (EU) citizens' and visitors' personal data. The regulation has introduced data safeguards through Pseudonymisation, Encryption, and Data protection-by-design. However, the regulation does not assist with technical and implementation procedures, such as video redaction, to establish those safeguards. This paper refers to the GDPR term “personal data” as “visual personal data” and aims to discuss regulatory safeguards of visual privacy, such as reversible protection, from the technological point-of-view. In the context of GDPR, the roles of machine learning (i.e. within computer vision), image processing, cryptography, and blockchain are explored as a way of deploying Data Protection-by-Design solutions for visual surveillance data. The paper surveys the existing market-based data protection solutions and provides suggestions for the development of GDPR-compliant Data Protection-by-Design video surveillance systems. The paper is also relevant for those entities interacting with EU citizens from outside the EU and for those regions not currently covered by such regulation that may soon implement similar provisions.

One of the most promising technologies that is driving digitalization in several industries is Digital Twin (DT). DT refers to the digital replica or model of any physical object (physical twin). What differentiates DT from simulation and other digital or CAD models is the automatic bidirectional exchange of data between digital and physical twins in real-time. The benefits of implementing DT in any sector include reduced operational costs and time, increased productivity, better decision making, improved predictive/preventive maintenance, etc. As a result, its implementation is expected to grow exponentially in the coming decades as, with the advent of Industry 4.0, products and systems have become more intelligent, relaying on collection and storing incremental amounts of data. Connecting that data effectively to DTs can open up many new opportunities and this paper explores different industrial sectors where the implementation of DT is taking advantage of these opportunities and how these opportunities are taking the industry forward. The paper covers the applications of DT in 13 different industries including the manufacturing, agriculture, education, construction, medicine, and retail, along with the industrial use case in these industries.

Digital Twin (DT) refers to the virtual copy or model of any physical entity (physical twin) both of which are interconnected via exchange of data in real time. Conceptually, a DT mimics the state of its physical twin in real time and vice versa. Application of DT includes real-time monitoring, designing/planning, optimization, maintenance, remote access, etc. Its implementation is expected to grow exponentially in the coming decades. The advent of Industry 4.0 has brought complex industrial systems that are more autonomous, smart, and highly interconnected. These systems generate considerable amounts of data useful for several applications such as improving performance, predictive maintenance, training, etc. A sudden influx in the number of publications related to ‘Digital Twin’ has led to confusion between different terminologies related to the digitalization of industries. Another problem that has arisen due to the growing popularity of DT is a lack of consensus on the description of DT as well as so many different types of DT, which adds to the confusion. This paper intends to consolidate the different types of DT and different definitions of DT throughout the literature for easy identification of DT from the rest of the complimentary terms such as ‘product avatar’, ‘digital thread’, ‘digital model’, and ‘digital shadow’. The paper looks at the concept of DT since its inception to its predicted future to realize the value it can bring to certain sectors. Understanding the characteristics and types of DT while weighing its pros and cons is essential for any researcher, business, or sector before investing in the technology.