Search

Compared with traditional methods, the action recognition model based on 3D convolutional deep neural network captures spatio-temporal features more accurately, resulting in higher accuracy. However, the large number of parameters and computational requirements of 3D models make it difficult to deploy on mobile devices with limited computing power. In order to achieve an efficient video action recognition model, we have analyzed and compared classic lightweight network principles and proposed the 3D-ShuffleViT network. By deeply integrating the self-attention mechanism with convolution, we have introduced an efficient ACISA module that further enhances the performance of our proposed model. This has resulted in exceptional performance in both context-sensitive and context-independent action recognition, while reducing deployment costs. It is worth noting that our 3D-ShuffleViT network, with a computational cost of only 6% of that of SlowFast-ResNet101, achieved 98% of the latter's Top1 accuracy on the EgoGesture dataset. Furthermore, on the same CPU (Intel i5-8300H), its speed was 2.5 times that of the latter. In addition, when we deployed our model on edge devices, our proposed network achieved the best balance between accuracy and speed among lightweight networks of the same order. [ABSTRACT FROM AUTHOR]

Abstract Pruning is showing huge potential for compressing and accelerating deep neural networks by eliminating redundant parameters. Along with more terminal chips integrated with AI accelerators for internet of things (IoT) devices, structured pruning is gaining popularity with the edge computing research area. Different from filter pruning and group-wise pruning, stripe-wise pruning (SWP) conducts pruning at the level of stripes in each filter. By introducing filter skeleton (FS) to each stripe, the existing SWP method sets an absolute threshold for the values in FS and removes the stripes whose corresponding values in FS could not meet the threshold. Starting with investigation into the process of stripe wise convolution, we use the statistical properties of the weights located on each stripe to learn the importance between those stripes in a filter and remove stripes with low importance. Our pruned VGG-16 achieves the existing results by a fourfold reduction in parameter with only 0.4% decrease in accuracy. Results from comprehensive experiments on IoT devices are also presented.

Detecting malware is a crucial defense mechanism against potential cyber-attacks. However, current methods illustrate significant limitations in achieving high performance while maintaining faster inference on edge devices. This study proposes a novel deep network with dual-attention feature refinement on a two-branch deep network to learn real-time malware detection on edge platforms. The proposed method introduces lightweight spatial-asymmetric attention for refining the extracted features of its backbone and multi-head attention to correlate learned features from the network branches. The experimental results show that the proposed method can significantly outperform existing methods in quantitative evaluation. In addition, this study also illustrates the practicability of a lightweight deep network on edge devices by optimizing and deploying the model directly on the actual edge hardware. The proposed optimization strategy achieves a frame rate of over 545 per second on low-power edge devices.

Audio classification using deep learning models, which is essential for applications like voice assistants and music analysis, faces challenges when deployed on edge devices due to their limited computational resources and memory. Achieving a balance between performance, efficiency, and accuracy is a significant obstacle to optimizing these models for such constrained environments. In this investigation, we evaluate diverse deep learning architectures, including Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM), for audio classification tasks on the ESC 50, UrbanSound8k, and Audio Set datasets. Our empirical findings indicate that Mel spectrograms outperform raw audio data, attributing this enhancement to their synergistic alignment with advanced image classification algorithms and their congruence with human auditory perception. To address the constraints of model size, we apply model-compression techniques, notably magnitude pruning, Taylor pruning, and 8-bit quantization. The research demonstrates that a hybrid pruned model achieves a commendable accuracy rate of 89 percent, which, although marginally lower than the 92 percent accuracy of the uncompressed CNN, strikingly illustrates an equilibrium between efficiency and performance. Subsequently, we deploy the optimized model on the Raspberry Pi 4 and NVIDIA Jetson Nano platforms for audio classification tasks. These findings highlight the significant potential of model-compression strategies in enabling effective deep learning applications on resource-limited devices, with minimal compromise on accuracy.

Amidst an energy crisis stemming from increased energy costs and the looming threat of war, there has been a burgeoning interest in energy conservation and management worldwide. Industrial complexes constitute a significant portion of total energy consumption. Hence, reducing energy consumption in these complexes is imperative for energy preservation. Typically, factories within similar industries aggregate in industrial complexes and share similar energy utilities. However, they often fail to capitalize on this shared infrastructure efficiently. To address this issue, a network system employing a virtual utility plant has been proposed. This system enables proactive measures to counteract energy surplus or deficit through AI-based predictions, thereby maximizing energy efficiency. Nevertheless, deploying conventional server systems within factories poses considerable challenges. Therefore, leveraging edge devices, characterized by low power consumption, high efficiency, and minimal space requirements, proves highly advantageous. Consequently, this study focuses on constructing and employing data collection and AI systems to utilize edge devices as standalone systems in each factory. To optimize the AI system for low-performance edge devices, we employed the integration-learning AI modeling technique. Evaluation results demonstrate that the proposed system exhibits high stability and reliability.

Ensuring personal safety and preventing accidents are critical aspects of power construction safety supervision. However, current monitoring methods are inefficient and unreliable as most of them rely on manual monitoring and transmission, which results in slow detection and delayed warnings regarding violations. To overcome these challenges, we propose an intelligent detection system that can accurately identify instances of illegal wearing of power construction workers in real-time. Firstly, we integrated the squeeze-and-excitation (SE) module into our convolutional neural network to enhance detection accuracy. This module effectively prioritizes informative features while suppressing less relevant ones, resulting in improved overall performance. Secondly, we present an embedded real-time detection system that utilizes Jetson Xavier NX and Edge-YOLO. This system promptly detects and alerts power construction workers of instances of illegal wearing behavior. To ensure a lightweight implementation, we design appropriate detection heads based on target size and distribution, reducing model parameters while enhancing detection speed and minimizing accuracy loss. Additionally, we employed data augmentation to enhance the system's robustness. Our experimental results demonstrate that our improved Edge-YOLO model achieves high detection precision and recall rates of 0.964 and 0.966, respectively, with a frame rate of 35.36 frames per second when deployed on Jetson Xavier NX. Therefore, Edge-YOLO proves to be an ideal choice for intelligent real-time detection systems, providing superior accuracy and speed performance compared to the original YOLOv5s model and other models in the YOLO series for safety monitoring at construction sites. [ABSTRACT FROM AUTHOR]

The illegal dumping of aluminum and plastic into cities and marine areas leads to negative impacts on the ecosystem and contributes to increased environmental pollution. Although volunteer trash pickup activities have increased in recent years, they require significant effort, time, and money. Therefore, we propose automated trash pickup robot, which incorporates autonomous movement and trash pickup arms. Although these functions have been actively developed, relatively little research has focused on trash detection. As such, we have developed a trash detection function by using deep learning models to improve the accuracy. First, we created a new trash dataset that classifies four types of trash with high illegal dumping volumes (cans, plastic bottles, cardboard, and cigarette butts). Next, we developed a new you only look once (YOLO)-based model with low parameters and computations. We trained the model on a created dataset and a dataset consisting of marine trash created during previous research. In consequence, the proposed models achieve the same detection accuracy as the existing models on both datasets, with fewer parameters and computations. Furthermore, the proposed models accelerate the edge device's frame rate. [ABSTRACT FROM AUTHOR]

Offloading tasks from edge devices to the cloud is an important method to enhance the performance of the edge device. With the help of EH (Energy Harvesting) technology, the edge device can use the collected green energy to support its operations. Most offloading scheduling methods use as much green energy as the edge device collected. Unlike prior research, we consider the long-term benefits of energy. In this paper, we forecast the solar energy supply with meteorological methods which are based on the weather forecast data. Then, we use quadratic programming to allocate energy based on the forecast energy to maximize energy efficiency. Finally, we use NSGA (Non-dominated Sorting Genetic Algorithms) to offload tasks in the edge device. Simulations show that our proposed method not only minimizes the execution time and the energy consumption of clouds, but also enhances the QoE of users. • We model the edge system considering the battery and the weather information. • Energy allocation for edge devices based on the weather forecast information. • A offloading task method is proposed based on NGSA III. • Simulations are given with benefit function. • Comparisons are given considering multiple aspects. [ABSTRACT FROM AUTHOR]

Deep Neural Network based wake word (such as Hi Alexa or Hey Siri) systems allow increasingly accurate speech communication between humans and machines. However, this setup requires high processing power or cloud services which may not be accessible by edge devices. Currently, the accuracy of machine learning methods for cloudless edge devices in voice activation hovers below 90%. This paper explores wake word implementation on edge devices using a 2-Dimensional Convolutional Neural Network (CNN) with improved and balanced accuracy and latency. The proposed CNN model is created, trained and quantized using TensorFlow on a PC and exported to a Raspberry Pi Zero 2 W. The quantization method reduces the model size by 20% and spectral gating is adopted to lower wake word inaccuracy detection in moderately noisy environment. The proposed system achieved more than 90% wake word detection accuracy across 30 to 50 dB background noise with an average of 1.03 second of response time for the intended user. The result shows low-powered edge device still offers competitive performance for detecting wake word without cloud services. [ABSTRACT FROM AUTHOR]

Nowadays, the Internet of Things (IoT) has become widely used at various places and for various applications. To facilitate this trend, we have developed the IoT application server platform called SEMAR (Smart Environmental Monitoring and Analytical in Real-Time), which offers standard features for collecting, displaying, and analyzing sensor data. An edge device is usually installed to connect sensors with the server, where the interface configuration, the data processing, the communication protocol, and the transmission interval need to be defined by the user. In this paper, we proposed an edge device framework for SEMAR to remotely optimize the edge device utilization with three phases. In the initialization phase, it automatically downloads the configuration file to the device through HTTP communications. In the service phase, it converts data from various sensors into the standard data format and sends it to the server periodically. In the update phase, it remotely updates the configuration through MQTT communications. For evaluations, we applied the proposal to the fingerprint-based indoor localization system (FILS15.4) and the data logging system. The results confirm the effectiveness in utilizing SEMAR to develop IoT application systems. [ABSTRACT FROM AUTHOR]

Stereo matching is an important component technology that constitutes the 3D perception capability of autonomous vehicles. On resource-constrained edge devices, it is very important to compute in real-time with very low time. However, most stereo matching networks focus on generating disparity maps on high-end GPUs, which do not meet the real-time requirements on edge devices. To solve this problem, a new stereo matching network is proposed in this paper to achieve real-time stereo matching on edge devices. The proposed network greatly improves the inference speed by constructing a low-resolution feature extractor, and by using multi-stage residual methods for stereo matching. In particular, we propose a method that combines the group-wise L1 distance & the group-wise correlation cost volume and an effective attention-based 3D cost aggregation method. Our network achieves a good balance between speed and accuracy on the KITTI 2012 and KITTI 2015 datasets. The proposed network achieves 2.77% and 3.44% accuracy (D1-all) on KITTI 2012 and KITTI 2015, respectively. With TensorRT, the proposed network achieves 31.8 FPS and outperforms the real-time results of most state-of-the-art networks on NVIDIA Jetson Nano edge devices.

Wheat is the most important and dominating crop in Pakistan in terms of production and acreage, which is grown on 37% of the cultivated area, accounting for 70% of the total production. However, wheat yield is highly affected by stripe rust, which is considered the most devastating fungal disease, causing 5.5 million tonnes of loss per year globally. In order to minimize this loss, the accurate and timely detection of rust disease is crucial instead of manual inspection. Towards this end, we propose a system to detect wheat rust disease and classify its infection types into four classes, including healthy, resistant, moderate (moderately resistant to moderately susceptible), and susceptible. The wheat rust dataset is collected indigenously from the National Agricultural Research Centre, Islamabad. A pre-trained U2 Net model is used to remove the background and extract the leaf containing the rust disease. Subsequently, two deep learning classifiers, including the Xception model and ResNet-50 are applied to classify the stripe rust severity levels, where the ResNet-50 model outperformed with the highest accuracy of 96%. This research presents a comparison between two state-of-the-art deep learning classifiers in terms of accuracy, memory utilization, and prediction time, which will assist the research community in selecting the most appropriate model for plant disease detection. Moreover, to assess the external validity, the performance of these classifiers is compared with the existing technique using a publicly available dataset, which confirms the validity of the results. Additionally, an intelligent edge computing rust detection device has been developed, where the trained ResNet-50 model is deployed, which facilitates the farmers to monitor the rust attack. The proposed research is aimed to assist the agricultural community to employ preventive measures in a site-specific manner based on the accurate diagnosis of rust disease & its severity, which is intended to improve the quality of the wheat as well as production.

Pruning is showing huge potential for compressing and accelerating deep neural networks by eliminating redundant parameters. Along with more terminal chips integrated with AI accelerators for internet of things (IoT) devices, structured pruning is gaining popularity with the edge computing research area. Different from filter pruning and group-wise pruning, stripe-wise pruning (SWP) conducts pruning at the level of stripes in each filter. By introducing filter skeleton (FS) to each stripe, the existing SWP method sets an absolute threshold for the values in FS and removes the stripes whose corresponding values in FS could not meet the threshold. Starting with investigation into the process of stripe wise convolution, we use the statistical properties of the weights located on each stripe to learn the importance between those stripes in a filter and remove stripes with low importance. Our pruned VGG-16 achieves the existing results by a fourfold reduction in parameter with only 0.4% decrease in accuracy. Results from comprehensive experiments on IoT devices are also presented. [ABSTRACT FROM AUTHOR]

Competitiveness in industry requires smooth, efficient, and high-quality operation. For some industrial applications or process control and monitoring applications, it is necessary to achieve high availability and reliability because, for example, the failure of availability in industrial production can have serious consequences for the operation and profitability of the company, as well as for the safety of employees and the surrounding environment. At present, many new technologies that use data obtained from various sensors for evaluation or decision-making require the minimization of data processing latency to meet the needs of real-time applications. Cloud/Fog and Edge computing technologies have been proposed to overcome latency issues and to increase computing power. However, industrial applications also require the high availability and reliability of devices and systems. The potential malfunction of Edge devices can cause a failure of applications, and the unavailability of Edge computing results can have a significant impact on manufacturing processes. Therefore, our article deals with the creation and validation of an enhanced Edge device model, which in contrast to the current solutions, is aimed not only at the integration of various sensors within manufacturing solutions, but also brings the required redundancy to enable the high availability of Edge devices. In the model, we use Edge computing, which performs the recording of sensed data from various types of sensors, synchronizes them, and makes them available for decision making by applications in the Cloud. We focus on creating a suitable Edge device model that works with the redundancy, by using either mirroring or duplexing via a secondary Edge device. This enables high Edge device availability and rapid system recovery in the event of a failure of the primary Edge device. The created model of high availability is based on the mirroring and duplexing of the Edge devices, which support two protocols: OPC UA and MQTT. The models were implemented in the Node-Red software, tested, and subsequently validated and compared to confirm the required recovery time and 100% redundancy of the Edge device. In the contrast to the currently available Edge solutions, our proposed extended model based on Edge mirroring is able to address most of the critical cases, where fast recovery is required, and no adjustments are needed for critical applications. The maturity level of Edge high availability can be further extended by applying Edge duplexing for process control. [ABSTRACT FROM AUTHOR]

The ever-increasing number of recording sites of silicon-based probes imposes a great challenge for detecting and evaluating single-unit activities in an accurate and efficient manner. Currently separate solutions are available for high precision offline evaluation and separate solutions for embedded systems where computational resources are more limited. We propose a deep learning-based spike sorting system, that utilizes both unsupervised and supervised paradigms to learn a general feature embedding space and detect neural activity in raw data as well as predict the feature vectors for sorting. The unsupervised component uses contrastive learning to extract features from individual waveforms, while the supervised component is based on the MobileNetV2 architecture. One of the key advantages of our system is that it can be trained on multiple, diverse datasets simultaneously, resulting in greater generalizability than previous deep learning-based models. We demonstrate that the proposed model does not only reaches the accuracy of current state-of-art offline spike sorting methods but has the unique potential to run on edge Tensor Processing Units (TPUs), specialized chips designed for artificial intelligence and edge computing. We compare our model performance with state of art solutions on paired datasets as well as on hybrid recordings as well. The herein demonstrated system paves the way to the integration of deep learning-based spike sorting algorithms into wearable electronic devices, which will be a crucial element of high-end brain–computer interfaces. • Deep learning methods can generalize on spike sorting data. • Contrastive learning delivers high-quality feature space for spike waveforms. • MobileNet architecture detects spikes at a state-of-art performance. • Edge TPU can be used to perform fast spike sorting on devices with limited resources. [ABSTRACT FROM AUTHOR]

The tremendous growth in the internet of things (IoT) technology has provided great support for e-healthcare applications. The remote health monitoring of patients at home is gaining popularity with the increase in design of number of IoT enabled wearable devices. The secure and efficient health data retrieval during continuous uploading of huge data on network is a challenge. In addition, another challenge exists due to limited resources available with IoT device to proceed efficient data retrieval. On the other hand, the edge cloud in edge computing technology provide abundant resources to its devices for various computations, storage and processing. The integration of IoT with edge would solve the resource constraint issue. However, efficient retrieval of health data from large pool of data would still be an issue due to IP based solutions offered by IoT. The Named Data Networking (NDN) is a promising solution for efficient delivery of data based on its name based content searching. This paper proposed NDN communication based secure remote healthcare monitoring framework by integrating IoT with edge computing where security to the each individual content is provided through hashing and encryption mechanism. The performance of the proposed framework is evaluated against its companion in Icarus simulator. The results proved the efficiency of our framework for content retrieval delay and cost associated in accessing the required data.

The rapid development of computer biometrics over the past 2-3 decades is largely due to the development and widespread introduction into clinical practice of new methods of studying human body health, including pulse methods. It is possible to judge changes in hemodynamic characteristics, heart rate, and blood flow rate in the studied part of the body based on the parameters of the pulse wave signal. At the same time, the physical processes of the formation of the pulse wave shape have not been fully studied, although the number of biophysical models of blood circulation is quite significant. The development of such a model will allow us to effectively apply modern developments in digital signal processing to the pulse wave and increase its diagnostic value. A qualitative model of pulse signal can be entrusted to the development of the base unit of the biotechnical system as a type of edge device. The work is devoted to the improvement of methods of rapid diagnosis of the cardiovascular system based on the analysis of model pulse-grams. An adequate mathematical model of the pulse wave, which corresponds to real pulse signals in different states of the human body and contains mathematical relationships between the main parameters of pulse-grams, has been refined. The algorithm of express diagnostics with the established criteria of the analysis of pulsograms is offered.

Taking China's "Chengde Petroleum College" as an example, this paper analyzes the current situation of computer room monitoring in colleges and universities. The Internet of Things technology is used to remotely and automatically monitor the environment of the network computer room. This paper studies and puts forward the method and model of edge calculation in the network computer room monitoring system. The monitoring methods of critical parameters such as thermal environment and energy consumption of computer room are given. The practical application shows that, compared with the historical data in the same period, the energy saving rate of the scheme is as high as 15% ~ 27%, and has good market promotion value. [ABSTRACT FROM AUTHOR]

In order to solve the problem of an accurate recognition of tea picking through tea picking robots, an edge device detection method is proposed in this paper based on ShuffleNetv2-YOLOv5-Lite-E for tea with one bud and two leaves. This replaces the original feature extraction network by removing the Focus layer and using the ShuffleNetv2 algorithm, followed by a channel pruning of YOLOv5 at the neck layer head, thus achieving the purpose of reducing the model size. The results show that the size of the improved generated weight file is 27% of that of the original YOLOv5 model, and the mAP value of ShuffleNetv2-YOLOv5-Lite-E is 97.43% and 94.52% on the pc and edge device respectively, which are 1.32% and 1.75% lower compared to that of the original YOLOv5 model. The detection speeds of ShuffleNetv2-YOLOv5-Lite-E, YOLOv5, YOLOv4, and YOLOv3 were 8.6 fps, 2.7 fps, 3.2 fps, and 3.4 fps respectively after importing the models into an edge device, and the improved YOLOv5 detection speed was 3.2 times faster than that of the original YOLOv5 model. Through the detection method, the size of the original YOLOv5 model is effectively reduced while essentially ensuring recognition accuracy. The detection speed is also significantly improved, which is conducive to the realization of intelligent and accurate picking for future tea gardens, laying a solid foundation for the realization of tea picking robots. [ABSTRACT FROM AUTHOR]

Solar energy is considered the most abundant form of energy available on earth. However, the efficiency of photovoltaic (PV) panels is greatly reduced due to the accumulation of dust particles on the surface of PV panels. The optimization of the cleaning cycles of a PV power plant through condition monitoring of PV panels is crucial for its optimal performance. Specialized equipment and weather stations are deployed for large-scale PV plants to monitor the amount of soil accumulated on panel surface. However, not much focus is given to small- and medium-scale PV plants, where the costs associated with specialized weather stations cannot be justified. To overcome this hurdle, a cost-effective and scalable solution is required. Therefore, a new centralized cloud-based solar conversion recovery system (SCRS) is proposed in this research work. The proposed system utilizes the Internet of Things (IoT) and cloud-based centralized architecture, which allows users to remotely monitor the amount of soiling on PV panels, regardless of the scale. To improve scalability and cost-effectiveness, the proposed system uses low-cost sensors and an artificial neural network (ANN) to reduce the amount of hardware required for a soiling station. Multiple ANN models with different numbers of neurons in hidden layers were tested and compared to determine the most suitable model. The selected ANN model was trained using the data collected from an experimental setup. After training the ANN model, the mean squared error (MSE) value of 0.0117 was achieved. Additionally, the adjusted R-squared ( R 2 ) value of 0.905 was attained on the test data. Furthermore, data is transmitted from soiling station to the cloud server wirelessly using a message queuing telemetry transport (MQTT) lightweight communication protocol over Wi-Fi network. Therefore, SCRS depicts a complete wireless sensor network eliminating the need for extra wiring. The average percentage error in the soiling ratio estimation was found to be 4.33%. [ABSTRACT FROM AUTHOR]

Facial emotion recognition (FER) systems are imperative in recent advanced artificial intelligence (AI) applications to realize better human–computer interactions. Most deep learning-based FER systems have issues with low accuracy and high resource requirements, especially when deployed on edge devices with limited computing resources and memory. To tackle these problems, a lightweight FER system, called Light-FER, is proposed in this paper, which is obtained from the Xception model through model compression. First, pruning is performed during the network training to remove the less important connections within the architecture of Xception. Second, the model is quantized to half-precision format, which could significantly reduce its memory consumption. Third, different deep learning compilers performing several advanced optimization techniques are benchmarked to further accelerate the inference speed of the FER system. Lastly, to experimentally demonstrate the objectives of the proposed system on edge devices, Light-FER is deployed on NVIDIA Jetson Nano. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Harbin University of Science & Technology is the property of Journal of Harbin University of Science & Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

A lot of recent works have shown that deep learning-based visual odometry methods outperform existing feature-based approaches in the monocular case. However, most of them cannot be used in mobile robotics because they require a sufficiently powerful computing device. In this paper, we propose a method with a significant reduction in computing resources and with a slight decrease in accuracy. To achieve that we replaced recurrent block by lightweight transformer-based module. We evaluate the proposed model on the KITTI dataset for calculating accuracy and test a computation cost on NVIDIA Jetson Nano and NVIDIA Jetson AGX Xavier. Our experiments show that the proposed model works faster than the considered original model.

Digital dermatitis (DD) is a bovine claw disease responsible for ulcerative lesions on the coronary band of the foot. It causes significant animal welfare and economic losses to the cattle industry. Early detection of DD can lead to prompt treatment and decrease lameness. Current detection and staging methods require a trained individual to evaluate the interdigital space on each foot for clinical signs of DD. Computer vision (CV), a type of artificial intelligence for image analysis, has demonstrated promising results on object detection tasks. However, farms require robust solutions that can be deployed in harsh conditions including dust, debris, humidity, precipitation, other equipment issues. The study aims to train, deploy, and benchmark DD detection models on edge devices. Images were collected from commercial dairy farms with the camera facing the interdigital space on the plantar surface of the foot. Images were scored for M-stages of DD by a trained investigator using the M-stage DD classification system. Models were trained to detect and score DD lesions and embedded on an edge device. The Tiny YOLOv4 model deployed on a CV specific integrated camera module connected to a single board computer achieved a mean average precision (mAP) of 0.895, an overall prediction accuracy of 0.873, and a Cohen's kappa of 0.830 for agreement between the computer vision model and the trained investigator. The model reached a final inference speed of 40 frames per second (FPS) and ran stably without any interruptions. The CV model was able to detect DD lesions on an edge device with high performance and speed. The CV tool can be used for early detection and prompt treatment of DD in dairy cows. Real-time detection of DD on edge device will improve health outcomes, while simultaneously decreasing labor costs. We demonstrate that the deployed model can be a low-power and portable solution for real-time detection of DD on dairy farms. This result is a step towards applying CV algorithms to veterinary medicine and implementing real-time detection of health outcomes in precision farming. [ABSTRACT FROM AUTHOR]

• Automatic detection and targeting system developed for futuristic cotton weed management. • YOLOv8 performed better with CBAM and C3Ghost modules in the backbone architecture. • Heatmap visualization showed accurate interpretability with consistent patterns. • Bot-SORT tracking algorithm reduced computational load in software and hardware. • Higher intersection over union resulted in correct localization of targeting system. Traditional means of weed removal, such as human work or the use of pesticides, frequently require significant amounts of effort, incur high expenses, and can negatively impact the environment. This study introduces a modified version of the YOLOv8 nano architecture that is suitable for running on edge devices for real-time applications. The proposed model uses an augmented version of the well-known CottonWeedDet12 dataset consisting of a total of 16,944 images with characteristic annotations to develop a model capable of correctly distinguishing 12 different cotton weed classes with an increased mean average precision of 97.6 % that is about 1.2 % more than the model trained using original, unaugmented dataset. The final selected model uses a convolutional block attention module (CBAM) and a unique C3Ghost block within the YOLOv8 backbone, which together increase the model's reliability for more accurate predictions with reduced computational complexity. Upon training with the augmented dataset, the proposed model with only 3.6 million parameters was able to achieve an mAP@50 score of 97.6 %, which surpasses all previous studies conducted using this dataset. Additionally, a high F1 score of 94.4 % proves that the model has a good balance between recall and precision. Class Activation Map (CAM) approaches such as EigenCAM, Grad-CAM++, and LayerCAM explainable AI (XAI) showed promising results for each of the customized models upon testing their interpretability for cotton weed detection. Furthermore, based on this model, a fast and cost-efficient targeting system was developed using a yaw-pitch mechanism for automatic weed tracking and herbicide spraying. [Display omitted] [ABSTRACT FROM AUTHOR]

Compared with traditional methods, the action recognition model based on 3D convolutional deep neural network captures spatio-temporal features more accurately, resulting in higher accuracy. However, the large number of parameters and computational requirements of 3D models make it difficult to deploy on mobile devices with limited computing power. In order to achieve an efficient video action recognition model, we have analyzed and compared classic lightweight network principles and proposed the 3D-ShuffleViT network. By deeply integrating the self-attention mechanism with convolution, we have introduced an efficient ACISA module that further enhances the performance of our proposed model. This has resulted in exceptional performance in both context-sensitive and context-independent action recognition, while reducing deployment costs. It is worth noting that our 3D-ShuffleViT network, with a computational cost of only 6% of that of SlowFast-ResNet101, achieved 98% of the latter’s Top1 accuracy on the EgoGesture dataset. Furthermore, on the same CPU (Intel i5-8300H), its speed was 2.5 times that of the latter. In addition, when we deployed our model on edge devices, our proposed network achieved the best balance between accuracy and speed among lightweight networks of the same order.

With the rapid development of Industrial Internet of Things (IIoT) technology, video surveillance devices and video data in IIoT environments are massively growing and increasingly important. Deploying rapidly evolving deep learning-based object detection algorithms in IIoT can improve the efficiency of video data utilization and increase the automation and intelligence of the IIoT. Facing the transmission latency problem of massive video data, the algorithms are better deployed in edge devices. However, due to the large size and high computing power requirements of existing object detection algorithms, it is difficult to deploy them on edge devices with limited performance. Therefore, to solve this problem, we propose a smaller and less computationally intensive object detection algorithm, PG-YOLO. We replaced all the convolutional modules of YOLOV5 with low computational ghost modules and streamlined the backbone network to improve the network performance. Then, we propose a better pruning algorithm to compress the network, and finally improve the accuracy by distillation to obtain PG-YOLO. PG-YOLO is smaller and faster and can be deployed in edge devices. In addition, we conducted experimental validation on the SHWD dataset. The experimental results show that PG-YOLO can compress the volume of the model by 9 times compared with YOLOv5s, and the compressed detection accuracy reaches 0.934, with only 0.1% loss of accuracy. Also, the time required for inference is reduced by 32.7%, and the inference speed is improved by 10 FPS. Compared with other object detection algorithms, PG-YOLO also has advantages.

Advanced Driver Assistance Systems (ADAS) are a collection of intelligent solutions integrated into next-generation vehicles to assist in safe driving. When building ADAS systems, the main goals are that they are stable, flexible, easy to maintain, and allow for error tracing. If a driving assistance algorithm is designed to be implemented on one machine or in one model, there is a potential disadvantage that if one component fails, then the entire system would stop. We work on modularizing the ADAS system to be flexible to accommodate any changes or improvements based on up-to-date requirements. Using advanced current edge (or network) devices, we propose a Detection-based Driving Assistance algorithm, which can collaborate or integrate with an existing system in a vehicle. The core of any process is to ensure that the system has a predictable level of functionality and that any misbehavior can be easily traced to the root cause. The proposed system shows fast, real-time performance on edge devices with limited computing power.

The lens distortion process is essential for displaying VR contents on a head-mounted display (HMD) with a distorted display surface. This paper proposes a novel lens distortion algorithm to achieve real-time performance on edge devices with an embedded GPU. We employ unified memory space to reduce the data transfer overhead based on an architectural characteristic: an integrated CPU and GPU memory system. The lens distortion kernel is based on the lookup table-based mapping algorithm whose performance is bounded by memory operations rather than computations. To improve the kernel’s performance, we propose a compressed lookup table approach that reduces the memory transactions on the kernel while slightly increasing computation. We tested our method on three different edge devices and a desktop system while varying the image resolution from 720p (1, $280\times 720$ ) to 8K (7, $680\times 4.320$ ). Compared with prior GPU-based lookup table algorithms, our method achieved up to 1.72-times higher performance while consuming up to 28.93% less power. Also, our method demonstrates real-time performance for up to a 4K image with a low-end edge device (e.g., 56 FPS on Jetson Nano) and up to an 8K image with a mid-range device (e.g., 94 FPS on Jetson NX). These results demonstrate the benefits of our approach from the perspectives of both performance and energy.

The intrusion of foreign objects in tracks, one of the sources of injuries and fatalities in metro stations, can be solved as an anomaly detection task. However, the existing anomaly detection methods rarely consider the importance of updating the new knowledge from false alarm data, resulting in repeated mistakes. These methods are also impractical for edge devices that cannot afford the high calculation cost. A sequential updatable anomaly detection (SUAD) framework is proposed to tackle these problems. This framework is based on the Robbin-Monro algorithm and a fast version of Mahalanobis distance. A well-trained model of SUAD can continue to learn new knowledge through the sequential knowledge update module based on the Robbin-Monro algorithm without reviewing the old data. SUAD utilizes a new Mahalanobis distance calculation method based on principal component analysis. This new method exhibits a fast inference speed with a lighter model size than before. SUAD is evaluated on a self-built Metro Anomaly Detection (MAD) dataset and three public datasets. SUAD achieves an average area under the receiver operating characteristic curve score of 99.4% at image level and 99.6% at pixel level on MAD. SUAD also reduces at least 78% model size and 60% memory usage. Competitive results are also achieved in public datasets, including MVTec AD, beanTech Anomaly Detection, and CIFAR-10. [ABSTRACT FROM AUTHOR]

In the field of computer vision technology, deep learning of image processing has become an emerging research area. The semantic segmentation of an image is among the utmost essential and significant tasks in image-processing research, offering a wide range of application fields such as autonomous driving systems, medical diagnosis, surveillance security, etc. Thus far, many studies have suggested and developed neural network modules in deep learning. To the best of our knowledge, all existing neural networks for semantic segmentation have large parameter sizes and it is therefore unfeasible to implement those architectures in low-power and memory-limited embedded platforms such as FPGAs. Building an embedded platform with that architecture is possible after reducing the parameter size without affecting the module's architecture. The quantization technique lowers the precision of the neural network parameters while mostly keeping the accuracy. In this paper, we propose a quantization algorithm for a semantic segmentation deep learning architecture, which reduces the parameter size by four to eight times with a negligible accuracy abatement. As long as the parameter size is reduced, the deep learning architecture is improved in terms of required storage, computational speed, and power efficiency. [ABSTRACT FROM AUTHOR]

Within the scope of precision agriculture, many applications have been developed to support decision making and yield enhancement. Fruit detection has attracted considerable attention from researchers, and it can be used offline. In contrast, some applications, such as robot vision in orchards, require computer vision models to run on edge devices while performing inferences at high speed. In this area, most modern applications use an integrated graphics processing unit (GPU). In this work, we propose the use of a tensor processing unit (TPU) accelerator with a Raspberry Pi target device and the state-of-the-art, lightweight, and hardware-aware MobileDet detector model. Our contribution is the extension of the possibilities of using accelerators (the TPU) for edge devices in precision agriculture. The proposed method was evaluated using a novel dataset of peaches with three cultivars, which will be made available for further studies. The model achieved an average precision (AP) of 88.2% and a performance of 19.84 frames per second (FPS) at an image size of 640 × 480. The results obtained show that the TPU accelerator can be an excellent alternative for processing on the edge in precision agriculture. [ABSTRACT FROM AUTHOR]

The development of deep neural network (DNN) has provided transformative impacts on many fields, including computer vision and video recognition. However, the impact is limited by the need for large, labeled datasets to enable effective training. To address this fundamental problem, we propose a novel inter-network system (Dandelion), providing architecture support (Dandelion-architecture) for data augmentation that trains DNNs with rare images generated by the generative adversarial network (GAN) with orthogonal attributes modified (Dandelion-function. The approach can account for the latency requirement and resource limitation of target applications by exploiting data and computation reuses between the two networks; this amortizes the impact of bottleneck brought by GAN and facilitates design of inter-network accelerator. Moreover, we show how to implement two-network design on 3D architecture to further enhance the accelerator. Our results show that with the generated images, DNN yields 13.6% - 37.5% improvement on accuracy, depending on the data scarcity level. Our architecture achieves at least 30% speedup compared with the baseline while 40% of the overhead brought by the incorporation of GAN is reduced in our design compared with ScaleDeep, and 26.3% of performance improvement over TETRIS. [ABSTRACT FROM AUTHOR]

Due to the random distribution and irregular growth of fruit clusters on fruit trees, litchi mainly depends on manual picking and harvesting. Given this situation, the visual-guidance automatic picking system is utilised in existing studies to reduce the picking time effectively and improve economic benefits. However, the existing methods are primarily based on high-performance computing equipment in the laboratory. This equipment's volume and energy consumption makes it challenging to deploy, severely limiting the application of automatic picking. This study adopts portable, low-power edge devices and proposes an effective litchi detection method to solve the shortcomings mentioned above. First, litchi images in the orchard were collected, and data augmentation was adopted to improve the generalisation of the images. Then, an effective litchi detection model was constructed to detect the litchi. Finally, the trained model was compressed to remove redundant parameters according to the channel pruning and layer pruning algorithms to deploy the model on edge devices. The knowledge distillation was leveraged to fine-tune the model. The experimental results demonstrated that the proposed method could construct a model of only 7.6 MB with a compression rate of 96.8%. The model can achieve an average precision of 95.3% and an average recall of 97.3% at 117.6 frames per second (FPS), 1.8 times that of the original model. The proposed method can be easily deployed on portable and low-computing automatic picking equipment to achieve rapid and accurate detection of litchi, which is of great significance for biological systems. • A litchi detection method based on edge device for automatic picking is proposed. • Trained model was compressed at a 96.8% rate for better deployment on edge device. • The compressed model can achieve 95.3% precision and 97.3% recall at 117.6 FPS. • The proposed method is portable, low power consumption, and suitable for orchard. [ABSTRACT FROM AUTHOR]

Portable devices play an essential role where edge computing is necessary and mobility is required (e.g., robots in agriculture within remote-sensing applications). With the increasing applications of deep neural networks (DNNs) and accelerators for edge devices, several methods and applications have been proposed for simultaneous crop and weed detection. Although preliminary studies have investigated the performance of inference time for semantic segmentation of crops and weeds in edge devices, performance degradation has not been evaluated in detail when the required optimization is applied to the model for operation in such edge devices. This paper investigates the relationship between model tuning hyperparameters to improve inference time and its effect on segmentation performance. The study was conducted using semantic segmentation model DeeplabV3 with a MobileNet backbone. Different datasets (Cityscapes, PASCAL and ADE20K) were analyzed for a transfer learning strategy. The results show that, when using a model hyperparameter depth multiplier (DM) of 0.5 and the TensorRT framework, segmentation performance mean intersection over union (mIOU) decreased by 14.7% compared to that of a DM of 1.0 and no TensorRT. However, inference time accelerated dramatically by a factor of 14.8. At an image resolution of 1296 × 966 , segmentation performance of 64% mIOU and inference of 5.9 frames per second (FPS) was achieved in Jetson Nano's device. With an input image resolution of 513 × 513 , and hyperparameters output stride OS = 32 and DM = 0.5, an inference time of 0.04 s was achieved resulting in 25 FPS. The results presented in this paper provide a deeper insight into how the performance of the semantic segmentation model of crops and weeds degrades when optimization is applied to adapt the model to run on edge devices. Lastly, an application is described for the semantic segmentation of weeds embedded in the edge device (Jetson Nano) and integrated with the robotic orchard. The results show good spraying accuracy and feasibility of the method. [ABSTRACT FROM AUTHOR]

Efficient deployment of Deep Neural Networks (DNNs) on edge devices (i.e., FPGAs and mobile platforms) is very challenging, especially under a recent witness of the increasing DNN model size and complexity. Model compression strategies, including weight quantization and pruning, are widely recognized as effective approaches to significantly reduce computation and memory intensities, and have been implemented in many DNNs on edge devices. However, most state-of-the-art works focus on ad hoc optimizations, and there lacks a thorough study to comprehensively reveal the potentials and constraints of different edge devices when considering different compression strategies. In this article, we qualitatively and quantitatively compare the energy efficiency of FPGA-based and mobile-based DNN executions using mobile GPU and provide a detailed analysis. Based on the observations obtained from the analysis, we propose a unified optimization framework using block-based pruning to reduce the weight storage and accelerate the inference speed on mobile devices and FPGAs, achieving high hardware performance and energy-efficiency gain while maintaining accuracy. [ABSTRACT FROM AUTHOR]