Search

Improving agricultural productivity is essential due to rapid population growth, making early detection of crop diseases crucial. Although deep learning shows promise in smart agriculture, practical applications for identifying wheat diseases in complex backgrounds are limited. In this paper, we propose CropNet, a hybrid method that utilizes Red, Green, and Blue (RGB) imaging and a transfer learning approach combined with shallow convolutional neural networks (CNN) for further feature refinement. To develop our customized model, we conducted an extensive search for the optimal deep learning architecture. Our approach involves freezing the pre-trained model for feature extraction and adding a custom trainable CNN layer. Unlike traditional transfer learning, which typically uses trainable dense layers, our method integrates a trainable CNN, deepening the architecture. We argue that pre-trained features in transfer learning are better suited for a custom shallow CNN followed by a fully connected layer, rather than being fed directly into fully connected layers. We tested various architectures for pre-trained models including EfficientNetB0 and B2, DenseNet, ResNet50, MobileNetV2, MobileNetV3-Small, and Inceptionv3. Our approach combines the strengths of pre-trained models with the flexibility of custom architecture design, offering efficiency, effective feature extraction, customization options, reduced overfitting, and differential learning rates. It distinguishes itself from classical transfer learning techniques, which typically fine-tune the entire pre-trained network. Our aim is to provide a lightweight model suitable for resource-constrained environments, capable of delivering outstanding results. CropNet achieved 99.80% accuracy in wheat disease detection with reduced training time and computational cost. This efficient performance makes CropNet promising for practical implementation in resource-constrained agricultural settings, benefiting farmers and enhancing production. [ABSTRACT FROM AUTHOR]

Improving agricultural productivity is essential due to rapid population growth, making early detection of crop diseases crucial. Although deep learning shows promise in smart agriculture, practical applications for identifying wheat diseases in complex backgrounds are limited. In this paper, we propose CropNet, a hybrid method that utilizes Red, Green, and Blue (RGB) imaging and a transfer learning approach combined with shallow convolutional neural networks (CNN) for further feature refinement. To develop our customized model, we conducted an extensive search for the optimal deep learning architecture. Our approach involves freezing the pre-trained model for feature extraction and adding a custom trainable CNN layer. Unlike traditional transfer learning, which typically uses trainable dense layers, our method integrates a trainable CNN, deepening the architecture. We argue that pre-trained features in transfer learning are better suited for a custom shallow CNN followed by a fully connected layer, rather than being fed directly into fully connected layers. We tested various architectures for pre-trained models including EfficientNetB0 and B2, DenseNet, ResNet50, MobileNetV2, MobileNetV3-Small, and Inceptionv3. Our approach combines the strengths of pre-trained models with the flexibility of custom architecture design, offering efficiency, effective feature extraction, customization options, reduced overfitting, and differential learning rates. It distinguishes itself from classical transfer learning techniques, which typically fine-tune the entire pre-trained network. Our aim is to provide a lightweight model suitable for resource-constrained environments, capable of delivering outstanding results. CropNet achieved 99.80% accuracy in wheat disease detection with reduced training time and computational cost. This efficient performance makes CropNet promising for practical implementation in resource-constrained agricultural settings, benefiting farmers and enhancing production.

In the promising Artificial Intelligence of Things technology, deep learning algorithms are implemented on edge devices to process data locally. However, high-performance deep learning algorithms are accompanied by increased computation and parameter storage costs, leading to difficulties in implementing huge deep learning algorithms on memory and power constrained edge devices, such as smartphones and drones. Thus various compression methods are proposed, such as channel pruning. According to the analysis of low-level operations on edge devices, existing channel pruning methods have limited effect on latency optimization. Due to data processing operations, the pruned residual blocks still result in significant latency, which hinders real-time processing of CNNs on edge devices. Hence, we propose a generic deep learning architecture optimization method to achieve further acceleration on edge devices. The network is optimized in two stages, Global Constraint and Start-up Latency Reduction, and pruning of both channels and residual blocks is achieved. Optimized networks are evaluated on desktop CPU, FPGA, ARM CPU, and PULP platforms. The experimental results show that the latency is reduced by up to 70.40%, which is 13.63% higher than only applying channel pruning and achieving real-time processing in the edge device. [ABSTRACT FROM AUTHOR]

The state of Kerala in India has seen multiple instances of intense cyclones in recent years, resulting in heavy flooding. One of the biggest challenges faced by rescuers is the accessibility to flooded areas and buildings during rescue operations. In such scenarios, unmanned aerial vehicles (UAVs) can deliver reliable aerial visual data to aid planning and operations during rescue. Object detectors based on deep learning methods provide an effective solution to automate the process of detecting relevant information from image/video data. These models are complex and resource-hungry, leading to severe speed constraints during field operations. The pixel displacement algorithm (PDA), a portable and effective technique, is developed in this work to speed up object detection models on devices with limited resources, such as edge devices. This method can be integrated with all object detection models to speed up the inference time. The proposed method is combined with multiple object detection models in this work to show its effectiveness. The YOLOv4 model combined with the proposed method outperformed the AP50 performance of the YOLOv4-tiny model by 6 % while maintaining the same processing time. This approach gave almost 10 × speed improvement to Jetson Nano at an accuracy cost of 3 % when compared to YOLOv4. Further, a model to predict maximum pixel shift with respect to frame skip is proposed using parameters such as the altitude and velocity of the UAV and the tilt of the camera. Accurate prediction of pixel shift leads to a reduced search area, leading to reduced inference time. The effectiveness of the proposed model was tested against annotated locations, and it was found that the method was able to predict the search area for each test video segment with a high degree of accuracy. [ABSTRACT FROM AUTHOR]

Audio classification using deep learning models, which is essential for applications like voice assistants and music analysis, faces challenges when deployed on edge devices due to their limited computational resources and memory. Achieving a balance between performance, efficiency, and accuracy is a significant obstacle to optimizing these models for such constrained environments. In this investigation, we evaluate diverse deep learning architectures, including Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM), for audio classification tasks on the ESC 50, UrbanSound8k, and Audio Set datasets. Our empirical findings indicate that Mel spectrograms outperform raw audio data, attributing this enhancement to their synergistic alignment with advanced image classification algorithms and their congruence with human auditory perception. To address the constraints of model size, we apply model-compression techniques, notably magnitude pruning, Taylor pruning, and 8-bit quantization. The research demonstrates that a hybrid pruned model achieves a commendable accuracy rate of 89 percent, which, although marginally lower than the 92 percent accuracy of the uncompressed CNN, strikingly illustrates an equilibrium between efficiency and performance. Subsequently, we deploy the optimized model on the Raspberry Pi 4 and NVIDIA Jetson Nano platforms for audio classification tasks. These findings highlight the significant potential of model-compression strategies in enabling effective deep learning applications on resource-limited devices, with minimal compromise on accuracy. [ABSTRACT FROM AUTHOR]

Featured Application: The proposed method reveals the widespread real-world applicability of malware detection, particularly in securing IoT devices. Its faster inference speed and high fidelity score illustrate the practicability of on-device malware detection without incorporating desktop/server-class hardware via the internet for faster inference. Detecting malware is a crucial defense mechanism against potential cyber-attacks. However, current methods illustrate significant limitations in achieving high performance while maintaining faster inference on edge devices. This study proposes a novel deep network with dual-attention feature refinement on a two-branch deep network to learn real-time malware detection on edge platforms. The proposed method introduces lightweight spatial-asymmetric attention for refining the extracted features of its backbone and multi-head attention to correlate learned features from the network branches. The experimental results show that the proposed method can significantly outperform existing methods in quantitative evaluation. In addition, this study also illustrates the practicability of a lightweight deep network on edge devices by optimizing and deploying the model directly on the actual edge hardware. The proposed optimization strategy achieves a frame rate of over 545 per second on low-power edge devices. [ABSTRACT FROM AUTHOR]

As open-source RISC-V cores continue to be released, the development of low-power multicore processors utilizing these cores is invigorating the edge/IoT device market. Nevertheless, comprehensive research on developing low-power multicore processors with integrated security features using existing open RISC-V cores remains limited. This study addresses this gap by introducing AESware, a dedicated lightweight hardware designed for energy-efficient AES (Advanced Encryption Standard) task execution, contributing to the development of AES-specific low-power RISC-V multicore processors. AESware supports variable key lengths and ensures minimal power consumption with a compact design. This standalone IP (Intellectual Property) is compatible with various open RISC-V cores, offering scalability and convenience. And importantly, we propose the most energy-efficient architecture for multicore processors equipped with AESware. Instead of assigning dedicated AESware to each core, we introduce a shared AESware architecture to maximize energy efficiency. We develop an operational algorithm for task scheduling in AESware, achieving maximum utilization and minimal latency while maintaining its lightweight nature. To evaluate our solution, we developed 24 processors into three groups: AESware-equipped, baseline, and those with an external AES accelerator per core. After FPGA (Field-Programmable Gate Array) prototyping for functional verification and power consumption analysis via 45 nm process technology synthesis, our findings revealed significant energy savings. AESware-equipped processors achieved up to 76%, 47%, and 33% energy savings at dual-, quad-, and octa-core configurations compared to baseline, respectively, and were more energy-efficient in running AES applications than those with individual accelerators.

Traditional weed management often involves blanket herbicide spraying, resulting in substantial herbicide wastage, environmental concerns, and herbicide resistant issues. Smart spraying systems utilizing robotics and sensors technologies can minimize herbicide usage and provide a sustainable solution for site-specific weed management. A machine vision-based spraying system was designed and developed for weed identification and precise spray application onto the target weeds. The sprayer platform utilizes a deep learning YOLOv4 model to accurately recognize multiple weed species, facilitating targeted spray application. The platform is equipped with an FLIR RGB camera for real-time image acquisition and Nvidia Jetson AGX Orin edge device for deploying weed detection deep-learning model. The GPIO pins of Nvidia Jetson were utilized to activate relay, providing precise on/off control over the TeeJet solenoid valves for spot spraying. Both indoor and field experiments were conducted to evaluate and compare the performance of vision-based sprayer system for weed identification and precise spraying onto the target weeds. In the indoor experiment, the sprayer system showed the average effective spraying rate of 93.33 %, with the precision of 100 % and recall of 92.8 %. Conversely, the field experiment resulted in a slightly lower average effective spraying rate of 90.6 %, while maintaining a precision of 95.5 % and a recall of 89.47 %. The reduced accuracy of the spraying system in field experiment was due to varying outdoor conditions such as lighting, shadows, and wind velocity. Overall, the result of this study demonstrates the spraying system's potential for targeted herbicide application onto the grid cells containing weeds, effectively reducing herbicide usage and overall weed management costs.

Advancements in technology has accelerated the evolution of bone age assessment (BAA) methodologies, one of which is deep learning algorithms, which overcome the drawbacks of conventional approaches. In spite of excellent effectiveness of deep neural networks in detection of the correct class for bone age, they have a significant degree of complexity due to the numerous parameters they employ for each region of interest (ROI). In this paper, we propose a BAA method using a hybrid knowledge distillation (KD) paradigm in order to conquer this difficulty by mapping different ROIs into a single ROI. In this regard, the student receives knowledge from a teacher network that has been pre-trained on six ROIs including bones of five fingers and the wrist, transfers the knowledge of its final response layer and internal layers to the student. Then, six student models each of which is constructed based on just one of these ROIs, while receiving the information of the teacher model. Empirical results on digital hand atlas report that our student model trained on one ROI obtains 95% accuracy on 19 classes of bone age.

In this work, we present a system that detects Freezing of Gait Detection (FOG) that uses of a single wearable inertial sensor to automatically evaluate a Parkinson's patient's gait instability and detect FOG in real-time. A wearable vibrator is our cueing system which is triggered by the FOG detection whenever a FOG episode takes place. The vibration impulses help the patient to prevent FOG by switching to voluntarily movement execution. Sensor data were collected from nine patients with Parkinson's disease performing Unified Parkinson's Disease Rating Scale (UPDRS) test under the supervision of a clinical expert. Along with data recording, a video was taken from patient's parkour. The data were labeled through the recorded video of the patient's tests and FOG and non-FOG data were assigned. A machine learning model using a deep Long-Short-Term-Memory (LSTM) employ the accelerometer data from the sensor and the inference leads to a FOG or non-FOG classification. The FOG detection model is pruned, quantized and used for real-time inference on Google Coral board worn on the patient' body. The model deployed on a Google coral board sends a trigger to the cuing device right after the FOG detection and the patient get alert for the happening FOG. The individualized model for one-second windows applied in this work performed an average of 91.5% of sensitivity and 86.5% specificity for models running on PC and 91.7% of sensitivity and 86.7% of specificity for the models tested on Google coral with the latency of 50 millisecond on real-time testing. [ABSTRACT FROM AUTHOR]

This study introduces a novel selective multi-branch network architecture designed to speed up object localization and classification on low-performance edge devices. The concept builds upon the You Only Look at Interested Cells (YOLIC) method, which was proposed by us earlier. In this approach, we categorize cells of interest (CoIs) into distinct regions of interest (RoIs) based on their locations and urgency. We then employ some expert branch networks for detailed object detection in each of the RoIs. To steer these branches effectively, a selective attention unit is added into the detection process. This unit can locate RoIs that are likely to contain objects under concern and trigger corresponding expert branch networks. The inference can be more efficient because only part of the feature map is used to make decisions. Through extensive experiments on various datasets, the proposed network demonstrates its ability to reduce the inference time while still maintaining competitive performance levels compared to the current detection algorithms. [ABSTRACT FROM AUTHOR]

Amidst an energy crisis stemming from increased energy costs and the looming threat of war, there has been a burgeoning interest in energy conservation and management worldwide. Industrial complexes constitute a significant portion of total energy consumption. Hence, reducing energy consumption in these complexes is imperative for energy preservation. Typically, factories within similar industries aggregate in industrial complexes and share similar energy utilities. However, they often fail to capitalize on this shared infrastructure efficiently. To address this issue, a network system employing a virtual utility plant has been proposed. This system enables proactive measures to counteract energy surplus or deficit through AI-based predictions, thereby maximizing energy efficiency. Nevertheless, deploying conventional server systems within factories poses considerable challenges. Therefore, leveraging edge devices, characterized by low power consumption, high efficiency, and minimal space requirements, proves highly advantageous. Consequently, this study focuses on constructing and employing data collection and AI systems to utilize edge devices as standalone systems in each factory. To optimize the AI system for low-performance edge devices, we employed the integration-learning AI modeling technique. Evaluation results demonstrate that the proposed system exhibits high stability and reliability. [ABSTRACT FROM AUTHOR]

Despite the widespread applicability of malware detection on the Internet of Things (IoT) and edge platforms, efficient malware detection on edge devices is significantly challenging due to hardware limitations. This study proposes a novel edge-friendly framework for learning malware detection by incorporating quantization awareness. The proposed framework incorporates a lightweight deep neural network leveraging long-distance pixel dependency, spatial-asymmetric attention, and normalization-free multi-head attention (MHA) block to refine salient features. Additionally, the proposed framework introduces an efficient quantization-aware training (QAT) strategy to optimize the proposed model for edge-friendly INT8 precision. The practicality of the proposed model has been validated through deployment on ARM-based hardware and extensive evaluations on standard x64 hardware. Experimental results show that the proposed method significantly outperforms existing malware detection methods in numerous precision modes on both platforms. The proposed method achieved 99.36% accuracy on the MalImg dataset and 98.41% on the IoT malware dataset with FP16 precision at 662 fps. With INT8 precision, it achieved 95.62% accuracy on the MalImg dataset and 97.36% on the IoT malware dataset, maintaining 1197 fps on a low-power edge device. To our knowledge, this is the first work in the open literature demonstrating QAT’s practicality for malware detection on edge platforms.

In this paper, we present a novel off-chip network architecture, the Tile Serial Peripheral Interface (TSPI), designed for low-power, scalable edge AI accelerators. Our approach modifies the conventional SPI to support a modular network structure that facilitates the scalable connection of multiple accelerators. The TSPI network employs a subset mapping algorithm for efficient routing and integrates the message passing interface (MPI) protocol to ensure rapid data distribution and aggregation. This modular architecture significantly reduces power consumption and improves processing speed. Experimental results demonstrate that our proposed TSPI network achieves a 54.7% reduction in power consumption and an 82.3% decrease in switching power compared to traditional SPI networks, along with a 23% increase in processing speed when utilizing 16 nodes. These advancements make the TSPI network an effective solution for enhancing AI performance in edge computing environments.

While computer vision and computing technology advances have facilitated advanced driver assistance applications, systems with multi-task design remain highly demanding to operate at high speed on resource-constrained devices. Our study addresses this challenge by proposing a real-time driver assistance solution specifically developed for a single Jetson AGX Xavier embedded device. It simultaneously performs lane detection, traffic object detection and recognition, and rule-based scene analysis. To achieve high throughput (up to 43 frames per second) without reliance on additional hardware or cloud server, the system exploits Jetson device’s specialized AI accelerator and employs various optimization techniques: multithreaded programming, the TensorRT framework, and post-training quantization. The modular design integrates state-of-the-art task-specific methods and ensures adaptation to diverse traffic scenarios across countries as well as future hardware and solutions. Experimental results using a Korean dashcam traffic dataset validated the system’s effectiveness and practicality.

Despite the achievements of deep neural-network-based object detection, detecting small objects in low-resolution images remains a challenging task due to limited information. A possible solution to alleviate the issue involves integrating super-resolution (SR) techniques into object detectors, particularly enhancing feature maps for small-sized objects. This paper explores the impact of high-resolution super-resolved feature maps generated by SR techniques, especially for a one-stage detector that demonstrates a good compromise between detection accuracy and computational efficiency. Firstly, this paper suggests the integration of an SR module named feature texture transfer (FTT) into the one-stage detector, YOLOv4. Feature maps from the backbone and the neck of vanilla YOLOv4 are combined to build a super-resolved feature map for small-sized object detection. Secondly, it proposes a novel SR module with more impressive performance and slightly lower computation demand than the FTT. The proposed SR module utilizes three input feature maps with different resolutions to generate a super-resolved feature map for small-sized object detection. Lastly, it introduces a simplified version of an SR module that maintains similar performance while using only half the computation of the FTT. This attentively simplified module can be effectively used for real-time embedded systems. Experimental results demonstrate that the proposed approach substantially enhances the detection performance of small-sized objects on two benchmark datasets, including a self-built surveillance dataset and the VisDrone2019 dataset. In addition, this paper employs the proposed approach on an embedded system with a Qualcomm QCS610 and demonstrates its feasibility for real-time operation on edge devices. [ABSTRACT FROM AUTHOR]

The 4th Edge Computing Workshop (doors 2024) held in Zhytomyr, Ukraine, aimed to bring together researchers to share advances and applications in edge computing, a critical technology for enabling the Internet of Things and analytics close to data sources. This report summarises the event, highlighting the peer-reviewed papers covering topics from edge hardware to AI and security. The nine accepted papers, selected from 19 submissions, propose innovative approaches to advance the field. Common themes include leveraging edge computing for sensing applications and addressing resource utilisation and management challenges. The workshop stimulated valuable discussion and collaboration around these important research directions.

In deep learning, acquiring sufficient data is crucial for making informed decisions. However, due to concerns regarding security and privacy, obtaining enough data for training models in the era of deep learning is challenging. There is a growing need for machine learning (ML) solutions that can derive accurate conclusions from small data while preserving privacy. Smartphones, which are widely used and generate large amounts of data, can serve as an excellent source for data generation. One suitable approach for regularly evaluating real-world data from edge devices is Tiny Machine Learning (TinyML). With the increasing number of edge devices involved in transmitting private data, it's vital to have a method that allows computations to be performed on edge devices and pushed to the edge rather than over the network. Considering these obstacles, the combination of TinyML edge devices and Federated Learning can be applied in the early treatment of Bronchus Cancer. Under the framework of federated learning, local edge devices are trained independently and then integrated into the server without exchanging edge device data. This approach enables the creation of secure models without sharing information, resulting in a highly efficient solution with enhanced data security and accessibility. This article provides a comprehensive discussion of the key challenges addressed in recent literature, accompanied by an extensive examination of relevant studies. Additionally, a novel model based on edge devices and federated learning is proposed.

This study presents the concept of a computationally efficient machine learning (ML) model for diagnosing and monitoring Parkinson's disease (PD) using rest-state EEG signals (rs-EEG) from 20 PD subjects and 20 normal control (NC) subjects at a sampling rate of 128 Hz. Based on the comparative analysis of the effectiveness of entropy calculation methods, fuzzy entropy showed the best results in diagnosing and monitoring PD using rs-EEG, with classification accuracy (ARKF) of ~99.9%. The most important frequency range of rs-EEG for PD-based diagnostics lies in the range of 0–4 Hz, and the most informative signals were mainly received from the right hemisphere of the head. It was also found that ARKF significantly decreased as the length of rs-EEG segments decreased from 1000 to 150 samples. Using a procedure for selecting the most informative features, it was possible to reduce the computational costs of classification by 11 times, while maintaining an ARKF ~99.9%. The proposed method can be used in the healthcare internet of things (H-IoT), where low-performance edge devices can implement ML sensors to enhance human resilience to PD. [ABSTRACT FROM AUTHOR]

User authentication has traditionally been performed using methods such as passwords or fingerprints. However, passwords have security vulnerabilities, and fingerprints may hinder user convenience. To address these issues, a novel user authentication method based on biosignals, specifically electromyogram (EMG) signals, is proposed. Using biosignals like EMG offers several advantages, including the ability to acquire data without user awareness, independence from the user's environment, rapid acquisition, and enhanced security. However, one challenge with using EMG signals for authentication has been their relatively low accuracy. In this paper, a neural network is implemented using a small number of parameters (fewer than 7000) to produce a wearable device using biosignals, and user authentication accuracy is secured using the maximal overlap discrete wavelet transform (MODWT) method and the Siamese network. The MODWT method is highly effective for the time and frequency analysis of time series data, and the Siamese network is a representative method for few-shot learning. The proposed neural network is verified using Chosun University's user authentication dataset, encompassing data from 100 individuals. Finally, this proposed network is implemented on an edge device such as field-programmable gate arrays (FPGAs) so that it can be applied to a wearable user authentication system. By implementing the Siamese network in FPGA-based edge devices, it was possible to secure user authentication performance at 94% accuracy and an authentication speed within 1.5 ms. In the case of accuracy, it is expected to be further improved by using the multimodal technique of biosignals. Also, the proposed system can be easily fabricated for digital integrated chips (ICs). [ABSTRACT FROM AUTHOR]

Compared with traditional methods, the action recognition model based on 3D convolutional deep neural network captures spatio-temporal features more accurately, resulting in higher accuracy. However, the large number of parameters and computational requirements of 3D models make it difficult to deploy on mobile devices with limited computing power. In order to achieve an efficient video action recognition model, we have analyzed and compared classic lightweight network principles and proposed the 3D-ShuffleViT network. By deeply integrating the self-attention mechanism with convolution, we have introduced an efficient ACISA module that further enhances the performance of our proposed model. This has resulted in exceptional performance in both context-sensitive and context-independent action recognition, while reducing deployment costs. It is worth noting that our 3D-ShuffleViT network, with a computational cost of only 6% of that of SlowFast-ResNet101, achieved 98% of the latter's Top1 accuracy on the EgoGesture dataset. Furthermore, on the same CPU (Intel i5-8300H), its speed was 2.5 times that of the latter. In addition, when we deployed our model on edge devices, our proposed network achieved the best balance between accuracy and speed among lightweight networks of the same order. [ABSTRACT FROM AUTHOR]

Abstract Pruning is showing huge potential for compressing and accelerating deep neural networks by eliminating redundant parameters. Along with more terminal chips integrated with AI accelerators for internet of things (IoT) devices, structured pruning is gaining popularity with the edge computing research area. Different from filter pruning and group-wise pruning, stripe-wise pruning (SWP) conducts pruning at the level of stripes in each filter. By introducing filter skeleton (FS) to each stripe, the existing SWP method sets an absolute threshold for the values in FS and removes the stripes whose corresponding values in FS could not meet the threshold. Starting with investigation into the process of stripe wise convolution, we use the statistical properties of the weights located on each stripe to learn the importance between those stripes in a filter and remove stripes with low importance. Our pruned VGG-16 achieves the existing results by a fourfold reduction in parameter with only 0.4% decrease in accuracy. Results from comprehensive experiments on IoT devices are also presented.

Detecting malware is a crucial defense mechanism against potential cyber-attacks. However, current methods illustrate significant limitations in achieving high performance while maintaining faster inference on edge devices. This study proposes a novel deep network with dual-attention feature refinement on a two-branch deep network to learn real-time malware detection on edge platforms. The proposed method introduces lightweight spatial-asymmetric attention for refining the extracted features of its backbone and multi-head attention to correlate learned features from the network branches. The experimental results show that the proposed method can significantly outperform existing methods in quantitative evaluation. In addition, this study also illustrates the practicability of a lightweight deep network on edge devices by optimizing and deploying the model directly on the actual edge hardware. The proposed optimization strategy achieves a frame rate of over 545 per second on low-power edge devices.

Audio classification using deep learning models, which is essential for applications like voice assistants and music analysis, faces challenges when deployed on edge devices due to their limited computational resources and memory. Achieving a balance between performance, efficiency, and accuracy is a significant obstacle to optimizing these models for such constrained environments. In this investigation, we evaluate diverse deep learning architectures, including Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM), for audio classification tasks on the ESC 50, UrbanSound8k, and Audio Set datasets. Our empirical findings indicate that Mel spectrograms outperform raw audio data, attributing this enhancement to their synergistic alignment with advanced image classification algorithms and their congruence with human auditory perception. To address the constraints of model size, we apply model-compression techniques, notably magnitude pruning, Taylor pruning, and 8-bit quantization. The research demonstrates that a hybrid pruned model achieves a commendable accuracy rate of 89 percent, which, although marginally lower than the 92 percent accuracy of the uncompressed CNN, strikingly illustrates an equilibrium between efficiency and performance. Subsequently, we deploy the optimized model on the Raspberry Pi 4 and NVIDIA Jetson Nano platforms for audio classification tasks. These findings highlight the significant potential of model-compression strategies in enabling effective deep learning applications on resource-limited devices, with minimal compromise on accuracy.

Amidst an energy crisis stemming from increased energy costs and the looming threat of war, there has been a burgeoning interest in energy conservation and management worldwide. Industrial complexes constitute a significant portion of total energy consumption. Hence, reducing energy consumption in these complexes is imperative for energy preservation. Typically, factories within similar industries aggregate in industrial complexes and share similar energy utilities. However, they often fail to capitalize on this shared infrastructure efficiently. To address this issue, a network system employing a virtual utility plant has been proposed. This system enables proactive measures to counteract energy surplus or deficit through AI-based predictions, thereby maximizing energy efficiency. Nevertheless, deploying conventional server systems within factories poses considerable challenges. Therefore, leveraging edge devices, characterized by low power consumption, high efficiency, and minimal space requirements, proves highly advantageous. Consequently, this study focuses on constructing and employing data collection and AI systems to utilize edge devices as standalone systems in each factory. To optimize the AI system for low-performance edge devices, we employed the integration-learning AI modeling technique. Evaluation results demonstrate that the proposed system exhibits high stability and reliability.

Ensuring personal safety and preventing accidents are critical aspects of power construction safety supervision. However, current monitoring methods are inefficient and unreliable as most of them rely on manual monitoring and transmission, which results in slow detection and delayed warnings regarding violations. To overcome these challenges, we propose an intelligent detection system that can accurately identify instances of illegal wearing of power construction workers in real-time. Firstly, we integrated the squeeze-and-excitation (SE) module into our convolutional neural network to enhance detection accuracy. This module effectively prioritizes informative features while suppressing less relevant ones, resulting in improved overall performance. Secondly, we present an embedded real-time detection system that utilizes Jetson Xavier NX and Edge-YOLO. This system promptly detects and alerts power construction workers of instances of illegal wearing behavior. To ensure a lightweight implementation, we design appropriate detection heads based on target size and distribution, reducing model parameters while enhancing detection speed and minimizing accuracy loss. Additionally, we employed data augmentation to enhance the system's robustness. Our experimental results demonstrate that our improved Edge-YOLO model achieves high detection precision and recall rates of 0.964 and 0.966, respectively, with a frame rate of 35.36 frames per second when deployed on Jetson Xavier NX. Therefore, Edge-YOLO proves to be an ideal choice for intelligent real-time detection systems, providing superior accuracy and speed performance compared to the original YOLOv5s model and other models in the YOLO series for safety monitoring at construction sites. [ABSTRACT FROM AUTHOR]

The illegal dumping of aluminum and plastic into cities and marine areas leads to negative impacts on the ecosystem and contributes to increased environmental pollution. Although volunteer trash pickup activities have increased in recent years, they require significant effort, time, and money. Therefore, we propose automated trash pickup robot, which incorporates autonomous movement and trash pickup arms. Although these functions have been actively developed, relatively little research has focused on trash detection. As such, we have developed a trash detection function by using deep learning models to improve the accuracy. First, we created a new trash dataset that classifies four types of trash with high illegal dumping volumes (cans, plastic bottles, cardboard, and cigarette butts). Next, we developed a new you only look once (YOLO)-based model with low parameters and computations. We trained the model on a created dataset and a dataset consisting of marine trash created during previous research. In consequence, the proposed models achieve the same detection accuracy as the existing models on both datasets, with fewer parameters and computations. Furthermore, the proposed models accelerate the edge device's frame rate. [ABSTRACT FROM AUTHOR]

Offloading tasks from edge devices to the cloud is an important method to enhance the performance of the edge device. With the help of EH (Energy Harvesting) technology, the edge device can use the collected green energy to support its operations. Most offloading scheduling methods use as much green energy as the edge device collected. Unlike prior research, we consider the long-term benefits of energy. In this paper, we forecast the solar energy supply with meteorological methods which are based on the weather forecast data. Then, we use quadratic programming to allocate energy based on the forecast energy to maximize energy efficiency. Finally, we use NSGA (Non-dominated Sorting Genetic Algorithms) to offload tasks in the edge device. Simulations show that our proposed method not only minimizes the execution time and the energy consumption of clouds, but also enhances the QoE of users. • We model the edge system considering the battery and the weather information. • Energy allocation for edge devices based on the weather forecast information. • A offloading task method is proposed based on NGSA III. • Simulations are given with benefit function. • Comparisons are given considering multiple aspects. [ABSTRACT FROM AUTHOR]

Nowadays, the Internet of Things (IoT) has become widely used at various places and for various applications. To facilitate this trend, we have developed the IoT application server platform called SEMAR (Smart Environmental Monitoring and Analytical in Real-Time), which offers standard features for collecting, displaying, and analyzing sensor data. An edge device is usually installed to connect sensors with the server, where the interface configuration, the data processing, the communication protocol, and the transmission interval need to be defined by the user. In this paper, we proposed an edge device framework for SEMAR to remotely optimize the edge device utilization with three phases. In the initialization phase, it automatically downloads the configuration file to the device through HTTP communications. In the service phase, it converts data from various sensors into the standard data format and sends it to the server periodically. In the update phase, it remotely updates the configuration through MQTT communications. For evaluations, we applied the proposal to the fingerprint-based indoor localization system (FILS15.4) and the data logging system. The results confirm the effectiveness in utilizing SEMAR to develop IoT application systems. [ABSTRACT FROM AUTHOR]

Stereo matching is an important component technology that constitutes the 3D perception capability of autonomous vehicles. On resource-constrained edge devices, it is very important to compute in real-time with very low time. However, most stereo matching networks focus on generating disparity maps on high-end GPUs, which do not meet the real-time requirements on edge devices. To solve this problem, a new stereo matching network is proposed in this paper to achieve real-time stereo matching on edge devices. The proposed network greatly improves the inference speed by constructing a low-resolution feature extractor, and by using multi-stage residual methods for stereo matching. In particular, we propose a method that combines the group-wise L1 distance & the group-wise correlation cost volume and an effective attention-based 3D cost aggregation method. Our network achieves a good balance between speed and accuracy on the KITTI 2012 and KITTI 2015 datasets. The proposed network achieves 2.77% and 3.44% accuracy (D1-all) on KITTI 2012 and KITTI 2015, respectively. With TensorRT, the proposed network achieves 31.8 FPS and outperforms the real-time results of most state-of-the-art networks on NVIDIA Jetson Nano edge devices.

Wheat is the most important and dominating crop in Pakistan in terms of production and acreage, which is grown on 37% of the cultivated area, accounting for 70% of the total production. However, wheat yield is highly affected by stripe rust, which is considered the most devastating fungal disease, causing 5.5 million tonnes of loss per year globally. In order to minimize this loss, the accurate and timely detection of rust disease is crucial instead of manual inspection. Towards this end, we propose a system to detect wheat rust disease and classify its infection types into four classes, including healthy, resistant, moderate (moderately resistant to moderately susceptible), and susceptible. The wheat rust dataset is collected indigenously from the National Agricultural Research Centre, Islamabad. A pre-trained U2 Net model is used to remove the background and extract the leaf containing the rust disease. Subsequently, two deep learning classifiers, including the Xception model and ResNet-50 are applied to classify the stripe rust severity levels, where the ResNet-50 model outperformed with the highest accuracy of 96%. This research presents a comparison between two state-of-the-art deep learning classifiers in terms of accuracy, memory utilization, and prediction time, which will assist the research community in selecting the most appropriate model for plant disease detection. Moreover, to assess the external validity, the performance of these classifiers is compared with the existing technique using a publicly available dataset, which confirms the validity of the results. Additionally, an intelligent edge computing rust detection device has been developed, where the trained ResNet-50 model is deployed, which facilitates the farmers to monitor the rust attack. The proposed research is aimed to assist the agricultural community to employ preventive measures in a site-specific manner based on the accurate diagnosis of rust disease & its severity, which is intended to improve the quality of the wheat as well as production.

Pruning is showing huge potential for compressing and accelerating deep neural networks by eliminating redundant parameters. Along with more terminal chips integrated with AI accelerators for internet of things (IoT) devices, structured pruning is gaining popularity with the edge computing research area. Different from filter pruning and group-wise pruning, stripe-wise pruning (SWP) conducts pruning at the level of stripes in each filter. By introducing filter skeleton (FS) to each stripe, the existing SWP method sets an absolute threshold for the values in FS and removes the stripes whose corresponding values in FS could not meet the threshold. Starting with investigation into the process of stripe wise convolution, we use the statistical properties of the weights located on each stripe to learn the importance between those stripes in a filter and remove stripes with low importance. Our pruned VGG-16 achieves the existing results by a fourfold reduction in parameter with only 0.4% decrease in accuracy. Results from comprehensive experiments on IoT devices are also presented. [ABSTRACT FROM AUTHOR]

Competitiveness in industry requires smooth, efficient, and high-quality operation. For some industrial applications or process control and monitoring applications, it is necessary to achieve high availability and reliability because, for example, the failure of availability in industrial production can have serious consequences for the operation and profitability of the company, as well as for the safety of employees and the surrounding environment. At present, many new technologies that use data obtained from various sensors for evaluation or decision-making require the minimization of data processing latency to meet the needs of real-time applications. Cloud/Fog and Edge computing technologies have been proposed to overcome latency issues and to increase computing power. However, industrial applications also require the high availability and reliability of devices and systems. The potential malfunction of Edge devices can cause a failure of applications, and the unavailability of Edge computing results can have a significant impact on manufacturing processes. Therefore, our article deals with the creation and validation of an enhanced Edge device model, which in contrast to the current solutions, is aimed not only at the integration of various sensors within manufacturing solutions, but also brings the required redundancy to enable the high availability of Edge devices. In the model, we use Edge computing, which performs the recording of sensed data from various types of sensors, synchronizes them, and makes them available for decision making by applications in the Cloud. We focus on creating a suitable Edge device model that works with the redundancy, by using either mirroring or duplexing via a secondary Edge device. This enables high Edge device availability and rapid system recovery in the event of a failure of the primary Edge device. The created model of high availability is based on the mirroring and duplexing of the Edge devices, which support two protocols: OPC UA and MQTT. The models were implemented in the Node-Red software, tested, and subsequently validated and compared to confirm the required recovery time and 100% redundancy of the Edge device. In the contrast to the currently available Edge solutions, our proposed extended model based on Edge mirroring is able to address most of the critical cases, where fast recovery is required, and no adjustments are needed for critical applications. The maturity level of Edge high availability can be further extended by applying Edge duplexing for process control. [ABSTRACT FROM AUTHOR]

The tremendous growth in the internet of things (IoT) technology has provided great support for e-healthcare applications. The remote health monitoring of patients at home is gaining popularity with the increase in design of number of IoT enabled wearable devices. The secure and efficient health data retrieval during continuous uploading of huge data on network is a challenge. In addition, another challenge exists due to limited resources available with IoT device to proceed efficient data retrieval. On the other hand, the edge cloud in edge computing technology provide abundant resources to its devices for various computations, storage and processing. The integration of IoT with edge would solve the resource constraint issue. However, efficient retrieval of health data from large pool of data would still be an issue due to IP based solutions offered by IoT. The Named Data Networking (NDN) is a promising solution for efficient delivery of data based on its name based content searching. This paper proposed NDN communication based secure remote healthcare monitoring framework by integrating IoT with edge computing where security to the each individual content is provided through hashing and encryption mechanism. The performance of the proposed framework is evaluated against its companion in Icarus simulator. The results proved the efficiency of our framework for content retrieval delay and cost associated in accessing the required data.

The rapid development of computer biometrics over the past 2-3 decades is largely due to the development and widespread introduction into clinical practice of new methods of studying human body health, including pulse methods. It is possible to judge changes in hemodynamic characteristics, heart rate, and blood flow rate in the studied part of the body based on the parameters of the pulse wave signal. At the same time, the physical processes of the formation of the pulse wave shape have not been fully studied, although the number of biophysical models of blood circulation is quite significant. The development of such a model will allow us to effectively apply modern developments in digital signal processing to the pulse wave and increase its diagnostic value. A qualitative model of pulse signal can be entrusted to the development of the base unit of the biotechnical system as a type of edge device. The work is devoted to the improvement of methods of rapid diagnosis of the cardiovascular system based on the analysis of model pulse-grams. An adequate mathematical model of the pulse wave, which corresponds to real pulse signals in different states of the human body and contains mathematical relationships between the main parameters of pulse-grams, has been refined. The algorithm of express diagnostics with the established criteria of the analysis of pulsograms is offered.

In order to solve the problem of an accurate recognition of tea picking through tea picking robots, an edge device detection method is proposed in this paper based on ShuffleNetv2-YOLOv5-Lite-E for tea with one bud and two leaves. This replaces the original feature extraction network by removing the Focus layer and using the ShuffleNetv2 algorithm, followed by a channel pruning of YOLOv5 at the neck layer head, thus achieving the purpose of reducing the model size. The results show that the size of the improved generated weight file is 27% of that of the original YOLOv5 model, and the mAP value of ShuffleNetv2-YOLOv5-Lite-E is 97.43% and 94.52% on the pc and edge device respectively, which are 1.32% and 1.75% lower compared to that of the original YOLOv5 model. The detection speeds of ShuffleNetv2-YOLOv5-Lite-E, YOLOv5, YOLOv4, and YOLOv3 were 8.6 fps, 2.7 fps, 3.2 fps, and 3.4 fps respectively after importing the models into an edge device, and the improved YOLOv5 detection speed was 3.2 times faster than that of the original YOLOv5 model. Through the detection method, the size of the original YOLOv5 model is effectively reduced while essentially ensuring recognition accuracy. The detection speed is also significantly improved, which is conducive to the realization of intelligent and accurate picking for future tea gardens, laying a solid foundation for the realization of tea picking robots. [ABSTRACT FROM AUTHOR]

Solar energy is considered the most abundant form of energy available on earth. However, the efficiency of photovoltaic (PV) panels is greatly reduced due to the accumulation of dust particles on the surface of PV panels. The optimization of the cleaning cycles of a PV power plant through condition monitoring of PV panels is crucial for its optimal performance. Specialized equipment and weather stations are deployed for large-scale PV plants to monitor the amount of soil accumulated on panel surface. However, not much focus is given to small- and medium-scale PV plants, where the costs associated with specialized weather stations cannot be justified. To overcome this hurdle, a cost-effective and scalable solution is required. Therefore, a new centralized cloud-based solar conversion recovery system (SCRS) is proposed in this research work. The proposed system utilizes the Internet of Things (IoT) and cloud-based centralized architecture, which allows users to remotely monitor the amount of soiling on PV panels, regardless of the scale. To improve scalability and cost-effectiveness, the proposed system uses low-cost sensors and an artificial neural network (ANN) to reduce the amount of hardware required for a soiling station. Multiple ANN models with different numbers of neurons in hidden layers were tested and compared to determine the most suitable model. The selected ANN model was trained using the data collected from an experimental setup. After training the ANN model, the mean squared error (MSE) value of 0.0117 was achieved. Additionally, the adjusted R-squared ( R 2 ) value of 0.905 was attained on the test data. Furthermore, data is transmitted from soiling station to the cloud server wirelessly using a message queuing telemetry transport (MQTT) lightweight communication protocol over Wi-Fi network. Therefore, SCRS depicts a complete wireless sensor network eliminating the need for extra wiring. The average percentage error in the soiling ratio estimation was found to be 4.33%. [ABSTRACT FROM AUTHOR]

Facial emotion recognition (FER) systems are imperative in recent advanced artificial intelligence (AI) applications to realize better human–computer interactions. Most deep learning-based FER systems have issues with low accuracy and high resource requirements, especially when deployed on edge devices with limited computing resources and memory. To tackle these problems, a lightweight FER system, called Light-FER, is proposed in this paper, which is obtained from the Xception model through model compression. First, pruning is performed during the network training to remove the less important connections within the architecture of Xception. Second, the model is quantized to half-precision format, which could significantly reduce its memory consumption. Third, different deep learning compilers performing several advanced optimization techniques are benchmarked to further accelerate the inference speed of the FER system. Lastly, to experimentally demonstrate the objectives of the proposed system on edge devices, Light-FER is deployed on NVIDIA Jetson Nano. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Harbin University of Science & Technology is the property of Journal of Harbin University of Science & Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

A lot of recent works have shown that deep learning-based visual odometry methods outperform existing feature-based approaches in the monocular case. However, most of them cannot be used in mobile robotics because they require a sufficiently powerful computing device. In this paper, we propose a method with a significant reduction in computing resources and with a slight decrease in accuracy. To achieve that we replaced recurrent block by lightweight transformer-based module. We evaluate the proposed model on the KITTI dataset for calculating accuracy and test a computation cost on NVIDIA Jetson Nano and NVIDIA Jetson AGX Xavier. Our experiments show that the proposed model works faster than the considered original model.

Compared with traditional methods, the action recognition model based on 3D convolutional deep neural network captures spatio-temporal features more accurately, resulting in higher accuracy. However, the large number of parameters and computational requirements of 3D models make it difficult to deploy on mobile devices with limited computing power. In order to achieve an efficient video action recognition model, we have analyzed and compared classic lightweight network principles and proposed the 3D-ShuffleViT network. By deeply integrating the self-attention mechanism with convolution, we have introduced an efficient ACISA module that further enhances the performance of our proposed model. This has resulted in exceptional performance in both context-sensitive and context-independent action recognition, while reducing deployment costs. It is worth noting that our 3D-ShuffleViT network, with a computational cost of only 6% of that of SlowFast-ResNet101, achieved 98% of the latter’s Top1 accuracy on the EgoGesture dataset. Furthermore, on the same CPU (Intel i5-8300H), its speed was 2.5 times that of the latter. In addition, when we deployed our model on edge devices, our proposed network achieved the best balance between accuracy and speed among lightweight networks of the same order.

With the rapid development of Industrial Internet of Things (IIoT) technology, video surveillance devices and video data in IIoT environments are massively growing and increasingly important. Deploying rapidly evolving deep learning-based object detection algorithms in IIoT can improve the efficiency of video data utilization and increase the automation and intelligence of the IIoT. Facing the transmission latency problem of massive video data, the algorithms are better deployed in edge devices. However, due to the large size and high computing power requirements of existing object detection algorithms, it is difficult to deploy them on edge devices with limited performance. Therefore, to solve this problem, we propose a smaller and less computationally intensive object detection algorithm, PG-YOLO. We replaced all the convolutional modules of YOLOV5 with low computational ghost modules and streamlined the backbone network to improve the network performance. Then, we propose a better pruning algorithm to compress the network, and finally improve the accuracy by distillation to obtain PG-YOLO. PG-YOLO is smaller and faster and can be deployed in edge devices. In addition, we conducted experimental validation on the SHWD dataset. The experimental results show that PG-YOLO can compress the volume of the model by 9 times compared with YOLOv5s, and the compressed detection accuracy reaches 0.934, with only 0.1% loss of accuracy. Also, the time required for inference is reduced by 32.7%, and the inference speed is improved by 10 FPS. Compared with other object detection algorithms, PG-YOLO also has advantages.

Advanced Driver Assistance Systems (ADAS) are a collection of intelligent solutions integrated into next-generation vehicles to assist in safe driving. When building ADAS systems, the main goals are that they are stable, flexible, easy to maintain, and allow for error tracing. If a driving assistance algorithm is designed to be implemented on one machine or in one model, there is a potential disadvantage that if one component fails, then the entire system would stop. We work on modularizing the ADAS system to be flexible to accommodate any changes or improvements based on up-to-date requirements. Using advanced current edge (or network) devices, we propose a Detection-based Driving Assistance algorithm, which can collaborate or integrate with an existing system in a vehicle. The core of any process is to ensure that the system has a predictable level of functionality and that any misbehavior can be easily traced to the root cause. The proposed system shows fast, real-time performance on edge devices with limited computing power.

The lens distortion process is essential for displaying VR contents on a head-mounted display (HMD) with a distorted display surface. This paper proposes a novel lens distortion algorithm to achieve real-time performance on edge devices with an embedded GPU. We employ unified memory space to reduce the data transfer overhead based on an architectural characteristic: an integrated CPU and GPU memory system. The lens distortion kernel is based on the lookup table-based mapping algorithm whose performance is bounded by memory operations rather than computations. To improve the kernel’s performance, we propose a compressed lookup table approach that reduces the memory transactions on the kernel while slightly increasing computation. We tested our method on three different edge devices and a desktop system while varying the image resolution from 720p (1, $280\times 720$ ) to 8K (7, $680\times 4.320$ ). Compared with prior GPU-based lookup table algorithms, our method achieved up to 1.72-times higher performance while consuming up to 28.93% less power. Also, our method demonstrates real-time performance for up to a 4K image with a low-end edge device (e.g., 56 FPS on Jetson Nano) and up to an 8K image with a mid-range device (e.g., 94 FPS on Jetson NX). These results demonstrate the benefits of our approach from the perspectives of both performance and energy.

The intrusion of foreign objects in tracks, one of the sources of injuries and fatalities in metro stations, can be solved as an anomaly detection task. However, the existing anomaly detection methods rarely consider the importance of updating the new knowledge from false alarm data, resulting in repeated mistakes. These methods are also impractical for edge devices that cannot afford the high calculation cost. A sequential updatable anomaly detection (SUAD) framework is proposed to tackle these problems. This framework is based on the Robbin-Monro algorithm and a fast version of Mahalanobis distance. A well-trained model of SUAD can continue to learn new knowledge through the sequential knowledge update module based on the Robbin-Monro algorithm without reviewing the old data. SUAD utilizes a new Mahalanobis distance calculation method based on principal component analysis. This new method exhibits a fast inference speed with a lighter model size than before. SUAD is evaluated on a self-built Metro Anomaly Detection (MAD) dataset and three public datasets. SUAD achieves an average area under the receiver operating characteristic curve score of 99.4% at image level and 99.6% at pixel level on MAD. SUAD also reduces at least 78% model size and 60% memory usage. Competitive results are also achieved in public datasets, including MVTec AD, beanTech Anomaly Detection, and CIFAR-10. [ABSTRACT FROM AUTHOR]

In the field of computer vision technology, deep learning of image processing has become an emerging research area. The semantic segmentation of an image is among the utmost essential and significant tasks in image-processing research, offering a wide range of application fields such as autonomous driving systems, medical diagnosis, surveillance security, etc. Thus far, many studies have suggested and developed neural network modules in deep learning. To the best of our knowledge, all existing neural networks for semantic segmentation have large parameter sizes and it is therefore unfeasible to implement those architectures in low-power and memory-limited embedded platforms such as FPGAs. Building an embedded platform with that architecture is possible after reducing the parameter size without affecting the module's architecture. The quantization technique lowers the precision of the neural network parameters while mostly keeping the accuracy. In this paper, we propose a quantization algorithm for a semantic segmentation deep learning architecture, which reduces the parameter size by four to eight times with a negligible accuracy abatement. As long as the parameter size is reduced, the deep learning architecture is improved in terms of required storage, computational speed, and power efficiency. [ABSTRACT FROM AUTHOR]

Within the scope of precision agriculture, many applications have been developed to support decision making and yield enhancement. Fruit detection has attracted considerable attention from researchers, and it can be used offline. In contrast, some applications, such as robot vision in orchards, require computer vision models to run on edge devices while performing inferences at high speed. In this area, most modern applications use an integrated graphics processing unit (GPU). In this work, we propose the use of a tensor processing unit (TPU) accelerator with a Raspberry Pi target device and the state-of-the-art, lightweight, and hardware-aware MobileDet detector model. Our contribution is the extension of the possibilities of using accelerators (the TPU) for edge devices in precision agriculture. The proposed method was evaluated using a novel dataset of peaches with three cultivars, which will be made available for further studies. The model achieved an average precision (AP) of 88.2% and a performance of 19.84 frames per second (FPS) at an image size of 640 × 480. The results obtained show that the TPU accelerator can be an excellent alternative for processing on the edge in precision agriculture. [ABSTRACT FROM AUTHOR]

The development of deep neural network (DNN) has provided transformative impacts on many fields, including computer vision and video recognition. However, the impact is limited by the need for large, labeled datasets to enable effective training. To address this fundamental problem, we propose a novel inter-network system (Dandelion), providing architecture support (Dandelion-architecture) for data augmentation that trains DNNs with rare images generated by the generative adversarial network (GAN) with orthogonal attributes modified (Dandelion-function. The approach can account for the latency requirement and resource limitation of target applications by exploiting data and computation reuses between the two networks; this amortizes the impact of bottleneck brought by GAN and facilitates design of inter-network accelerator. Moreover, we show how to implement two-network design on 3D architecture to further enhance the accelerator. Our results show that with the generated images, DNN yields 13.6% - 37.5% improvement on accuracy, depending on the data scarcity level. Our architecture achieves at least 30% speedup compared with the baseline while 40% of the overhead brought by the incorporation of GAN is reduced in our design compared with ScaleDeep, and 26.3% of performance improvement over TETRIS. [ABSTRACT FROM AUTHOR]

Portable devices play an essential role where edge computing is necessary and mobility is required (e.g., robots in agriculture within remote-sensing applications). With the increasing applications of deep neural networks (DNNs) and accelerators for edge devices, several methods and applications have been proposed for simultaneous crop and weed detection. Although preliminary studies have investigated the performance of inference time for semantic segmentation of crops and weeds in edge devices, performance degradation has not been evaluated in detail when the required optimization is applied to the model for operation in such edge devices. This paper investigates the relationship between model tuning hyperparameters to improve inference time and its effect on segmentation performance. The study was conducted using semantic segmentation model DeeplabV3 with a MobileNet backbone. Different datasets (Cityscapes, PASCAL and ADE20K) were analyzed for a transfer learning strategy. The results show that, when using a model hyperparameter depth multiplier (DM) of 0.5 and the TensorRT framework, segmentation performance mean intersection over union (mIOU) decreased by 14.7% compared to that of a DM of 1.0 and no TensorRT. However, inference time accelerated dramatically by a factor of 14.8. At an image resolution of 1296 × 966 , segmentation performance of 64% mIOU and inference of 5.9 frames per second (FPS) was achieved in Jetson Nano's device. With an input image resolution of 513 × 513 , and hyperparameters output stride OS = 32 and DM = 0.5, an inference time of 0.04 s was achieved resulting in 25 FPS. The results presented in this paper provide a deeper insight into how the performance of the semantic segmentation model of crops and weeds degrades when optimization is applied to adapt the model to run on edge devices. Lastly, an application is described for the semantic segmentation of weeds embedded in the edge device (Jetson Nano) and integrated with the robotic orchard. The results show good spraying accuracy and feasibility of the method. [ABSTRACT FROM AUTHOR]