Search

Smart agriculture is an enabling technology addressing the increasing challenges of efficiency, sustainability, and quality of food production. It requires rich data from the farming area at high spatial and temporal resolution. Although remote sensing systems have become readily available recently, in-situ sensing is still required to capture important properties of soil, crops, and their close environment. Ubiquitous sensor networks (USNs) provide a seamless and real-time in-situ sensing infrastructure that could overcome some limitations of smart agriculture. Resource efficiency is essential for USNs due to 1) the expected long operation time in typically resource-constrained environments, 2) the vast amount of captured and processed data, and 3) the ever-increasing application requirements. This survey comprehensively analyzes resource optimization techniques for USNs along three USN layers: the sensing, the communication & connectivity, and the processing & analysis layer. It discusses the application of these techniques in the smart agriculture domain and identifies current challenges and open research issues.

Text-based person re-identification (re-ID) is an emerging research domain in multi-camera surveillance systems. It involves identifying individuals across different camera views using textual descriptions as queries. Although text-based person re-ID holds significant potential for surveillance systems, its practical applications are limited by the high computational demands of existing algorithms. This is because most state-of-the-art algorithms prioritize identification accuracy over resource efficiency. Therefore, surveillance systems based on existing algorithms rely heavily on centralized architectures, where a central application server aggregates videos from camera nodes, generates galleries from the videos, and performs person identification. However, such centralized systems face challenges in large-scale deployments, including scalability issues, high bandwidth utilization, and processing bottlenecks. This paper presents a decentralized approach to text-based person re-ID to overcome these limitations. First, it introduces U-TextReIDNet, a resource-efficient model designed to identify persons in multi-person images on the NVIDIA Jetson Nano embedded board. U-TextReIDNet achieves Top-1 accuracies of 54.02% on the CUHK-PEDES dataset and 38.45% on the RSTPReid dataset. With only 38.77 million parameters, U-TextReIDNet is significantly smaller than most existing text-based person re-ID models. Using U-TextReIDNet, we implement a decentralized system that distributes the person identification task across camera nodes, transmitting only videos containing persons of interest to the command station. Additionally, we developed a prototype of this decentralized system, and conducted performance and usability tests using real human subjects. The prototype successfully performs real-time person re-ID, reduces bandwidth utilization, improves system scalability, and eliminates processing bottlenecks.

Existing software tools for specifying and executing multidrone missions are limited to route planning or tightly coupled to specific drone hardware. We introduce EAMOS (Execution of Aerial Multidrone Missions and Operations Specification Framework), which allows us to specify missions intuitively, text-based, and provides a mission compiler, a mission middle layer, and a distributed drone execution environment. The middle layer wraps the control of individual drone-specific capabilities, such as launch, fly to position, or perform a maneuver, into a public API that transparently utilizes the capabilities of numerous drone platforms. We exploit the Go programming language to implement critical components of the framework and provide an interface for ROS-based drone platforms. EAMOS automates the mission execution on real, virtual, and even hybrid robotic setups involving real and virtual drones. We demonstrate the successful deployment of EAMOS with four missions executed on Pixhawk/PX4-equipped quadcopters and virtual drones simulated with Airsim. We assess the performance of our proposed approach by analyzing the number of nodes and arcs of the mission graphs, which are an essential artifact of our mission compilation, the utilization of ROS service calls during mission execution, and the duration of compilation, deployment, and mission execution. Overall, our experiments showed that our drones correctly behaved during mission execution as expected and specified by their mission, the generated mission artifacts were efficiently manageable, and processing times allowed for a fluent workflow.

In many missions, multiple robots need to cooperate to complete tasks. The workflow of these multi-robot tasks involves forming coalitions of robots, assigning them to available tasks, and jointly executing the tasks. In this article, we investigate such a workflow in an application-independent yet realistic setting. We abstract the key properties of tasks and robots and propose three distributed coalition formation and task assignment methods. Distributed methods rely on a timely and accurate exchange of state information in multi-robot systems (MRS). Thus we focus on two important communication aspects: (i) how to achieve consistent coalition formation and task assignment in the presence of communication faults, and (ii) how to reduce the communication effort required for the state updates in the MRS. In particular, we investigate the effect of event-triggered, time-triggered, and hybrid communication. We evaluate our distributed approaches in a simulation study using ns-3 and compare them with centralized methods and different network conditions. We demonstrate the sensitivity of complex missions to failure-prone MRS communication and provide robust, effective, and communication-aware methods for coalition formation and task assignment.

Image data contain spatial information only, thus making two-dimensional (2D) Convolutional Neural Networks (CNN) ideal for solving image classification problems. On the other hand, video data contain both spatial and temporal information that must be simultaneously analyzed to solve action recognition problems. 3D CNNs are successfully used for these tasks, but they suffer from their extensive inherent parameter set. Increasing the network’s depth, as is common among 2D CNNs, and hence increasing the number of trainable parameters does not provide a good trade-off between accuracy and complexity of the 3D CNN. In this work, we propose Pooling Block (PB) as an enhanced pooling operation for optimizing action recognition by 3D CNNs. PB comprises three kernels of different sizes. The three kernels simultaneously sub-sample feature maps, and the outputs are concatenated into a single output vector. We compare our approach with three benchmark 3D CNNs (C3D, I3D, and Asymmetric 3D CNN) and three datasets (HMDB51, UCF101, and Kinetics 400). Our PB method yields significant improvement in 3D CNN performance with a comparatively small increase in the number of trainable parameters. We further investigate (1) the effect of video frame dimension and (2) the effect of the number of video frames on the performance of 3D CNNs using C3D as the benchmark.

Mobile robots, especially unmanned aerial vehicles (UAVs), are of increasing interest for surveillance and disaster response scenarios. We consider the problem of multi-robot persistent surveillance with connectivity constraints where robots have to visit sensing locations periodically and maintain a multi-hop connection to a base station. We formally define several problem instances closely related to multi-robot persistent surveillance with connectivity constraints, i.e. connectivity-constrained multi-robot persistent surveillance (CMPS), connectivity-constrained multi-robot reachability (CMR), and connectivity-constrained multi-robot reachability with relay dropping (CMRD), and show that they are all NP-hard on general graphs. We introduce three heuristics with different planning horizons for convex grid graphs and combine them with a tree traversal approach, which can be applied to a partitioning of non-convex grid graphs (CMPS with tree traversal, CMPSTT). In simulation studies we show that a short horizon greedy approach, which requires parameters to be optimized beforehand, can outperform a full horizon approach, which requires a tour through all sensing locations, if the number of robots is larger than the minimum number of robots required to reach all sensing locations. The minimum number required is the number of robots necessary for building a relay chain to the farthest sensing location from the base station. Furthermore, we show that partitioning the area and applying the tree traversal approach can achieve a similar performance to the unpartitioned case up to a certain number of robots but requires less optimization time.

Photographs taken in public places often contain faces of bystanders thus leading to a perceived or actual violation of privacy. To address this issue, we propose to pseudo-randomly modify the appearance of face regions in the images using a privacy filter that prevents a human or a face recogniser from inferring the identity of people. The filter, which is applied only when the resolution is high enough for a face to be recognisable, adaptively distorts the face appearance as a function of its resolution. Moreover, the proposed filter locally changes the values of its parameters to counter attacks that attempt to estimate them. The filter exploits both global adaptiveness to reduce distortion and local parameter hopping to make their estimation difficult for an attacker. In order to evaluate the efficiency of the proposed approach, we consider an important scenario of oblique face images: photographs taken with low altitude Micro Aerial Vehicles (MAVs). We use a state-of-the-art face recognition algorithm and synthetically generated face data with 3D geometric image transformations that mimic faces captured from an MAV at different heights and pitch angles. Experimental results show that the proposed filter protects privacy while reducing distortion, and is also robust against attacks.

Crowd surveillance plays a key role to ensure safety and security in public areas. Surveillance systems traditionally rely on fixed camera networks, which suffer from limitations, as coverage of the monitored area, video resolution and analytic performance. On the other hand, a smart camera network provides the ability to reconfigure the sensing infrastructure by incorporating active devices such as pan-tilt-zoom (PTZ) cameras and UAV-based cameras, thus enabling the network to adapt over time to changes in the scene. We propose a new decentralised approach for network reconfiguration, where each camera dynamically adapts its parameters and position to optimise scene coverage. Two policies for decentralised camera reconfiguration are presented: a greedy approach and a reinforcement learning approach. In both cases, cameras are able to locally control the state of their neighbourhood and dynamically adjust their position and PTZ parameters. When crowds are present, the network balances between global coverage of the entire scene and high resolution for the crowded areas. We evaluate our approach in a simulated environment monitored with fixed, PTZ and UAV-based cameras.

As manufacturing demographics change from mass production to mass customization, advances in human-robot interaction in industries have taken many forms. However, the topic of reducing the programming effort required by an expert using natural modes of communication is still open. To answer this challenge, we propose an approach based on Interactive Reinforcement Learning that learns a complete collaborative assembly process. The learning approach is done in two steps. First step consists of modeling simple tasks that compose the assembly process, using task based formalism. The robotic system then uses these modeled simple tasks and proposes to the user a set of possible actions at each step of the assembly process via a GUI. The user then “interacts” with the robotic system by selecting an option from the given choice. The robot records the action chosen and performs it, progressing the assembly process. Thereby, the user teaches the system which task to perform when. In order to reduce the number of actions proposed, the system considers additional information such as user and robot capabilities and object affordances. These set of action proposals are further reduced by modeling the proposed actions into a goal based hierarchy and by including action prerequisites. The learning framework highlights its ability to learn a complicated human robot collaborative assembly process in a user intuitive fashion. The framework also allows different users to teach different assembly processes to the robot.

Smart cameras are key sensors in Internet of Things (IoT) applications and often capture highly sensitive information. Therefore, security and privacy protection is a key concern. This paper introduces a lightweight security approach for smart camera IoT applications based on elliptic-curve (EC) signcryption that performs data signing and encryption in a single step. We deploy signcryption to efficiently protect sensitive data onboard the cameras and secure the data transfer from multiple cameras to multiple monitoring devices. Our multi-sender/multi-receiver approach provides integrity, authenticity, and confidentiality of data with decryption fairness for multiple receivers throughout the entire lifetime of the data. It further provides public verifiability and forward secrecy of data. Our certificateless multi-receiver aggregate-signcryption protection has been implemented for a smart camera IoT scenario, and the runtime and communication effort has been compared with single-sender/single-receiver and multi-sender/single-receiver setups.

Wireless sensor networks (WSNs) are an attractive platform for monitoring and measuring physical phenomena. WSNs typically consist of hundreds or thousands of battery-operated tiny sensor nodes which are connected via a low data rate wireless network. A WSN application, such as object tracking or environmental monitoring, is composed of individual tasks which must be scheduled on each node. Naturally the order of task execution influences the performance of the WSN application. Scheduling the tasks such that the performance is increased while the energy consumption remains low is a key challenge. In this paper we apply online learning to task scheduling in order to explore the tradeoff between performance and energy consumption. This helps to dynamically identify effective scheduling policies for the sensor nodes. The energy consumption for computation and communication is represented by a parameter for each application task. We compare resource-aware task scheduling based on three online learning methods: independent reinforcement learning (RL), cooperative reinforcement learning (CRL), and exponential weight for exploration and exploitation (Exp3). Our evaluation is based on the performance and energy consumption of a prototypical target tracking application. We further determine the communication overhead and computational effort of these methods.