Search

Integration of soft electronic system components is critical to create a closed‐loop system that seamlessly integrates sensing, driving, processing, and autonomous control capabilities. However, the existence of these components, particularly sensors and actuators, in isolated or discrete forms complicates the endeavor to achieve seamless interface matching and in situ integration, no more than the autonomic system. Here, an intelligent soft quasi‐organism (SQO) equipped with sensor‐driven integrated tentacles is demonstrated, featuring real‐time state recognition and autonomous object recognition inspired by the sea anemones. By employing a heterogeneous mechanisms homotopic integration (HMHI) strategy, the tentacles of the SQO possess the unique ability to simultaneously perceive state changes and flexibly drive motions, utilizing the same material and structure. The sea anemone‐shaped SQO exhibits real‐time autonomous state identification capabilities through the integration of machine learning and customized circuits, attaining 100% recognition accuracy across sixteen states. With a neuromuscular system that facilitates active autonomous perception, the anemone‐shaped SQO can recognize and intelligently grasp static objects with an accuracy of 80.7%, surpassing that of a human hand (74.7%). The SQO provides a promising approach for realizing artificial neuromuscular systems, with great potential for applications in crucial areas such as intelligent soft robotics and in vivo therapy. [ABSTRACT FROM AUTHOR]

Public spaces in traditional villages are the root of the development, preservation, and inheritance of rural cultural heritage. However, these spaces in tourism-oriented traditional villages continue to suffer from issues such as inconsistent vitality, more competition than cooperation, and inefficient revitalisation. Limited by the low precision of spatiotemporal data in rural areas, previous static and materialised vitality evaluation methods have failed to explore the dynamic nature of the flow of people in traditional village public spaces. This study investigated a collaborative active perception method using Wi-Fi probes and static snapshots to assess their dynamic vitality. An empirical study was conducted using Yantou Village in Lishui City, China, as an example. The empirical results indicate that this method can qualitatively and quantitatively analyze the spatiotemporal variations and the scenarios of the dynamic vitality of small public spaces in traditional villages, deepening the cognitive depth of their vitality from time and humanity, providing a basis for the spatiotemporal synergistic enhancement of their public spaces, and offering insights for their revitalisation. [ABSTRACT FROM AUTHOR]

Robots in tomato greenhouses need to perceive the plant and plant parts accurately to automate monitoring, harvesting, and de-leafing tasks. Existing perception systems struggle with the high levels of occlusion in plants and often result in poor perception accuracy. One reason for this is because they use fixed cameras or predefined camera movements. Next-best-view (NBV) planning presents an alternate approach, in which the camera viewpoints are reasoned and strategically planned such that the perception accuracy is improved. However, existing NBV-planning algorithms are agnostic to the task-at-hand and give equal importance to all the plant parts. This strategy is inefficient for greenhouse tasks that require targeted perception of specific plant parts, such as the perception of leaf nodes for de-leafing. To improve targeted perception in complex greenhouse environments, NBV planning algorithms need an attention mechanism to focus on the task-relevant plant parts. In this paper, the role of attention in improving targeted perception using an attention-driven NBV planning strategy was investigated. Through simulation experiments using plants with high levels of occlusion and structural complexity, it was shown that focusing attention on task-relevant plant parts can significantly improve the speed and accuracy of 3D reconstruction. Further, with real-world experiments, it was shown that these benefits extend to complex greenhouse conditions with natural variation and occlusion, natural illumination, sensor noise, and uncertainty in camera poses. The results clearly indicate that using attention-driven NBV planning in greenhouses can significantly improve the efficiency of perception and enhance the performance of robotic systems in greenhouse crop production. • Method to plan the next-best viewpoint to reconstruct the nodes of tomato plants. • Adding attention mechanism improves the efficiency of next-best-view planning. • Performed systematic evaluations with both simulation and real-world tomato plants. • Effectively handles occlusion in complex greenhouse environments. [ABSTRACT FROM AUTHOR]

In this study, an artificial intelligence framework is developed to facilitate the use of robotics for autonomous damage inspection. While considerable progress has been achieved by utilizing state-of-the-art computer vision approaches for damage detection, these approaches are still far away from being used for autonomous robotic inspection systems due to the uncertainties in data collection and data interpretation. To address this gap, this study proposes a framework that will enable robots to select the best course of action for active damage perception and reduction of uncertainties. By doing so, the required information is collected efficiently for a better understanding of damage severity which leads to reliable decision-making. More specifically, the active damage perception task is formulated as a Partially Observable Markov Decision Process, and a deep reinforcement learning-based active perception agent is proposed to learn the near-optimal policy for this task. The proposed framework is evaluated for the autonomous assessment of cracks on metallic surfaces of an underwater nuclear reactor. Active perception exhibits a notable enhancement in the crack Intersection over Union (IoU) performance, yielding an increase of up to 69% when compared to its raster scanning counterpart given a similar inspection time. Additionally, the proposed method can perform a rapid inspection that reduces the overall inspection time by more than two times while achieving a 15% higher crack IoU than that of the dense raster scanning approach. [ABSTRACT FROM AUTHOR]

Abstract Public spaces in traditional villages are the root of the development, preservation, and inheritance of rural cultural heritage. However, these spaces in tourism-oriented traditional villages continue to suffer from issues such as inconsistent vitality, more competition than cooperation, and inefficient revitalisation. Limited by the low precision of spatiotemporal data in rural areas, previous static and materialised vitality evaluation methods have failed to explore the dynamic nature of the flow of people in traditional village public spaces. This study investigated a collaborative active perception method using Wi-Fi probes and static snapshots to assess their dynamic vitality. An empirical study was conducted using Yantou Village in Lishui City, China, as an example. The empirical results indicate that this method can qualitatively and quantitatively analyze the spatiotemporal variations and the scenarios of the dynamic vitality of small public spaces in traditional villages, deepening the cognitive depth of their vitality from time and humanity, providing a basis for the spatiotemporal synergistic enhancement of their public spaces, and offering insights for their revitalisation.

Featured Application: The gained knowledge can be applied by road marking material manufacturers and automotive proving ground operators to provide road marking scenarios on test tracks to test perception systems that work with camera or LiDAR sensors. The investigated method is especially useful for urban scenarios with road markings that have complex geometric shapes, such as small radii, symbols, lettering, and specific corner shapes for parking slots. In order to test camera- and LiDAR-based perception of road markings for automated driving, vehicle developers have started to utilize concepts for the agile alteration of road marking patterns on proving grounds. Road marking materials commonly used within this concept are different types of tape that can easily be applied and removed on asphalt and concrete. Due to the elasticity of tape, it cannot be used efficiently for small radii, symbols, lettering, and specific corner shapes (e.g., for parking slots). These road marking patterns are common in urban environments. With the growing capability of automated driving systems and more applications for urban environments, edgy road marking shapes gain importance for proving ground testing. This study examines the use of water-soluble road marking paint specifically designed for the use case of temporary applications on proving grounds for camera- and LiDAR-based perception testing. We found that white, water-soluble paint with 1.5% binder content and 2.25% coalescing agent content can provide realistic road markings for vehicle testing purposes. However, solubility affects the paint's vulnerability to fading during rain. Hence, renewal activities over the course of longer test drives might be necessary. The paint could be removed using water pressure without significant residue or damaging of the asphalt. [ABSTRACT FROM AUTHOR]

Performing tasks in agriculture, such as fruit monitoring or harvesting, requires perceiving the objects' spatial position. RGB-D cameras are limited under open-field environments due to lightning interferences. So, in this study, we state to answer the research question: "How can we use and control monocular sensors to perceive objects' position in the 3D task space?" Towards this aim, we approached histogram filters (Bayesian discrete filters) to estimate the position of tomatoes in the tomato plant through the algorithm MonoVisual3DFilter. Two kernel filters were studied: the square kernel and the Gaussian kernel. The implemented algorithm was essayed in simulation, with and without Gaussian noise and random noise, and in a testbed at laboratory conditions. The algorithm reported a mean absolute error lower than 10 mm in simulation and 20 mm in the testbed at laboratory conditions with an assessing distance of about 0.5 m. So, the results are viable for real environments and should be improved at closer distances. [ABSTRACT FROM AUTHOR]

According to a common view, olfactory experiences lack well-developed spatial content. Nevertheless, there is also an important opposition to such a restricted perspective on olfactory spatiality, which claims that a view ascribing only rudimentary spatial content to olfaction arises from a narrow focus on short and passive olfactory experiences. In particular, it is claimed that due to the active and diachronic aspects of olfaction, olfactory experiences represent ‘smellscapes,’ i.e., spatially organized arrangements of odor plumes. This paper considers the thesis that olfaction represents smellscapes by distinguishing weaker and stronger understandings of smellscapes. Weak smellscapes are odors standing in allocentric spatial relations, while strong smellscapes, in addition, are odors located at places having specific sizes and shapes. It is argued that only weak smellscapes are plausibly represented by human olfaction. [ABSTRACT FROM AUTHOR]

Inferential decision-making algorithms typically assume that an underlying probabilistic model of decision alternatives and outcomes may be learned a priori or online. Furthermore, when applied to robots in real-world settings they often perform unsatisfactorily or fail to accomplish the necessary tasks because this assumption is violated and/or because they experience unanticipated external pressures and constraints. Cognitive studies presented in this and other papers show that humans cope with complex and unknown settings by modulating between near-optimal and satisficing solutions, including heuristics, by leveraging information value of available environmental cues that are possibly redundant. Using the benchmark inferential decision problem known as “treasure hunt”, this paper develops a general approach for investigating and modeling active perception solutions under pressure. By simulating treasure hunt problems in virtual worlds, our approach learns generalizable strategies from high performers that, when applied to robots, allow them to modulate between optimal and heuristic solutions on the basis of external pressures and probabilistic models, if and when available. The result is a suite of active perception algorithms for camera-equipped robots that outperform treasure-hunt solutions obtained via cell decomposition, information roadmap, and information potential algorithms, in both high-fidelity numerical simulations and physical experiments. The effectiveness of the new active perception strategies is demonstrated under a broad range of unanticipated conditions that cause existing algorithms to fail to complete the search for treasures, such as unmodelled time constraints, resource constraints, and adverse weather (fog).

Exploring the complex meanings in architectural scenic images is crucial for modern intelligent systems. We propose a novel method combining multi-channel perceptual features to semantically understand scenes with intricate spatial structures. Our deep hierarchical model captures human gaze dynamics using a binary objectness measure to detect objects and details at various architectural scales. We introduce a multi-layer active learning paradigm to derive gaze shifting paths (GSPs) and calculate corresponding deep representations, tolerant to label noise via a penalty term that discards irrelevant GSP features. A manifold-regularized feature selector is used to choose high-quality GSP features, and a linear classifier distinguishes architectural scenes. Our technique outperforms traditional models, as shown in tests on three image retrieval datasets, demonstrating the discriminative capacity of our deep gaze-guided features.

In this article, we address a form of active perception characterized by curiosity-driven, open-ended exploration with intrinsic motivation, carried out by a group of agents. The multiple agents and a large number of possible actions are the main motivation for incorporating Multi-Agent Reinforcement Learning used to train a neural network in order to derive agent’s policy. Partially Observable Markov Decision Process framework is used to accommodate the inaccuracy of sensors and probabilistic nature of agent’s actions. The proposed method incorporates a consensus that derives the common belief vector, thus allowing each agent to make its decisions based on information acquired by all agents involved in the process of active perception. A well-known benchmark problem with a decentralized tiger scenario was used to demonstrate the possibility of the method to generate agents with different perceptual characteristics by simply changing the agents’ reward function related to their intrinsic motivation. The main validation of the proposed approach was performed by using an example of multi-agent search mission. Final results are presented and discussed, and possible avenues for progress on open problems are identified.

In order to test camera- and LiDAR-based perception of road markings for automated driving, vehicle developers have started to utilize concepts for the agile alteration of road marking patterns on proving grounds. Road marking materials commonly used within this concept are different types of tape that can easily be applied and removed on asphalt and concrete. Due to the elasticity of tape, it cannot be used efficiently for small radii, symbols, lettering, and specific corner shapes (e.g., for parking slots). These road marking patterns are common in urban environments. With the growing capability of automated driving systems and more applications for urban environments, edgy road marking shapes gain importance for proving ground testing. This study examines the use of water-soluble road marking paint specifically designed for the use case of temporary applications on proving grounds for camera- and LiDAR-based perception testing. We found that white, water-soluble paint with 1.5% binder content and 2.25% coalescing agent content can provide realistic road markings for vehicle testing purposes. However, solubility affects the paint’s vulnerability to fading during rain. Hence, renewal activities over the course of longer test drives might be necessary. The paint could be removed using water pressure without significant residue or damaging of the asphalt.

Although the relationship between prejudice and predictive coding has attracted more attention recently, many important issues remain to be investigated, such as why prejudice is so persistent and how to accommodate seemingly conflicting studies. In this paper, we offer an integrated framework to explain the functional-computational mechanism of prejudice. We argue that this framework better explains (i) why prejudice is somewhat immune to revision, (ii) how inconsistent processing (e.g. one's moral belief and biased emotional reaction) may occur, (iii) the dispute over the content of 'alief' (unconscious belief), and (iv) conflicting data reported in disinformation studies. Finally, we briefly review the main intervention strategies and offer recommendations. [ABSTRACT FROM AUTHOR]

Copyright of Telecommunication Engineering is the property of Telecommunication Engineering and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Visual-language navigation (VLN) is the task of entailing an agent to carry out navigational instructions inside photo-realistic environments. One of the key challenges in VLN is how to conduct robust navigation by mitigating the uncertainty caused by ambiguous instructions and insufficient observation of the environment. Agents trained by current approaches typically suffer from this and would consequently struggle to take navigation actions at every step. In contrast, when humans face such a challenge, we can still maintain robust navigation by actively exploring the surroundings to gather more information and thus make a more confident navigation decision. This work draws inspiration from human navigation behavior and endows an agent with an active perception ability for more intelligent navigation. To achieve this, we propose an end-to-end framework for learning an exploration policy that decides (i) when and where to explore, (ii) what information is worth gathering during exploration, and (iii) how to adjust the navigation decision after the exploration. In this way, the agent is able to turn its past experiences as well as new explored knowledge to contexts for more confident navigation decision making. In addition, an external memory is used to explicitly store the visited visual environments and thus allows the agent to adopt a late action-taking strategy to avoid duplicate exploration and navigation movements. Our experimental results on two standard benchmark datasets show promising exploration strategies emerged from training, which leads to significant boost in navigation performance. [ABSTRACT FROM AUTHOR]

Miniature jumping robots (MJRs) have difficulty executing autonomous movements in unstructured environments with obstacles because of their limited perception and computing resources. This study investigates the obstacle detection and autonomous stair climbing methods for MJRs. We propose an obstacle detection method based on a combination of attitude and distance detections, as well as MJRs’ motion. A MEMS inertial sensor collects the yaw angle of the robot, and a ranging sensor senses the distance between the robot and the obstacle to estimate the size of the obstacle. We also propose an autonomous stair climbing algorithm based on the obstacle detection method. The robot can detect the height and width of stairs and its position relative to the stairs and then repeatedly jump to climb them step by step. Moreover, the height, width, and position are sent to a control terminal through a wireless sensor network to update the information regarding the MJR and stairs in a control interface. Furthermore, we conduct stair detection, modeling, and stair climbing experiments on the MJR and obtain acceptable precisions for autonomous obstacle negotiation. Thus, the proposed obstacle detection and stair climbing methods can enhance the locomotion capability of the MJR in environmental monitoring, search and rescue, etc.

This paper concerns online learning of terrain properties combining haptic perception with exteroceptive sensing to reason about forces needed to pass through terrains that visually appear as untraversable obstacles. Terrain learning is studied within the context of autonomous exploration. We propose predicting the traversability of potentially obstructing terrains by active perception to establish a connection between the observed geometric environment model and deliberately sampled forces to pass through the terrain using a haptic sensor that probes the terrain in front of the robot. The developed solution uses a Gaussian Process regressor in online learning and force prediction. The robot is navigated by following the information gain to improve traversability and spatial models. The proposed approach has been experimentally verified in fully autonomous exploration with a multi-legged walking robot. The robot is navigated through visually looking obstacles and explores "hidden" areas while following the expected information gain to explore the terrain properties of the mission area. [ABSTRACT FROM AUTHOR]

This paper presents a Transformer-based Visual Exploration Network (TVENet) that capably serves as a solution for active perception problems, especially the visual exploration problem: How could a robot that is equipped with a camera explore an unknown 3D environment? The TVENet consists of a Mapper, a Global Policy and a Local Policy. The mapper is trained by supervised learning to take the visual observation as input and generate an occupancy grid map for the explored environment. The Global Policy and the Local Policy are trained by reinforcement learning in order to make navigation decision. Most state-of-the-art methods in visual exploration domain use ResNet as feature extractor, and few of them pay attention to the extraction capability of the extractor. Therefore, this paper focuses on enhancing the extraction capability, and proposes a Transformer-based Feature Pyramid Module (TFPM). Moreover, two tricks for training process are introduced to improve the performance (M.F. and Aux.) Our experiments in photo-realistic simulated environment (Habitat) demonstrate the higher-accuracy mapping of TVENet. Experimental results prove that the TFPM and tricks have positive impacts on the mapping accuracy of the visual exploration and increase it by 5.31% compared with the state-of-the-art. Most importantly, the TVENet is deployed on a real robot (NVIDIA Jetbot) to prove the feasibility of Embodied AI approaches. To the authors’ knowledge, this paper is the first one that proves the viability of the Embodied AI style approach for visual exploration tasks and deploys the pre-trained model on the NVIDIA Jetson robot.

As the operational domain of autonomous vehicles expands, encountering occlusions during navigation becomes unavoidable. Most of the existing research on occlusion-aware motion planning focuses only on the longitudinal motion of the ego vehicle and neglects its lateral motion, resulting in output motion that can be overly conservative. This paper proposes a motion planner capable of actively adjusting the ego vehicle’s lateral position to minimize occlusions. The proposed planner is applicable in various scenarios and can function under perception uncertainty. This work also extends our previously proposed 3D visibility estimation approach for addressing occlusions caused by objects which are not present in HD maps. The proposed planner first generates candidate trajectories. The current and future visibility of each trajectory is then estimated using live LiDAR data and HD maps. These estimated visibilities are converted into visibility costs, which are then used to determine the optimal output trajectory in conjunction with other planning costs. The proposed planner is tested in three scenarios using the CARLA simulator: an occluded T-junction crossing, turning at a low-visibility corner and preparing to pass a parked vehicle, using live localization and object detection results. The experimental results reveal that the proposed planner allows the ego vehicle to minimize occlusions by diverging from the center of the lane and, consequently, to discover occluded vehicles earlier than a baseline planner in most situations. Moreover, occlusions caused by a parked vehicle not present in the HD maps were estimated correctly using our extended visibility estimation method.

[Introduction] Aiming at the current problems of the continuous update of the security requirements and low efficiency and uncontrollable effects of security rectification on-site operations for the power monitoring system, and the status quo of large area off-grid operation of some secondary equipment without effective rectification methods, secondary equipment centralized security operation and maintenance and key technologies of active perception, diagnosis and early warning are studied. [Method] Compliant grid-connected devices were unified deployed in substations, the technologies of deep belief nets multi-class support vector machine intrusion detection and adaptive probabilistic marking scheme IP traceback were used to filter recording data, active percept and retrospective diagnosis invaded threats, the risk early warning and comprehensive assessment technology based on combining with threat and adversarial analysis were adopted to carry out impact assessment and rating, and the existing recording master station system was used to expand the security operation and maintenance management business to form centralized operation and maintenance management model. [Result] Practical results show that this method can achieve the goals that the security system of substation secondary equipment is centralized managed and efficiency of operation and maintenance is higher. [Conclusion] The proposed method is effective and feasible to solve the problem of on-site security rectification of secondary equipment combined with the Guangxi power grid operation case.

In everyday life, sound localization entails more than just the extraction and processing of auditory cues. When determining sound position in three dimensions, the brain also considers the available visual information (e.g., visual cues to sound position) and resolves perceptual ambiguities through active listening behavior (e.g., spontaneous head movements while listening). Here, we examined to what extent spontaneous head movements improve sound localization in 3D—azimuth, elevation, and depth—by comparing static vs. active listening postures. To this aim, we developed a novel approach to sound localization based on sounds delivered in the environment, brought into alignment thanks to a VR system. Our system proved effective for the delivery of sounds at predetermined and repeatable positions in 3D space, without imposing a physically constrained posture, and with minimal training. In addition, it allowed measuring participant behavior (hand, head and eye position) in real time. We report that active listening improved 3D sound localization, primarily by ameliorating accuracy and variability of responses in azimuth and elevation. The more participants made spontaneous head movements, the better was their 3D sound localization performance. Thus, we provide proof of concept of a novel approach to the study of spatial hearing, with potentials for clinical and industrial applications. [ABSTRACT FROM AUTHOR]

Inertial navigation for autonomous underwater vehicles (AUVs) is challenging because of the drift error caused by the noise and measurement errors of inertial sensors, typically packaged as an inertial measurement unit (IMU), integrated over time. To mitigate the drift error, recent AUV state estimation approaches incorporate external references or environmental information obtained from exteroceptive sensors, with increased costs and limited operational domains. For improved navigation under sensor constraints, this article proposes an active perception framework that exploits vehicle motion to estimate the flow state together with the vehicle state using IMU and depth sensors only. The proposed framework uses the estimated flow state as external information to improve vehicle state estimation. We construct a linear time-varying system for the flow state, separated from a nonlinear system for the vehicle state. This formulation allows us to analyze uniform complete observability for the flow state, which is found to depend on vehicle motion. Then, along with vehicle and flow state estimators, we design a vehicle controller to enable vehicle motion to maximize an information metric pertaining to estimation performance based on either observability or constructability Gramian for the flow state. The proposed framework is validated through simulations for a case study with a vehicle descending through the water column in a time-varying flow field. The effectiveness of the framework is demonstrated by comparing results obtained from its four implementations with those from baseline approaches without active perception. [ABSTRACT FROM AUTHOR]

When performing visual servoing or object tracking tasks, active sensor planning is essential to keep targets in sight or to relocate them when missing. In particular, when dealing with a known target missing from the sensor's field of view, we propose using prior knowledge related to contextual information to estimate its possible location. To this end, this study proposes a Dynamic Bayesian Network that uses contextual information to effectively search for targets. Monte Carlo particle filtering is employed to approximate the posterior probability of the target's state, from which uncertainty is defined. We define the robot's utility function via information theoretic formalism as seeking the optimal action which reduces uncertainty of a task, prompting robot agents to investigate the location where the target most likely might exist. Using a context state model, we design the agent's high-level decision framework using a Partially-Observable Markov Decision Process. Based on the estimated belief state of the context via sequential observations, the robot's navigation actions are determined to conduct exploratory and detection tasks. By using this multi-modal context model, our agent can effectively handle basic dynamic events, such as obstruction of targets or their absence from the field of view. We implement and demonstrate these capabilities on a mobile robot in real-time. [ABSTRACT FROM AUTHOR]

This article presents a robotic pick-and-place system that is capable of grasping and recognizing both known and novel objects in cluttered environments. The key new feature of the system is that it handles a wide range of object categories without needing any task-specific training data for novel objects. To achieve this, it first uses an object-agnostic grasping framework to map from visual observations to actions: inferring dense pixel-wise probability maps of the affordances for four different grasping primitive actions. It then executes the action with the highest affordance and recognizes picked objects with a cross-domain image classification framework that matches observed images to product images. Since product images are readily available for a wide range of objects (e.g., from the web), the system works out-of-the-box for novel objects without requiring any additional data collection or re-training. Exhaustive experimental results demonstrate that our multi-affordance grasping achieves high success rates for a wide variety of objects in clutter, and our recognition algorithm achieves high accuracy for both known and novel grasped objects. The approach was part of the MIT–Princeton Team system that took first place in the stowing task at the 2017 Amazon Robotics Challenge. All code, datasets, and pre-trained models are available online at http://arc.cs.princeton.edu/ [ABSTRACT FROM AUTHOR]

When completing a task, the ability to implement behavioral strategies to solve it in an effective and cognitively less-demanding way is extremely adaptive for humans. This behavior makes it possible to accumulate evidence and test one’s own predictions about the external world. In this work, starting from examples in the field of spatial hearing research, I analyze the importance of considering motor strategies in perceptual tasks, and I stress the urgent need to create ecological experimental settings, which are essential in allowing the implementation of such behaviors and in measuring them. In particular, I will consider head movements as an example of strategic behavior implemented to solve acoustic space-perception tasks. [ABSTRACT FROM AUTHOR]

View planning (VP) is a technique that guides the adjustment of the sensor's postures in multi-view perception tasks. It converts the perception process into active perception, which improves the intelligence and reduces the resource consumption of the robot. We propose a generic VP system for multiple kinds of visual perception. The VP system is built on the basis of the formal description of the visual task, and the next best view is calculated by the system. When dealing with a given visual task, we can simply update its description as the input of the VP system, and obtain the defined best view in real time. Formal description of the perception task includes the task's status, the objects' prior information library, the visual representation status and the optimization goal. The task's status and the visual representation status are updated when data are received at a new view. If the task's status has not reached its goal, candidate views are sorted based on the updated visual representation status, and the next best view that can minimize the entropy of the model space is chosen as the output of the VP system. Experiments of view planning for 3D recognition and reconstruction tasks are conducted, and the result shows that our algorithm has good performance on different tasks. [ABSTRACT FROM AUTHOR]

Search and rescue (SAR) operations can take significant advantage from supporting autonomous or teleoperated robots and multi-robot systems. These can aid in mapping and situational assessment, monitoring and surveillance, establishing communication networks, or searching for victims. This paper provides a review of multi-robot systems supporting SAR operations, with system-level considerations and focusing on the algorithmic perspectives for multi-robot coordination and perception. This is, to the best of our knowledge, the first survey paper to cover (i) heterogeneous SAR robots in different environments, (ii) active perception in multi-robot systems, while (iii) giving two complementary points of view from the multi-agent perception and control perspectives. We also discuss the most significant open research questions: shared autonomy, sim-to-real transferability of existing methods, awareness of victims' conditions, coordination and interoperability in heterogeneous multi-robot systems, and active perception. The different topics in the survey are put in the context of the different challenges and constraints that various types of robots (ground, aerial, surface, or underwater) encounter in different SAR environments (maritime, urban, wilderness, or other post-disaster scenarios). The objective of this survey is to serve as an entry point to the various aspects of multi-robot SAR systems to researchers in both the machine learning and control fields by giving a global overview of the main approaches being taken in the SAR robotics area.

We discuss the influence upon the fields of robotics and AI of the manner one conceives the relationships between artificial agents' perception, cognition, and action. We shed some light upon a widespread paradigm we call the isolated perception paradigm that addresses perception as isolated from cognition and action. By mobilizing the resources of philosophy (phenomenology and epistemology) and cognitive sciences, and by drawing on recent approaches in AI, we explore what it could mean for robotics and AI to take distance from the isolated perception paradigm. We argue that such a renouncement opens interesting ways to explore the possibilities for designing artificial agents with intrinsic motivations and constitutive autonomy. We then propose Artificial Interactionism, our approach that escapes the isolated perception paradigm by drawing on the inversion of the interaction cycle. When the interaction cycle is inverted, input data are not percepts directly received from the environment, but outcomes of control loops. Perception is not received from sensors in isolation from cognition but is actively constructed by the cognitive architecture through interaction. We give an example implementation of artificial interactionism that demonstrates basic intrinsically motivated learning behavior in a dynamic simulated environment.

One of the fundamental prerequisites for effective collaborations between interactive partners is the mutual sharing of the attentional focus on the same perceptual events. This is referred to as joint attention. In psychological, cognitive, and social sciences, its defining elements have been widely pinpointed. Also the field of human-robot interaction has extensively exploited joint attention which has been identified as a fundamental prerequisite for proficient human-robot collaborations. However, joint attention between robots and human partners is often encoded in prefixed robot behaviours that do not fully address the dynamics of interactive scenarios. We provide autonomous attentional behaviour for robotics based on a multi-sensory perception that robustly relocates the focus of attention on the same targets the human partner attends. Further, we investigated how such joint attention between a human and a robot partner improved with a new biologically-inspired memory-based attention component. We assessed the model with the humanoid robot iCub involved in performing a joint task with a human partner in a real-world unstructured scenario. The model showed a robust performance on capturing the stimulation, making a localisation decision in the right time frame, and then executing the right action. We then compared the attention performance of the robot against the human performance when stimulated from the same source across different modalities (audio-visual and audio only). The comparison showed that the model is behaving with temporal dynamics compatible with those of humans. This provides an effective solution for memory-based joint attention in real-world unstructured environments. Further, we analyzed the localisation performances (reaction time and accuracy), the results showed that the robot performed better in an audio-visual condition than an audio only condition. The performance of the robot in the audio-visual condition was relatively comparable with the behaviour of the human participants whereas it was less efficient in audio-only localisation. After a detailed analysis of the internal components of the architecture, we conclude that the differences in performance are due to egonoise which significantly affects the audio-only localisation performance. [ABSTRACT FROM AUTHOR]

The unique ability to identify one's own body and experience it as one's own is fundamental in goal-oriented behavior and survival. However, the mechanisms underlying the so-called body ownership are yet not fully understood. Evidence based on Rubber Hand Illusion (RHI) paradigms has demonstrated that body ownership is a product of reception and integration of self and externally generated multisensory information, feedforward and feedback processing of sensorimotor signals, and prior knowledge about the body. Crucially, however, these designs commonly involve the processing of proximal modalities while the contribution of distal sensory signals to the experience of ownership remains elusive. Here we propose that, like any robust percept, body ownership depends on the integration and prediction across all sensory modalities, including distal sensory signals pertaining to the environment. To test our hypothesis, we created an embodied goal-oriented Virtual Air Hockey Task, in which participants were to hit a virtual puck into a goal. In two conditions, we manipulated the congruency of distal multisensory cues (auditory and visual) while preserving proximal and action-driven signals entirely predictable. Compared to a fully congruent condition, our results revealed a significant decrease on three dimensions of ownership evaluation when distal signals were incongruent, including the subjective report as well as physiological and kinematic responses to an unexpected threat. Together, these findings support the notion that the way we represent our body is contingent upon all the sensory stimuli, including distal and action-independent signals. The present data extend the current framework of body ownership and may also find applications in rehabilitation scenarios. [ABSTRACT FROM AUTHOR]

The unique ability to identify one’s own body and experience it as one’s own is fundamental in goal-oriented behavior and survival. However, the mechanisms underlying the so-called body ownership are yet not fully understood. Evidence based on Rubber Hand Illusion (RHI) paradigms has demonstrated that body ownership is a product of reception and integration of self and externally generated multisensory information, feedforward and feedback processing of sensorimotor signals, and prior knowledge about the body. Crucially, however, these designs commonly involve the processing of proximal modalities while the contribution of distal sensory signals to the experience of ownership remains elusive. Here we propose that, like any robust percept, body ownership depends on the integration and prediction across all sensory modalities, including distal sensory signals pertaining to the environment. To test our hypothesis, we created an embodied goal-oriented Virtual Air Hockey Task, in which participants were to hit a virtual puck into a goal. In two conditions, we manipulated the congruency of distal multisensory cues (auditory and visual) while preserving proximal and action-driven signals entirely predictable. Compared to a fully congruent condition, our results revealed a significant decrease on three dimensions of ownership evaluation when distal signals were incongruent, including the subjective report as well as physiological and kinematic responses to an unexpected threat. Together, these findings support the notion that the way we represent our body is contingent upon all the sensory stimuli, including distal and action-independent signals. The present data extend the current framework of body ownership and may also find applications in rehabilitation scenarios.

Problem-solving strategies in visual reasoning tasks are often studied based on the analysis of eye movements, which yields high-quality data but is costly and difficult to implement on a large scale. We devised a new graphical user interface for matrix reasoning tasks where the analysis of computer mouse movements makes it possible to investigate item exploration and, in turn, problem-solving strategies. While relying on the same active perception principles underlying eye-tracking (ET) research, this approach has the additional advantages of being user-friendly and easy to implement in real-world testing conditions, and records only voluntary decisions. A pilot study confirmed that embedding items of Raven's Advanced Progressive Matrices (APM) in the interface did not significantly alter its psychometric properties. Experiment 1 indicated that mouse-based exploration indices, when used to assess two major problem-solving strategies in the APM, are related to final performance—as has been found in past ET research. Experiment 2 suggested that constraining some features of the interface favored the adoption of the more efficient solving strategy for some participants. Overall, the findings support the relevance of the present methodology for accessing and manipulating problem-solving strategies. [ABSTRACT FROM AUTHOR]

Recent advancements in sensors and deep learning techniques have improved reliability of robotic perceptual systems, but current systems are not robust enough for real-world challenges such as occlusions and sensing uncertainties in cluttered scenes. To overcome these issues, active or interactive perception actions are often necessary, such as sensor repositioning or object manipulation to reveal more information about the scene. Existing perception systems lack a comprehensive approach that incorporates both active and interactive action spaces, thereby limiting the robot's perception capabilities. Moreover, these systems focus on exploring a single object or scene, without utilizing object information to guide the exploration of multiple objects. In this work, we propose an object-aware hybrid perception system that selects the next best action by considering both active and interactive action spaces and enhances the selection process with an object-aware approach to guide the cognitive robot operating in tabletop scenarios. Novel volumetric utility metrics are used to evaluate actions that include positioning sensors from a heterogeneous set or manipulating objects to gain a better perspective of the scene. The proposed system maintains the volumetric information of the scene that includes semantic information about objects, enabling it to exploit object information, associate occlusion with corresponding objects, and make informed decisions about object manipulation. We evaluate the performance of our system both in simulated and real-world experiments using a Baxter robotic platform equipped with two arms, RGB and depth cameras. Our experimental results show that the proposed system outperforms the compared state-of-the-art methods in the given scenarios, achieving an 11.2% performance increase. [ABSTRACT FROM AUTHOR]

How do cells make efficient collective decisions during tissue morphogenesis? Humans and other organisms use feedback between movement and sensing known as 'sensorimotor coordination' or 'active perception' to inform behaviour, but active perception has not before been investigated at a cellular level within organs. Here we provide the first proof of concept in silico/in vivo study demonstrating that filopodia (actin-rich, dynamic, finger-like cell membrane protrusions) play an unexpected role in speeding up collective endothelial decisions during the time-constrained process of 'tip cell' selection during blood vessel formation (angiogenesis). We first validate simulation predictions in vivo with live imaging of zebrafish intersegmental vessel growth. Further simulation studies then indicate the effect is due to the coupled positive feedback between movement and sensing on filopodia conferring a bistable switch-like property to Notch lateral inhibition, ensuring tip selection is a rapid and robust process. We then employ measures from computational neuroscience to assess whether filopodia function as a primitive (basal) form of active perception and find evidence in support. By viewing cell behaviour through the 'basal cognitive lens' we acquire a fresh perspective on the tip cell selection process, revealing a hidden, yet vital time-keeping role for filopodia. Finally, we discuss a myriad of new and exciting research directions stemming from our conceptual approach to interpreting cell behaviour. This article is part of the theme issue 'Basal cognition: multicellularity, neurons and the cognitive lens'. [ABSTRACT FROM AUTHOR]