Search

Although image change detection (ICD) methods provide good detection accuracy for many scenarios, most existing methods rely on place-specific background modeling. The time/space cost for such place-specific models is prohibitive for large-scale scenarios, such as long-term robotic visual simultaneous localization and mapping (SLAM). Therefore, we propose a novel ICD framework that is specifically customized for long-term SLAM. This study is inspired by the multi-map-based SLAM framework, where multiple maps can perform mutual diagnosis and hence do not require any explicit background modeling/model. We extend this multi-map-based diagnosis approach to a more generic single-map-based object-level diagnosis framework (i.e., ICD), where the self-localization module of SLAM, which is the change object indicator, can be used in its original form. Furthermore, we consider map diagnosis on a state-of-the-art deep convolutional neural network (DCN)-based SLAM system (instead of on conventional bag-of-words or landmark-based systems), in which the blackbox nature of the DCN complicates the diagnosis problem. Additionally, we consider a three-dimensional point cloud (PC)-based (instead of typical monocular color image-based) SLAM and adopt a state-of-the-art scan context PC descriptor for map diagnosis for the first time.

Map matching, the ability to match a local map built by a mobile robot to previously built maps, is crucial in many robotic mapping, self-localization, and simultaneous localization and mapping (SL...

In exploring the 1-to-Nmap matching problem that exploits a compact map data description, we hope to improve map matching scalability used in robot vision tasks. We propose explicitly targeting fast succinct map matching, which consists of map matching subtasks alone. These tasks include offline map matching attempts to find compact part-based scene models that effectively explain individual maps by using fewer larger parts. These tasks also include online map matching to find correspondence between part-based maps efficiently. Our part-based scene modeling approach is unsupervised and uses common pattern discovery (CPD) between input and known reference maps. Results of our experiments, which use a publicly available radish dataset, confirm the effectiveness of our proposed approach.

We propose a novel bag-of-words (BoW) framework for building and retrieving a compact database of view images for use in robotic localization, mapping, and SLAM applications. Unlike most previous methods, our method does not describe an image based on its many small local features (e.g., bag-of-SIFT-features). Instead, the proposed bag-of-bounding-boxes (BoBB) approach attempts to describe an image based on fewer larger object patterns, which leads to a semantic and compact image descriptor. To make the view retrieval systemmore practical and autonomous, the object pattern discovery process is unsupervised through a common pattern discovery (CPD) between the input and known reference images without requiring the use of a pre-trained object detector. Moreover, our CPD subtask does not rely on good image segmentation techniques and is able to handle scale variations by exploiting the recently developed CPD technique, i.e., a spatial randompartition. Following a traditional bounding-box based object annotation and knowledge transfer, we compactly describe an image in a BoBB form. Using a slightly modified inverted file system, we efficiently index and/or search for the BoBB descriptors. Experiments using the publicly available “Robot-Car” dataset show that the proposed method achieves accurate object-level view image retrieval using significantly compact image descriptors, e.g., 20 words per image.

Obtaining a compact representation of a large-size datapoint map built by mapper robots is a critical issue for recent SLAM applications. This ”map compression” problem is explored from a novel perspective of dictionary-based map compression in the paper. The primary contribution of the paper is proposal of an incremental scheme for simultaneous mapping and map-compression applications. An incremental map compressor is presented by employing a modified RANSAC map-matching scheme as well as the compact projection technique. Experiments show promising results in terms of compression speed, compactness of data and structure, as well as an application to the compression distance.

Map retrieval, the problem of similarity search among a large collection of 2D pointset maps previously built by mobile robots, is crucial for autonomous navigation in both indoor and outdoor environments. Unlike previous Bag-of-words scene model which lacks spatial layout information, we exploit a holistic local map descriptor which is view-independent and highly discriminative. The current study is inspired by our recent papers on “local map descriptor”, in which the viewpoint or the origin of local map coordinate is planned by scene parsing and the spatial layout of a local map is represented with respect to the planned viewpoint. The main contribution of this study is to build on the framework of local map descriptor and experimentally evaluate the effectiveness of the proposed method. The key observation is that accuracy of the spatial layout should be improved by introducing an accurate local mapping technique. To this end, we implemented the state-of-the-art particle filter-based mapping algorithm of Grid Mapping. The next contribution is that we conducted map retrieval experiments using a real mobile robot in feature-less office-like environments. We experimentally demonstrate the effectiveness of the proposed approach.

Through practice, people are able to integrate a secondary sequence (e.g., a stimulus-based sequence) into a primary sequence (e.g., a response-based sequence), but it is still controversial whether the integrated sequences lead to better learning than only the primary sequence. In the present study, we aimed to investigate the effects of a sequence that integrated space and color sequences on early and late learning phases (corresponding to effector-independent and effector-dependent learning, respectively) and how the effects differed in the integrated and primary sequences in each learning phase. In the task, the participants were required to learn a sequence of button presses using trial-and-error and to perform the sequence successfully for 20 trials (m × n task). First, in the baseline task, all participants learned a non-colored sequence, in which the response button always turned red. Then, in the learning task, the participants were assigned to two groups: a colored sequence group (i.e., space and color) or a non-colored sequence group (i.e., space). In the colored sequence, the response button turned a pre-determined color and the participants were instructed to attend to the sequences of both location and color as much as they could. The results showed that the participants who performed the colored sequence acquired the correct button presses of the sequence earlier, but showed a slower mean performance time than those who performed the non-colored sequence. Moreover, the slower performance time in the colored sequence group remained in a subsequent transfer task in which the spatial configurations of the buttons were vertically mirrored from the learning task. These results indicated that if participants explicitly attended to both the spatial response sequence and color stimulus sequence at the same time, they could develop their spatial representations of the sequence earlier (i.e., early development of the effector-independent learning), but might not be able to enhance their motor representations of the sequence (i.e., late development of the effector-dependent learning). Thus, the undeveloped effector-dependent representations in the colored sequence group directly led to a long performance time in the transfer sequence.

It has been known remembered locations of visual stimuli are systematically distorted in a two-dimensional (i.e., retinal) coordinate, which includes the foveal bias, memory averaging, and landmark effect. The present study aimed at examining how the remembered depth position of a visual target would be distorted. Also, we examined whether depth distortion would differ for perceptual and motor-related tasks. In the experiments, a visual target was presented for one second at various distances from the observers by using a 3D projector. The fixation distance was also varied. After the disappearance of the target, observers performed either perceptual judgment (the method of constant to estimate the point of psychological subjective equality) or motor response (by using a 3D mouse) for the remembered target distance. The results showed that the remembered depth of the target was compressed toward the averaged depth of the possible depth range for both perceptual and motor responses. In addition, however, the perceptual responses tended to overestimate the depth when the fixation depth was closer than or the same as the nearest possible depth of the target. No bias was observed when the fixation depth was farther than the nearest possible depth. For the motor responses, there was a general underestimation irrespective of fixation depth. These results suggest that spatial memory of depth for visual stimuli may be qualitatively different between perceptual and motor responses.

Retrieving a large collection of environment maps built by mapper robots is a key problem in mobile robot self-localization. The map retrieval problem is studied from the novel perspective of the multi-scale Bag-Of-Features (BOF) approach in this paper. In general, the multi-scale approach is advantageous in capturing both the global structure and the local details of a given map. BOF map retrieval is advantageous in its compact map representation as well as the efficient map retrieval using an inverted file system. The main contribution of this paper is combining the advantages of both approaches. Our approach is based on multi cue BOF as well as packing BOF, and achieves the efficiency and compactness of the map retrieval system. Experiments evaluate the effectiveness of the techniques presented using a large collection of environment maps.

Obtaining a compact representation of a large-size feature map built by mapper robots is a critical issue in recent mobile robotics. This “map compression” problem is explored from a novel perspective of dictionary-based data compression techniques in the paper. The primary contribution of the paper is the proposal of the dictionary-based map compression approach. A map compression system is presented by employing RANSAC map matching and sparse coding as building blocks. The effectiveness levels of the proposed techniques is investigated in terms of map compression ratio, compression speed, the retrieval performance of compressed/decompressed maps, as well as applications to the Kolmogorov complexity.

This paper presents a novel approach for robot localization using landmark maps. With recent progress in SLAM researches, it has become crucial for a robot to obtain and use large-size maps that are incrementally built by other mapper robots. Our localization approach successfully works with such incremental and large-size maps. In literature, RANSAC map-matching has been a promising approach for large-size maps. We extend the RANSAC map-matching so as to deal with incremental maps. We combine the incremental RANSAC with an incremental LSH database and develop a hybrid of the position-based and the appearance-based approaches. A series of experiments using radish dataset show promising results.

This work is concerned with the problem of robot localization using standard RFID tags as landmarks and an RFID reader as a landmark sensor. A main advantage of such an RFID-based localization system is the availability of landmark ID measurement, which trivially solves the data association problem. While the main drawback of an RFID system is its low spatial accuracy. The result in this paper is an improvement of the localization accuracy for a standard short-range RFID sensor. One of the main contributions is a proposal of a machine learning approach in which multiple classifiers are trained to distinguish RFID-signal features of each location. Another contribution is a design tool for tag arrangement by which the tag configuration needs not be manually designed by the user, but can be automatically recommended by the system. The effectiveness of the proposed technique is evaluated experimentally with a real mobile robot and an RFID system.

A problem of probabilistic vision-based multi-person tracking with a monocular camera in a cluttered office environment is addressed. In such a case, it is often difficult to obtain reliable measurements of the absolute position of a person on the floor plane due to the inherent uncertainty in depth. To deal with this problem, a novel measurement technique based on occlusion reasoning is proposed and combined with a standard particle filtering technique. This technique is based on the relative position of a person with respect to background objects. The effectiveness of the proposed technique is investigated in an experiment where two persons overlap on the image plane.

It is fundamental problem for mobile robots to detect 2D free regions where they can move by local sensing. In many indoor environments, there may be lots of objects, such as chairs, desks, etc., and much portion of the free region can not be directly detected. To resolve this problem, we have developed a method for detecting free regions in which a robot visually tracks walking human, and regards 2D regions swept by the human as free regions. Based on this idea, we propose a method for building a map of obstacles and free regions, and navigation with an distance transform based path planning algorithm. The free regions were obtained reliably by combining stereo vision and visual tracking techniques.

One of the fundamental problems of the mobile robots is self-localization, i.e. to estimate the self-position by comparing sensor data and a map. In non-stationary enviroments, a robot should avoid to use changed objects as landmarks in the localization. In this paper, we propose a self-localization method that is robust against changes in environments. The method identifies changes from noisy and ambiguous sensor data. Since an object with a random shape may be added at a random position, it genetates and utilizes multiple hypotheses about the changes. A number of simulation experiments have been performed in various environments, to demonstrate the effectiveness of the method.

Many mobile robots use maps for path planning. However, due to changes of obstacle configurations in the environment, utility of the map is often deteriorated. In this paper, we address a problem of map updating task, where a robot automatically explores the environment to update a map. Map updating task is different from well known map making task in that the robot can estimate global structure of the environment by using the initial map. This enables a map updating robot to take into account possible map users, and to estimate utility of a map for them. We propose a method for motion planning in the task, for quickly increasing utility of a map. We also describle an algorithm to execute the planning in a practical computing time. A number of experiments have been performed in various environments and changes, to demonstrate the effectiveness of the method.

Vehicle relocation is the problem in which a mobile robot has to estimate the self-position with respect to an a priori map of landmarks using the perception and the motion measurements without using any knowledge of the initial self-position. Recently, RANdom SAmple Consensus (RANSAC), a robust multi-hypothesis estimator, has been successfully applied to offline relocation in static environments. On the other hand, online relocation in dynamic environments is still a difficult problem, for available computation time is always limited, and for measurement include many outliers. To realize real time algorithm for such an online process, we have developed an incremental version of RANSAC algorithm by extending an efficient preemption RANSAC scheme. This novel scheme named incremental RANSAC is able to find inlier hypotheses of self-positions out of large number of outlier hypotheses contaminated by outlier measurements., Comment: Offprint of ICRA2006 paper

Studies have found that preferences for visual features such as colors and shapes reveal systematic tendencies, certain colors and shapes were more liked than others [1, 8, 9, 11, 13]. However, little has been known about the relationship between preferences for color and shape. Here, using an individual differences approach, we investigated whether there would be any correlations between preferences for color and shape. Sixty-nine Japanese university students rated how much they liked colors and shapes in separate sessions. The visual stimuli contained 40 colors taken from the Natural Color System, and 102 visual shapes ranging from simple 2D shapes (i.e., circle and triangle) and 2D projections of 3D objects (i.e., cone and pyramid). Results indicated that preferences for certain colors and shapes correlated with each other, the simple 2D shapes were tended to be preferred by the participants who preferred "warm" colors and the 3D shapes were tended to be preferred by the participants who preferred "cold" colors. Those results might be interpreted as manifestations of the shared semantic impressions (i.e., light heavy) between colors and shapes.

This paper is concerned with the problem of mobile robot localization using a novel compact representation of visual landmarks. With recent progress in lifelong map-learning as well as in information sharing networks, compact representation of a large-size landmark database has become crucial. In this paper, we propose a compact binary code (e.g. 32bit code) landmark representation by employing the semantic hashing technique from web-scale image retrieval. We show how well such a binary representation achieves compactness of a landmark database while maintaining efficiency of the localization system. In our contribution, we investigate the cost-performance, the semantic gap, the saliency evaluation using the presented techniques as well as challenge to further reduce the resources (#bits) per landmark. Experiments using a high-speed car-like robot show promising results.

This paper addresses the problem of feature-based robot localization in large-size environments. With recent progress in SLAM techniques, it has become crucial for a robot to estimate the self-position in real-time with respect to a large-size map that can be incrementally build by other mapper robots. Self-localization using large-size maps have been studied in litelature, but most of them assume that a complete map is given prior to the self-localization task. In this paper, we present a novel scheme for robot localization as well as map representation that can successfully work with large-size and incremental maps. This work combines our two previous works on incremental methods, iLSH and iRANSAC, for appearance-based and position-based localization.

This paper studies the problem of mobile robot map-building using a low-accuracy RFID sensor. In recent years, there has been increasing number of mobile robot systems using RFID landmarks. A difficulty arises from the fact that the characteristics of RFID sensors are quite electrically-sensitive, which makes the map-building process difficult to converge. Our solution is to integrate a robust multiscan-based online sensor model within an SGD framework of map-optimization, which significantly improves the convergence and accuracy.

Mobile robot teleoperation is a method for a human user to interact with a mobile robot over time and distance. Successful teleoperation depends on how well sensor images taken by the mobile robot are visualized to the user. To enhance the efficiency and the flexibility of the visualization, an image retrieval system on such a robot's image database would be very useful. The main difference of this image retrieval problem from the standard one is that due to variety of viewing conditions, there exist various of relevant images. The main contribution of this paper is to propose an efficient retrieval approach for such a problem by utilizing correlation between visual features and real world locations of images. An active learning technique based on support vector machine is extended for this aim. The second contribution is to present a multi-hypothesis approach to deal with location uncertainty inherent in real world applications.

Vision-based multi-person tracking is the problem to estimate the positions of persons while simultaneously differentiating individual persons in a sequence of monocular images. For robustness and efficiency, it is crucial to measure positions of individual persons. Absolute position measurement is one of the standard measurements, which provides the absolute position of a person with respect to the floor plane. However, absolute position measurement alone is often not reliable due to the inherent uncertainty in depth. In this paper, a novel type measurement, called relative position measurement, is introduced to determine the position relationship between two persons. Such a measurement provides useful information for differentiating individual persons. Moreover, an effective tracking algorithm is proposed based on a particle filtering technique to combine the advantages of the absolute and the relative position measurements

Global localization is the problem in which a mobile robot has to estimate the self-position with respect to an a priori map of landmarks using the perception and the motion measurements without using any knowledge of the initial self-position. Recently, random sample consensus (RANSAC), a robust multi-hypothesis estimator, has been successfully applied to offline global localization in static environments. However, online global localization in dynamic environments is still a difficult problem, due to incrementally arriving measurements as well as many outlier measurements. To realize a real time algorithm for such an online process, we have developed an incremental version of RANSAC algorithm by extending an efficient preemptive RANSAC scheme, in order to find inlier hypotheses of self-positions out of a large number of outlier hypotheses contaminated by the outlier measurements

Images play important role in telerobotic interface systems. A set of images taken by an image sensor equipped on a mobile robot is helpful for a remote user to recognize the conditions of the robot as well as its operating environments. We view such an image set as a large-scale image database, and apply Content-Based Image Retrieval (CBIR) to visualize the image database. We will propose a novel CBIR method that efficiently searches such relevant objects based not only on their features, but also on their locations with respect to the operating environment.

Image based rendering (IBR) is one of standard approaches to view synthesis. IBR would be very important for human-robot interface system (HRI) such as a teleoperation system, since the synthesized views are helpful for a human user to understand the robot's operating environment. In this paper, we address the problem of IBR on images taken by a mobile robot. Main difficulty of our problem is that the viewpoint locations of input (real) images are not precisely known, due to estimation errors inherent in the positioning systems. To solve this problem, we will propose a novel IBR method, where the location uncertainty is reduced using a visual landmark, which is commonly used in mobile robotics. In addition, novel priors on real images are introduced to regularize the IBR problem. As a result, IBR can be performed successfully under the location uncertainty.

A set of images taken by a mobile robot during navigation can be viewed as a large-scale image database, content-based image retrieval (CBIR) is an effective approach to image retrieval on such large-scale image databases. For the aim of human-robot interface, CBIR on the robot image database would be very useful, to train the robot to learn features (e.g. color, texture) of the relevant objects (e.g. book, chair) that a human user is interested in. To achieve CBIR, a small but sufficient member of relevant objects' features are required. We will propose a novel CBIR method that efficiently searches such relevant objects based not only on their features, but also on their locations with respect to the robot's workspace. To combine both of the feature and the location information, a novel 2D occupancy map called query based occupancy map (QOM) is presented.

One of the fundamental problems of the mobile robots is self-localization, i.e., to estimate the self-position by comparing sensor data and a map. In non-stationary environments, a robot should avoid to use changed objects as landmarks in the localization. However, in most previous localization methods, it is assumed that there is no change, or changes are easily identified by sensing. In this paper, we propose a self-localization method that is robust against changes in environments. The method identifies changes from noisy and ambiguous sensor data. Since an object with a random shape may be added at a random position, it generates and utilizes multiple hypotheses about the changes. A number of simulation experiments have been performed in various environments, to demonstrate the effectiveness of the method.

It is a fundamental problem for mobile robots to detect free regions where they can move by local sensing. One of the most popular methods for detecting free regions is to observe the floor directly. In cluttered indoor environments, however, most part of the floor may be occluded by obstacles. To resolve the problem, we have developed an alternative method in which persons walking in the same room are observed, and 2D areas swept by the persons are regarded as free regions. For this purpose, we have combined two different methods: stereo vision and visual tracking of persons. Geometrical information of static obstacles, and topological information of free regions are respectively acquired by the methods. The uncertainty analysis of the methods, and experimental results are also shown.

In this paper, we propose a method for global localization in non-stationary environments, where the environment is partially or completely different from the map. We assume there is no moving object. It is difficult to detect changes when both of the self-position and the map have large uncertainties. To solve the problem, we extended Monte Carlo Localization (MCL) so as to generate a number of hypotheses about the change as well as the self-position. We also introduced Sensor Resetting Localization (SRL), in order to generate initial estimation of self-position, or to recover from large positioning errors. The proposed method has been tested in a number of environments as well as changes. As a results, we found the proposed method is effective even when "Rate Of Changes (ROC)" is high in the environment.

Our goal is to develop a new vision system for detecting mobile robots and their motion sequences in office environments. The vision system watches a sequence of images acquired by the vision sensor, as well as a sequence of robot's command transmitted via a wireless network. To identify the robot, a sequence of motion is extracted for a moving object in the image sequence, and compared with the sequence of command. Simple features are used and extracted with various types of methods for detecting moving objects. This simplicity helps improve the robustness against unpredictable uncertainties inherent in the environments. In. order to improve recognition rate per frame, the vision system interacts with the robot, to learn features of the robot. A support vector machine (SVM) is used for this learning. In addition, the system is optimized reflecting characteristics of the robot and the environment.

For mobile robots working in a dynamic environment, there is a great need for a global environment map that should be updated timely. In the paper, we propose methods for the map updating purpose by using a specialized mobile robot. The map updating system is composed of two basic processing components: local change detection and global observation planning. Principal strategies for the observation position planning are developed making good use of knowledge both from an outmoded initial map and from up-to-date sensory information. The paper gives a detailed description of the new method: moving around object clusters. Simulation results show this method is efficient even for detecting large changes in a complex environment.