Search

In this paper we present a data-driven approach to obtain the static image of a scene, eliminating dynamic objects that might have been present at the time of traversing the scene with a camera. The general objective is to improve vision-based localization and mapping tasks in dynamic environments, where the presence (or absence) of different dynamic objects in different moments makes these tasks less robust. We introduce an end-to-end deep learning framework to turn images of an urban environment that include dynamic content, such as vehicles or pedestrians, into realistic static frames suitable for localization and mapping. This objective faces two main challenges: detecting the dynamic objects, and inpainting the static occluded back-ground. The first challenge is addressed by the use of a convolutional network that learns a multi-class semantic segmentation of the image. The second challenge is approached with a generative adversarial model that, taking as input the original dynamic image and the computed dynamic/static binary mask, is capable of generating the final static image. This framework makes use of two new losses, one based on image steganalysis techniques, useful to improve the inpainting quality, and another one based on ORB features, designed to enhance feature matching between real and hallucinated image regions. To validate our approach, we perform an extensive evaluation on different tasks that are affected by dynamic entities, i.e., visual odometry, place recognition and multi-view stereo, with the hallucinated images. Code has been made available on https://github.com/bertabescos/EmptyCities_SLAM.

The sequential action of glutamine synthetase (GS) and glutamate synthase (GOGAT) in cyanobacteria allows the incorporation of ammonium into carbon skeletons. In the cyanobacterium Synechocystis sp. PCC 6803, the activity of GS is modulated by the interaction with proteins, which include a 65-residue-long intrinsically disordered protein (IDP), the inactivating factor IF7. This interaction is regulated by the presence of charged residues in both IF7 and GS. To understand how charged amino acids can affect the binding of an IDP with its target and to provide clues on electrostatic interactions in disordered states of proteins, we measured the pKa values of all IF7 acidic groups (Glu32, Glu36, Glu38, Asp40, Asp58, and Ser65, the backbone C-terminus) at 100 mM NaCl concentration, by using NMR spectroscopy. We also obtained solution structures of IF7 through molecular dynamics simulation, validated them on the basis of previous experiments, and used them to obtain theoretical estimates of the pKa values. Titration values for the two Asp and three Glu residues of IF7 were similar to those reported for random-coil models, suggesting the lack of electrostatic interactions around these residues. Furthermore, our results suggest the presence of helical structure at the N-terminus of the protein and of conformational changes at acidic pH values. The overall experimental and in silico findings suggest that local interactions and conformational equilibria do not play a role in determining the electrostatic features of the acidic residues of IF7.

Simultaneous localization and mapping (SLAM) consists in the concurrent construction of a model of the environment (the map), and the estimation of the state of the robot moving within it. The SLAM community has made astonishing progress over the last 30 years, enabling large-scale real-world applications and witnessing a steady transition of this technology to industry. We survey the current state of SLAM and consider future directions. We start by presenting what is now the de-facto standard formulation for SLAM. We then review related work, covering a broad set of topics including robustness and scalability in long-term mapping, metric and semantic representations for mapping, theoretical performance guarantees, active SLAM and exploration, and other new frontiers. This paper simultaneously serves as a position paper and tutorial to those who are users of SLAM. By looking at the published research with a critical eye, we delineate open challenges and new research issues, that still deserve careful scientific investigation. The paper also contains the authors' take on two questions that often animate discussions during robotics conferences: Do robots need SLAM? and Is SLAM solved?, Spain. Ministerio de Economía y Competitividad (Grant DPI2015-68905-P and Grupo DGA T04-FSE), Australian Research Council (Grants DP130104413, CE140100016 and FL130100102), National Centre of Competence in Research Robotics, Seventh Framework Programme (European Commission) (EU-FP7-ICTProject TRADR 609763, EU-H2020-688652 and SERI-15.0284)

The sequential action of glutamine synthetase (GS) and glutamate synthase (GOGAT) in cyanobacteria allows the incorporation of ammonium into carbon skeletons. In the cyanobacterium Synechocystis sp. PCC 6803, the activity of GS is modulated by the interaction with proteins, which include a 65-residue-long intrinsically disordered protein (IDP), the inactivating factor IF7. This interaction is regulated by the presence of charged residues in both IF7 and GS. To understand how charged amino acids can affect the binding of an IDP with its target and to provide clues on electrostatic interactions in disordered states of proteins, we measured the pKa values of all IF7 acidic groups (Glu32, Glu36, Glu38, Asp40, Asp58, and Ser65, the backbone C-terminus) at 100 mM NaCl concentration, by using NMR spectroscopy. We also obtained solution structures of IF7 through molecular dynamics simulation, validated them on the basis of previous experiments, and used them to obtain theoretical estimates of the pKa values. Titration values for the two Asp and three Glu residues of IF7 were similar to those reported for random-coil models, suggesting the lack of electrostatic interactions around these residues. Furthermore, our results suggest the presence of helical structure at the N-terminus of the protein and of conformational changes at acidic pH values. The overall experimental and in silico findings suggest that local interactions and conformational equilibria do not play a role in determining the electrostatic features of the acidic residues of IF7.

The sequential action of glutamine synthetase (GS) and glutamate synthase (GOGAT) in cyanobacteria allows the incorporation of ammonium into carbon skeletons. In the cyanobacterium Synechocystis sp. PCC 6803, the activity of GS is modulated by the interaction with proteins, which include a 65-residue-long intrinsically disordered protein (IDP), the inactivating factor IF7. This interaction is regulated by the presence of charged residues in both IF7 and GS. To understand how charged amino acids can affect the binding of an IDP with its target and to provide clues on electrostatic interactions in disordered states of proteins, we measured the pKa values of all IF7 acidic groups (Glu32, Glu36, Glu38, Asp40, Asp58, and Ser65, the backbone C-terminus) at 100 mM NaCl concentration, by using NMR spectroscopy. We also obtained solution structures of IF7 through molecular dynamics simulation, validated them on the basis of previous experiments, and used them to obtain theoretical estimates of the pKa values. Titration values for the two Asp and three Glu residues of IF7 were similar to those reported for random-coil models, suggesting the lack of electrostatic interactions around these residues. Furthermore, our results suggest the presence of helical structure at the N-terminus of the protein and of conformational changes at acidic pH values. The overall experimental and in silico findings suggest that local interactions and conformational equilibria do not play a role in determining the electrostatic features of the acidic residues of IF7.

The sequential action of glutamine synthetase (GS) and glutamate synthase (GOGAT) in cyanobacteria allows the incorporation of ammonium into carbon skeletons. In the cyanobacterium Synechocystis sp. PCC 6803, the activity of GS is modulated by the interaction with proteins, which include a 65-residue-long intrinsically disordered protein (IDP), the inactivating factor IF7. This interaction is regulated by the presence of charged residues in both IF7 and GS. To understand how charged amino acids can affect the binding of an IDP with its target and to provide clues on electrostatic interactions in disordered states of proteins, we measured the pKa values of all IF7 acidic groups (Glu32, Glu36, Glu38, Asp40, Asp58, and Ser65, the backbone C-terminus) at 100 mM NaCl concentration, by using NMR spectroscopy. We also obtained solution structures of IF7 through molecular dynamics simulation, validated them on the basis of previous experiments, and used them to obtain theoretical estimates of the pKa values. Titration values for the two Asp and three Glu residues of IF7 were similar to those reported for random-coil models, suggesting the lack of electrostatic interactions around these residues. Furthermore, our results suggest the presence of helical structure at the N-terminus of the protein and of conformational changes at acidic pH values. The overall experimental and in silico findings suggest that local interactions and conformational equilibria do not play a role in determining the electrostatic features of the acidic residues of IF7.

It is essential for a robot to be able to detect revisits or loop closures for long-term visual navigation.A key insight explored in this work is that the loop-closing event inherently occurs sparsely, that is, the image currently being taken matches with only a small subset (if any) of previous images. Based on this observation, we formulate the problem of loop-closure detection as a sparse, convex $\ell_1$-minimization problem. By leveraging fast convex optimization techniques, we are able to efficiently find loop closures, thus enabling real-time robot navigation. This novel formulation requires no offline dictionary learning, as required by most existing approaches, and thus allows online incremental operation. Our approach ensures a unique hypothesis by choosing only a single globally optimal match when making a loop-closure decision. Furthermore, the proposed formulation enjoys a flexible representation with no restriction imposed on how images should be represented, while requiring only that the representations are "close" to each other when the corresponding images are visually similar. The proposed algorithm is validated extensively using real-world datasets.

This paper describes a system for performing real-time multi-session visual mapping in large-scale environments. Multi-session mapping considers the problem of combining the results of multiple simultaneous localisation and mapping (SLAM) missions performed repeatedly over time in the same environment. The goal is to robustly combine multiple maps in a common metrical coordinate system, with consistent estimates of uncertainty. Our work employs incremental smoothing and mapping (iSAM) as the underlying SLAM state estimator and uses an improved appearance-based method for detecting loop closures within single mapping sessions and across multiple sessions. To stitch together pose graph maps from multiple visual mapping sessions, we employ spatial separator variables, called anchor nodes, to link together multiple relative pose graphs. The system architecture consists of a separate front-end for computing visual odometry and windowed bundle adjustment on individual sessions, in conjunction with a back-end for performing the place recognition and multi-session mapping. We provide experimental results for real-time multi-session visual mapping on wheeled and handheld datasets in the MIT Stata Center. These results demonstrate key capabilities that will serve as a foundation for future work in large-scale persistent visual mapping., Science Foundation Ireland (Strategic Research Cluster Grant 07/SRC/I1168), United States. Office of Naval Research (Grant N00014-10-1-0936), United States. Office of Naval Research (Grant N00014-12-1-0093), Spain. Direccion General de Investigacion (DPI2009-13710), Spain. Direccion General de Investigacion (DPI2009-07130)

Simultaneous Localization and Mapping (SLAM)consists in the concurrent construction of a model of the environment (the map), and the estimation of the state of the robot moving within it. The SLAM community has made astonishing progress over the last 30 years, enabling large-scale real-world applications, and witnessing a steady transition of this technology to industry. We survey the current state of SLAM. We start by presenting what is now the de-facto standard formulation for SLAM. We then review related work, covering a broad set of topics including robustness and scalability in long-term mapping, metric and semantic representations for mapping, theoretical performance guarantees, active SLAM and exploration, and other new frontiers. This paper simultaneously serves as a position paper and tutorial to those who are users of SLAM. By looking at the published research with a critical eye, we delineate open challenges and new research issues, that still deserve careful scientific investigation. The paper also contains the authors' take on two questions that often animate discussions during robotics conferences: Do robots need SLAM? and Is SLAM solved?

Simultaneous localization and mapping (SLAM) consists in the concurrent construction of a model of the environment (the map), and the estimation of the state of the robot moving within it. The SLAM community has made astonishing progress over the last 30 years, enabling large-scale real-world applications and witnessing a steady transition of this technology to industry. We survey the current state of SLAM and consider future directions. We start by presenting what is now the de-facto standard formulation for SLAM. We then review related work, covering a broad set of topics including robustness and scalability in long-term mapping, metric and semantic representations for mapping, theoretical performance guarantees, active SLAM and exploration, and other new frontiers. This paper simultaneously serves as a position paper and tutorial to those who are users of SLAM. By looking at the published research with a critical eye, we delineate open challenges and new research issues, that still deserve careful scientific investigation. The paper also contains the authors' take on two questions that often animate discussions during robotics conferences: Do robots need SLAM? and Is SLAM solved?

This paper describes a system for performing real-time multi-session visual mapping in large-scale environments. Multi-session mapping considers the problem of combining the results of multiple simultaneous localisation and mapping (SLAM) missions performed repeatedly over time in the same environment. The goal is to robustly combine multiple maps in a common metrical coordinate system, with consistent estimates of uncertainty. Our work employs incremental smoothing and mapping (iSAM) as the underlying SLAM state estimator and uses an improved appearance-based method for detecting loop closures within single mapping sessions and across multiple sessions. To stitch together pose graph maps from multiple visual mapping sessions, we employ spatial separator variables, called anchor nodes, to link together multiple relative pose graphs. The system architecture consists of a separate front-end for computing visual odometry and windowed bundle adjustment on individual sessions, in conjunction with a back-end for performing the place recognition and multi-session mapping. We provide experimental results for real-time multisession visual mapping on wheeled and handheld datasets in the MIT Stata Center. These results demonstrate key capabilities that will serve as a foundation for future work in large-scale persistent visual mapping.

This paper describes a system for performing real-time multi-session visual mapping in large-scale environments. Multi-session mapping considers the problem of combining the results of multiple simultaneous localisation and mapping (SLAM) missions performed repeatedly over time in the same environment. The goal is to robustly combine multiple maps in a common metrical coordinate system, with consistent estimates of uncertainty. Our work employs incremental smoothing and mapping (iSAM) as the underlying SLAM state estimator and uses an improved appearance-based method for detecting loop closures within single mapping sessions and across multiple sessions. To stitch together pose graph maps from multiple visual mapping sessions, we employ spatial separator variables, called anchor nodes, to link together multiple relative pose graphs. The system architecture consists of a separate front-end for computing visual odometry and windowed bundle adjustment on individual sessions, in conjunction with a back-end for performing the place recognition and multi-session mapping. We provide experimental results for real-time multisession visual mapping on wheeled and handheld datasets in the MIT Stata Center. These results demonstrate key capabilities that will serve as a foundation for future work in large-scale persistent visual mapping.

In this paper we show how to carry out robust place recognition using both near and far information provided by a stereo camera. Visual appearance is known to be very useful in place recognition tasks. In recent years, it has been shown that taking geometric information also into account further improves system robustness. Stereo visual systems provide 3D information and texture of nearby regions, as well as an image of far regions. In order to make use of all this information, our system builds two probabilistic undirected graphs, each considering either near or far information. Inference is carried out in the framework of conditional random fields. We evaluate our algorithm in public indoor and outdoor datasets from the Rawseeds project and in an outdoor dataset obtained at the MIT campus. Results show that this combination of information is very useful to solve challenging cases of perceptual aliasing.

In this paper we show how to carry out robust place recognition using both near and far information provided by a stereo camera. Visual appearance is known to be very useful in place recognition tasks. In recent years, it has been shown that taking geometric information also into account further improves system robustness. Stereo visual systems provide 3D information and texture of nearby regions, as well as an image of far regions. In order to make use of all this information, our system builds two probabilistic undirected graphs, each considering either near or far information. Inference is carried out in the framework of conditional random fields. We evaluate our algorithm in public indoor and outdoor datasets from the RAWSEEDS project and in an outdoor dataset obtained at the MIT campus. Results show that this combination of information is very useful to solve challenging cases of perceptual aliasing.

In this paper we show how to carry out robust place recognition using both near and far information provided by a stereo camera. Visual appearance is known to be very useful in place recognition tasks. In recent years, it has been shown that taking geometric information also into account further improves system robustness. Stereo visual systems provide 3D information and texture of nearby regions, as well as an image of far regions. In order to make use of all this information, our system builds two probabilistic undirected graphs, each considering either near or far information. Inference is carried out in the framework of conditional random fields. We evaluate our algorithm in public indoor and outdoor datasets from the RAWSEEDS project and in an outdoor dataset obtained at the MIT campus. Results show that this combination of information is very useful to solve challenging cases of perceptual aliasing.

This paper describes a system for performing multisession visual mapping in large-scale environments. Multi-session mapping considers the problem of combining the results of multiple Simultaneous Localisation and Mapping (SLAM) missions performed repeatedly over time in the same environment. The goal is to robustly combine multiple maps in a common metrical coordinate system, with consistent estimates of uncertainty. Our work employs incremental Smoothing and Mapping (iSAM) as the underlying SLAM state estimator and uses an improved appearance-based method for detecting loop closures within single mapping sessions and across multiple sessions. To stitch together pose graph maps from multiple visual mapping sessions, we employ spatial separator variables, called anchor nodes, to link together multiple relative pose graphs. We provide experimental results for multi-session visual mapping in the MIT Stata Center, demonstrating key capabilities that will serve as a foundation for future work in large-scale persistent visual mapping.

This paper describes a system for performing multisession visual mapping in large-scale environments. Multi-session mapping considers the problem of combining the results of multiple Simultaneous Localisation and Mapping (SLAM) missions performed repeatedly over time in the same environment. The goal is to robustly combine multiple maps in a common metrical coordinate system, with consistent estimates of uncertainty. Our work employs incremental Smoothing and Mapping (iSAM) as the underlying SLAM state estimator and uses an improved appearance-based method for detecting loop closures within single mapping sessions and across multiple sessions. To stitch together pose graph maps from multiple visual mapping sessions, we employ spatial separator variables, called anchor nodes, to link together multiple relative pose graphs. We provide experimental results for multi-session visual mapping in the MIT Stata Center, demonstrating key capabilities that will serve as a foundation for future work in large-scale persistent visual mapping.