Search

This paper presents a real-time 3D shape fusion system that faithfully integrates very high resolution 3D scans with the goal of maximizing details preservation. The system fully maps complex shapes while allowing free movement similarly to dense SLAM systems in robotics where sensor fusion techniques map large environments. We propose a novel framework to integrate shapes into a volume with fine details preservation of the reconstructed shape which is an important aspect in many applications, especially for industrial inspection. The truncated signed distance function is generalized with a global variational scheme that controls edge preservation and leads to updating cumulative rules adapted for GPU implementation. The framework also embeds a map deformation method to online deform the shape and correct the system trajectory drift at few microns accuracy. Results are presented from the integrated system on two mechanical objects which illustrate the benefits of the proposed approach., 2018 15th Conference on Computer and Robot Vision (CRV), May 8-10, 2018, Toronto, ON, Canada

We present an overview of our recent progress and the current state-of-the-art techniques of color image fusion for night vision applications. Inspired by previously developed color opponent fusing schemes, we initially developed a simple pixel-based false color-mapping scheme that yielded fused false color images with large color contrast and preserved the identity of the input signals. This method has been successfully deployed in different areas of research. However, since this color mapping did not produce realistic colors, we continued to develop a statistical color-mapping procedure that would transfer the color distribution of a given example image to a multiband nighttime image. This procedure yields a realistic color rendering. However, it is computationally expensive and achieves no color constancy since the mapping depends on the relative amounts of the different materials in the scene. By applying the statistical mapping approach in a color look-up-table framework, we finally achieved both color constancy and computational simplicity. This sample-based color transfer method is specific for different types of materials in a scene and can be easily adapted for the intended operating theatre and the task at hand. The method can be implemented as a look-up-table transform and is highly suitable for real-time implementations.

We present the design and evaluation of a new demonstrator rifle sight viewing system containing direct view, red aim point and fusion of an (uncooled, LWIR) thermal sensor with a digital image intensifier. Our goal is to create a system that performs well under a wide variety of (weather) conditions during daytime and nighttime and combines the advantages of the various sensor systems. A real-time colour image with salient hot targets is obtained from the night vision sensors by implementing the Colour-the-Night fusion method (Hogervorst & Toet, 2010) on the on-board processor. The prototype system was evaluated in a series of field trials with military observers performing detection and identification tasks. The tests showed that during daytime the addition of a thermal image to direct vision is advantageous, e.g. for the detection of hot targets. At nighttime, the fusion of thermal and image intensified imagery results in increased situational awareness and improved detection of (hot) targets. For identification of small (handheld) objects, the technology needs to be further refined.

We present the design and evaluation of a new demonstrator rifle sight viewing system containing direct view, red aim point and fusion of an (uncooled, LWIR) thermal sensor with a digital image intensifier. Our goal is to create a system that performs well under a wide variety of (weather) conditions during daytime and nighttime and combines the advantages of the various sensor systems. A real-time colour image with salient hot targets is obtained from the night vision sensors by implementing the Colour-the-Night fusion method (Hogervorst & Toet, 2010) on the on-board processor. The prototype system was evaluated in a series of field trials with military observers performing detection and identification tasks. The tests showed that during daytime the addition of a thermal image to direct vision is advantageous, e.g. for the detection of hot targets. At nighttime, the fusion of thermal and image intensified imagery results in increased situational awareness and improved detection of (hot) targets. For identification of small (handheld) objects, the technology needs to be further refined.

We present an overview of our recent progress and the current state-of-the-art techniques of color image fusion for night vision applications. Inspired by previously developed color opponent fusing schemes, we initially developed a simple pixel-based false color-mapping scheme that yielded fused false color images with large color contrast and preserved the identity of the input signals. This method has been successfully deployed in different areas of research. However, since this color mapping did not produce realistic colors, we continued to develop a statistical color-mapping procedure that would transfer the color distribution of a given example image to a multiband nighttime image. This procedure yields a realistic color rendering. However, it is computationally expensive and achieves no color constancy since the mapping depends on the relative amounts of the different materials in the scene. By applying the statistical mapping approach in a color look-up-table framework, we finally achieved both color constancy and computational simplicity. This sample-based color transfer method is specific for different types of materials in a scene and can be easily adapted for the intended operating theatre and the task at hand. The method can be implemented as a look-up-table transform and is highly suitable for real-time implementations.

[EN]In recent years, two dimensional laser range finders mounted on vehicles is becoming a fruitful solution to achieve safety and environment recognition requirements (Keicher & Seufert, 2000), (Stentz et al., 2002), (DARPA, 2007). They provide real-time accurate range measurements in large angular fields at a fixed height above the ground plane, and enable robots and vehicles to perform more confidently a variety of tasks by fusing images from visual cameras with range data (Baltzakis et al., 2003). Lasers have normally been used in industrial surveillance applications to detect unexpected objects and persons in indoor environments. In the last decade, laser range finder are moving from indoor to outdoor rural and urban applications for 3D imaging (Yokota et al., 2004), vehicle guidance (Barawid et al., 2007), autonomous navigation (Garcia-Pérez et al., 2008), and objects recognition and classification (Lee & Ehsani, 2008), (Edan & Kondo, 2009), (Katz et al., 2010). Unlike industrial applications, which deal with simple, repetitive and well-defined objects, cameralaser systems on board off-road vehicles require advanced real-time techniques and algorithms to deal with dynamic unexpected objects. Natural environments are complex and loosely structured with great differences among consecutive scenes and scenarios. Vision systems still present severe drawbacks, caused by lighting variability that depends on unpredictable weather conditions. Camera-laser objects feature fusion and classification is still a challenge within the paradigm of artificial perception and mobile robotics in outdoor environments with the presence of dust, dirty, rain, and extreme temperature and humidity. Real time relevant objects perception, task driven, is a main issue for subsequent actions decision in safe unmanned navigation. In comparison with industrial automation systems, the precision required in objects location is usually low, as it is the speed of most rural vehicles that operate in bounded and low structured outdoor environments. To this aim, current work is focused on the development of algorithms and strategies for fusing 2D laser data and visual images, to accomplish real-time detection and classification of unexpected objects close to the vehicle, to guarantee safe navigation. Next, class information can be integrated within the global navigation architecture, in control modules, such as, stop, obstacle avoidance, tracking or mapping.Section 2 includes a description of the commercial vehicle, robot-tractor DEDALO and the vision systems on board. Section 3 addresses some drawbacks in outdoor perception. Section 4 analyses the proposed laser data and visual images fusion method, focused in the reduction of the visual image area to the region of interest wherein objects are detected by the laser. Two methods of segmentation are described in Section 5, to extract the shorter area of the visual image (ROI) resulting from the fusion process. Section 6 displays the colour based classification results of the largest segmented object in the region of interest. Some conclusions are outlined in Section 7, and acknowledgements and references are displayed in Section 8 and Section 9., projects: CICYT- DPI-2006-14497 by the Science and Innovation Ministry, ROBOCITY2030 I y II: Service Robots-PRICIT-CAM-P-DPI-000176- 0505, and SEGVAUTO: Vehicle Safety-PRICIT-CAM-S2009-DPI-1509 by Madrid State Government.

We present the design and first field trial results of an all-day all-weather enhanced and synthetic-fused multi-band colour night vision surveillance and observation system. The system augments a fused and dynamic three-band natural-colour night vision image with synthetic 3D imagery in real-time. The night vision sensor suite consists of three cameras, sensitive in, respectively, the visual (400–700 nm), the near-infrared (NIR, 700–1000 nm) and the long-wave infrared (LWIR, 8–14 mm) bands of the electromagnetic spectrum. The optical axes of the three cameras are aligned. Image quality of the fused sensor signals is enhanced in real-time through dynamic noise reduction, super resolution and local adaptive contrast enhancement. The quality of the LWIR image is enhanced through scene-based non-uniformity correction. The visual and NIR signals are used to represent the fused multi-band night vision image in natural daytime colours, using the Colour-the-Night colour remapping technique. Colour remapping can also be deployed to enhance the visibility of thermal targets that are camouflaged in the visual and NIR range of the spectrum. The dynamic false-colour night-time images are augmented with corresponding synthetic 3D scene views, generated in real-time using a geometric 3D scene model in combination with position and orientation information supplied by the Global Positioning System and inertial sensors of the system. Initial field trials show that this system provides enhanced situational information in various low-visibility conditions.

We present the design and first field trial results of the all-day all-weather INVIS Integrated Night Vision surveillance and observation System. The INVIS augments a dynamic three-band false-color nightvision image with synthetic 3D imagery in a real-time display. The night vision sensor suite consists of three cameras, respectively sensitive in the visual (400-700 nm), the near-infrared (700-1000 nm) and the longwave infrared (8-14 μm) bands of the electromagnetic spectrum. The optical axes of the three cameras are aligned. Image quality of the fused sensor signals can be enhanced in real-time through Dynamic Noise Reduction, Superresolution, and Local Adaptive Contrast Enhancement. The quality of the longwave infrared image can be enhanced through Scene-Based Non-Uniformity Correction (SBNUC), intelligent clustering and thresholding. The visual and near-infrared signals are used to represent the resulting multiband nightvision image in realistic daytime colors, using the Color-the-Night color remapping principle. Color remapping can also be deployed to enhance the visibility of thermal targets that are camouflaged in the visual and near-infrared range of the spectrum. The dynamic false-color nighttime images can be augmented with corresponding synthetic 3D scene views, generated in real-time using a geometric 3D scene model in combination with position and orientation information supplied by the GPS and inertial sensors of the INVIS system. © 2010 SPIE.

We present the design and first field trial results of the all-day all-weather INVIS Integrated Night Vision surveillance and observation System. The INVIS augments a dynamic three-band false-color nightvision image with synthetic 3D imagery in a real-time display. The night vision sensor suite consists of three cameras, respectively sensitive in the visual (400-700 nm), the near-infrared (700-1000 nm) and the longwave infrared (8-14 μm) bands of the electromagnetic spectrum. The optical axes of the three cameras are aligned. Image quality of the fused sensor signals can be enhanced in real-time through Dynamic Noise Reduction, Superresolution, and Local Adaptive Contrast Enhancement. The quality of the longwave infrared image can be enhanced through Scene-Based Non-Uniformity Correction (SBNUC), intelligent clustering and thresholding. The visual and near-infrared signals are used to represent the resulting multiband nightvision image in realistic daytime colors, using the Color-the-Night color remapping principle. Color remapping can also be deployed to enhance the visibility of thermal targets that are camouflaged in the visual and near-infrared range of the spectrum. The dynamic false-color nighttime images can be augmented with corresponding synthetic 3D scene views, generated in real-time using a geometric 3D scene model in combination with position and orientation information supplied by the GPS and inertial sensors of the INVIS system. © 2010 SPIE.

We present a new Tri-band Color Low-light Observation (TRICLOBS) system The TRICLOBS is anall-day all-weather surveillance and navigation tool. Its sensor suite consists of two digital image intensifiers and an uncooled longwave infrared microbolometer. This sensor suite registers the visual, near-infrared and longwave infrared bands of the electromagnetic spectrum. The optical axes of the three cameras are aligned, using two dichroic beam splitters. A fast lookuptable based color transform (the Color-the-Night color mapping principle) is used to represent the TRICLOBS image in natural daylight colors (using information in the visual and NIR bands) and to maximize the detectability of thermal targets (using the LWIR signal). A bottom-up statistical visual saliency mode is deployed in the initial optimization of the color mapping for surveillance and navigation purposes. Extensive observer experiments will result in further optimization of the color representation for a range of different tasks.

We present a new Tri-band Color Low-light Observation (TRICLOBS) system The TRICLOBS is anall-day all-weather surveillance and navigation tool. Its sensor suite consists of two digital image intensifiers and an uncooled longwave infrared microbolometer. This sensor suite registers the visual, near-infrared and longwave infrared bands of the electromagnetic spectrum. The optical axes of the three cameras are aligned, using two dichroic beam splitters. A fast lookuptable based color transform (the Color-the-Night color mapping principle) is used to represent the TRICLOBS image in natural daylight colors (using information in the visual and NIR bands) and to maximize the detectability of thermal targets (using the LWIR signal). A bottom-up statistical visual saliency mode is deployed in the initial optimization of the color mapping for surveillance and navigation purposes. Extensive observer experiments will result in further optimization of the color representation for a range of different tasks.

We present the design and first test results of the TRICLOBS (TRI-band Color Low-light OBServation) system The TRICLOBS is an all-day all-weather surveillance and navigation tool. Its sensor suite consists of two digital image intensifiers (Photonis ICU's) and an uncooled longwave infrared microbolometer (XenICS Gobi 384). The night vision sensor suite registers the visual (400-700 nm), the near-infrared (700-1000 nm) and the longwave infrared (8-14 m) bands of the electromagnetic spectrum. The optical axes of the three cameras are aligned, using two dichroic beam splitters: an ITO filter to reflect the LWIR part of the incoming radiation into the thermal camera, and a B43-958 hot mirror to split the transmitted radiation into a visual and NIR part. The individual images can be monitored through two LCD displays. The TRICLOBS provides both digital and analog video output. The digital video signals can be transmitted to an external processing unit through an Ethernet connection. The analog video signals can be digitized and stored on on-board harddisks. An external processor is deployed to apply a fast lookup-table based color transform (the Color-the-Night color mapping principle) to represent the TRICLOBS image in natural daylight colors (using information in the visual and NIR bands) and to maximize the detectability of thermal targets (using the LWIR signal). The external processor can also be used to enhance the quality of all individual sensor signals, e.g. through noise reduction and contrast enhancement. © 2009 SPIE.

We present the design and first test results of the TRICLOBS (TRI-band Color Low-light OBServation) system The TRICLOBS is an all-day all-weather surveillance and navigation tool. Its sensor suite consists of two digital image intensifiers (Photonis ICU's) and an uncooled longwave infrared microbolometer (XenICS Gobi 384). The night vision sensor suite registers the visual (400-700 nm), the near-infrared (700-1000 nm) and the longwave infrared (8-14 m) bands of the electromagnetic spectrum. The optical axes of the three cameras are aligned, using two dichroic beam splitters: an ITO filter to reflect the LWIR part of the incoming radiation into the thermal camera, and a B43-958 hot mirror to split the transmitted radiation into a visual and NIR part. The individual images can be monitored through two LCD displays. The TRICLOBS provides both digital and analog video output. The digital video signals can be transmitted to an external processing unit through an Ethernet connection. The analog video signals can be digitized and stored on on-board harddisks. An external processor is deployed to apply a fast lookup-table based color transform (the Color-the-Night color mapping principle) to represent the TRICLOBS image in natural daylight colors (using information in the visual and NIR bands) and to maximize the detectability of thermal targets (using the LWIR signal). The external processor can also be used to enhance the quality of all individual sensor signals, e.g. through noise reduction and contrast enhancement. © 2009 SPIE.

We developed a simple and fast lookup-table based method to derive and apply natural daylight colors to multi-band night-time images. The method deploys an optimal color transformation derived from a set of samples taken from a daytime color reference image. The colors in the resulting colorized multiband night-time images closely resemble the colors in the daytime color reference image. Also, object colors remain invariant under panning operations and are independent of the scene content. Here we describe the implementation of this method in two prototype portable dual band realtime night vision systems. One system provides co-aligned visual and near-infrared bands of two image intensifiers, the other provides co-aligned images from a digital image intensifier and an uncooled longwave infrared microbolometer. The co-aligned images from both systems are further processed by a notebook computer. The color mapping is implemented as a realtime lookup table transform. The resulting colorised video streams can be displayed in realtime on head mounted displays and stored on the hard disk of the notebook computer. Preliminary field trials demonstrate the potential of these systems for applications like surveillance, navigation and target detection. Keywords: image fusion, false color, natural color, real-time fusion, lookup tables

We developed a simple and fast lookup-table based method to derive and apply natural daylight colors to multi-band night-time images. The method deploys an optimal color transformation derived from a set of samples taken from a daytime color reference image. The colors in the resulting colorized multiband night-time images closely resemble the colors in the daytime color reference image. Also, object colors remain invariant under panning operations and are independent of the scene content. Here we describe the implementation of this method in two prototype portable dual band realtime night vision systems. One system provides co-aligned visual and near-infrared bands of two image intensifiers, the other provides co-aligned images from a digital image intensifier and an uncooled longwave infrared microbolometer. The co-aligned images from both systems are further processed by a notebook computer. The color mapping is implemented as a realtime lookup table transform. The resulting colorised video streams can be displayed in realtime on head mounted displays and stored on the hard disk of the notebook computer. Preliminary field trials demonstrate the potential of these systems for applications like surveillance, navigation and target detection. Keywords: image fusion, false color, natural color, real-time fusion, lookup tables