Search

Image stitching algorithms combine multiple low resolution images and provide a single high resolution composite image with a larger field of view available for video surveillance. In this research work, we put forward and realize real-time panorama composition for a video surveillance application using the power of a GPU. We utilize a cross platform OpenGL graphics library for real-time online image processing. We parallelize panorama composition using OpenGL objects such as texture object, vertex buffer object and framebuffer object for image warping as well as edge blending to create a seamless panoramic image. We divide our panorama composition algorithm into two stages for image sources with fixed relative positions with each other. Initially in an offline stage, we compute inverse lookup maps and feather weight masks using an OpenCV image processing library for each input image. Subsequently in an online stage, we utilize these inverse lookup maps to generate warped images, which are further edge blended with each other using feather weight masks with the help of OpenGL objects. Our panorama composition is more than 8.5 times faster than the CUDA optimized OpenCV realization. It produces a high resolution seamless panoramic image using nine input image streams each with 800×600 image resolution at about more than 75 frames per second using less than 90MB of GPU memory.

In this research work, we propose and realize a real-time stereoscopic vision system for neurosurgery training using a surgical stereoscopic microscope. We connect a pair of high resolution IP cameras to a surgical microscope through a dual port microscope beam splitter and a pair of microscope video adaptors. We make use of the specifically designed chessboard calibration pattern and an open source OpenCV library for system calibration. We perform local calibration to align individual IP camera views with each other, whereas global calibration is carried out to line up local calibrated camera views parallel with the user defined reference coordinate system. We calibrate our stereoscopic system using a single input image of the calibration pattern. We perform real-time stereo image remapping for the comfortable stereoscopic vision using a GPU. A media server is developed and introduced in between the IP camera pair and viewer applications to support multiple stereo image streams as well as to provide one more layer of protection to IP cameras. We make use of a NVIDIA 3D Vision Pro system and a 3D ready 120Hz monitor for active stereoscopic vision. We utilize HDMI-1.4 frame packing for passive stereoscopic vision on a standard 3D-TV. We measure network latency of our stereoscopic system with and without using the media server application, which is within acceptable limits. Finally, we compare NVIDIA 3D Vision Pro based active stereoscopic vision and passive stereoscopic vision using a 3D-TV with their pros and cons.

High resolution displays utilizing an array of commodity projectors are becoming popular approach(also called tiled displays) for visualizing high resolution content like scientific data-sets. These systems provide a relatively cheaper and flexible alternative to displays based on single high resolution monitor or projector. This approach inherently provides a method to create displays with resolution much higher than that possible by using a single high resolution display device. However, bezels in monitor based tiled displays obstruct the geometric continuity of the rendered content. In this paper, we describe the algorithms for geometric alignment of projection regions of an array of arbitrarily placed projectors and attenuation of projected intensities in the overlapping regions of multiple projectors using camera as a feedback device. Combination of these algorithms help us achieve a seamless high resolution display. We have observed improvements in fidelity of the rendered content by utilizing collinearity constraint in geometric alignment calculations. We also propose a novel technique based on cross-ratio invariant for utilizing full projection region of individual projectors which was limited by the size of features used for geometric alignment in the earlier approaches for planar displays. This also results in more imperceptible edge blending artifacts for same physical setup of projectors.

Robots are deployed to perform repeatable as well as hazardous tasks because of their inherent safety, reliability, consistency and accuracy. Recently robots are manipulated remotely using the binocular stereo vision. In this paper, we propose and implement a real-time stereoscopic viewer for telerobotics using open source software. We manipulate the KUKA KR-6 robot remotely using the Phantom haptic device. We assemble our stereo camera using a pair of IP cameras. We make use of the GStreamer multimedia framework for capturing and processing image streams coming from these IP cameras. We utilize the OpenCV library for camera calibration, stereo calibration and real-time stereo rectification. We extend predefined widgets from the Qt framework with our own widgets for the OpenGL based Quad buffering. We apply the software engineering design patterns for development of our in-house stereoscopic viewer. We also discuss the stereo vision using the DLP Link, the NVIDIA 3D Vision and the NVIDIA 3D Vision Pro with their pros and cons.