Search

Image stitching algorithms construct a single composite high resolution image with a larger field of view using multiple low resolution images. Data wall systems combine the power of multiple graphics cards to provide large number of pixels. In this research work, we propose and implement scalable high resolution panorama composition on the data wall system using distributed rendering. We make use of a cross platform OpenGL library for real-time panorama composition. Our algorithm works for image sources with fixed relative positions. Initially, we employ an OpenCV image processing library for panorama calibration. Subsequently, these calibration results are utilized to generate a seamless panoramic image. We construct a high resolution 3×3 data wall system using commodity projectors. We achieve a seamless display with the help of geometric alignment of projector regions and edge blending in overlapping regions. Conventional display devices with their limited image resolution lose details provided by high resolution panorama because of data compression. We integrate panorama composition and the data wall system using distributed rendering. We distribute panorama composition to rendering workstations, which compute their respective screen space bounding regions and operate within it. It provides high resolution panorama with greater details on the data wall system.

Image stitching algorithms combine multiple low resolution images and provide a single high resolution composite image with a larger field of view available for video surveillance. In this research work, we put forward and realize real-time panorama composition for a video surveillance application using the power of a GPU. We utilize a cross platform OpenGL graphics library for real-time online image processing. We parallelize panorama composition using OpenGL objects such as texture object, vertex buffer object and framebuffer object for image warping as well as edge blending to create a seamless panoramic image. We divide our panorama composition algorithm into two stages for image sources with fixed relative positions with each other. Initially in an offline stage, we compute inverse lookup maps and feather weight masks using an OpenCV image processing library for each input image. Subsequently in an online stage, we utilize these inverse lookup maps to generate warped images, which are further edge blended with each other using feather weight masks with the help of OpenGL objects. Our panorama composition is more than 8.5 times faster than the CUDA optimized OpenCV realization. It produces a high resolution seamless panoramic image using nine input image streams each with 800×600 image resolution at about more than 75 frames per second using less than 90MB of GPU memory.

In this research work, we propose and realize a real-time stereoscopic vision system for neurosurgery training using a surgical stereoscopic microscope. We connect a pair of high resolution IP cameras to a surgical microscope through a dual port microscope beam splitter and a pair of microscope video adaptors. We make use of the specifically designed chessboard calibration pattern and an open source OpenCV library for system calibration. We perform local calibration to align individual IP camera views with each other, whereas global calibration is carried out to line up local calibrated camera views parallel with the user defined reference coordinate system. We calibrate our stereoscopic system using a single input image of the calibration pattern. We perform real-time stereo image remapping for the comfortable stereoscopic vision using a GPU. A media server is developed and introduced in between the IP camera pair and viewer applications to support multiple stereo image streams as well as to provide one more layer of protection to IP cameras. We make use of a NVIDIA 3D Vision Pro system and a 3D ready 120Hz monitor for active stereoscopic vision. We utilize HDMI-1.4 frame packing for passive stereoscopic vision on a standard 3D-TV. We measure network latency of our stereoscopic system with and without using the media server application, which is within acceptable limits. Finally, we compare NVIDIA 3D Vision Pro based active stereoscopic vision and passive stereoscopic vision using a 3D-TV with their pros and cons.

In this work, we manipulate an industrial robot from a remote place through a Local Area Network using a force feedback enabled Phantom haptic device as a master, and a stereo vision system for viewing as well as for depth estimation. We propose and implement a stereo vision guided telerobotics system for autonomous pick and place operations. The robot gets the location of pick up from a pre-specified marker, whose position is determined by the stereo vision system. We make use of a pair of IP cameras as the stereo camera, a 3D enabled monitor for viewing stereo contents and an NVIDIA 3D Vision Pro system for managing our stereo vision system. We perform real-time stereo rectification of a stereo image stream for comfortable stereoscopic viewing. We design object markers to support stereo matching up to sub-pixel level. The coordinate systems of the robot and the stereoscopic system are transformed to a pre-defined world coordinate system using a planar chessboard calibration pattern. We evaluate accuracy and repeatability of our stereo vision system on the chessboard plane as well as within the working volume of the robot. Our stereoscopic system is highly accurate with errors less than 2mm along X and Y directions, and below 5mm along Z direction which is direction of viewing.

High resolution displays utilizing an array of commodity projectors are becoming popular approach(also called tiled displays) for visualizing high resolution content like scientific data-sets. These systems provide a relatively cheaper and flexible alternative to displays based on single high resolution monitor or projector. This approach inherently provides a method to create displays with resolution much higher than that possible by using a single high resolution display device. However, bezels in monitor based tiled displays obstruct the geometric continuity of the rendered content. In this paper, we describe the algorithms for geometric alignment of projection regions of an array of arbitrarily placed projectors and attenuation of projected intensities in the overlapping regions of multiple projectors using camera as a feedback device. Combination of these algorithms help us achieve a seamless high resolution display. We have observed improvements in fidelity of the rendered content by utilizing collinearity constraint in geometric alignment calculations. We also propose a novel technique based on cross-ratio invariant for utilizing full projection region of individual projectors which was limited by the size of features used for geometric alignment in the earlier approaches for planar displays. This also results in more imperceptible edge blending artifacts for same physical setup of projectors.

Robots are deployed to perform repeatable as well as hazardous tasks because of their inherent safety, reliability, consistency and accuracy. Recently robots are manipulated remotely using the binocular stereo vision. In this paper, we propose and implement a real-time stereoscopic viewer for telerobotics using open source software. We manipulate the KUKA KR-6 robot remotely using the Phantom haptic device. We assemble our stereo camera using a pair of IP cameras. We make use of the GStreamer multimedia framework for capturing and processing image streams coming from these IP cameras. We utilize the OpenCV library for camera calibration, stereo calibration and real-time stereo rectification. We extend predefined widgets from the Qt framework with our own widgets for the OpenGL based Quad buffering. We apply the software engineering design patterns for development of our in-house stereoscopic viewer. We also discuss the stereo vision using the DLP Link, the NVIDIA 3D Vision and the NVIDIA 3D Vision Pro with their pros and cons.