Model Based Design of a Real Time FPGA-Based Lens Undistortion and Image Rectification Algorithm for Stereo Imaging

Low cost imaging sensors and powerful embedded computers have taken the field of computer vision to new heights. One of the challenges that remains is to shorten the development time that it takes to target one’s algorithm to hardware. This work details the steps necessary to use Model Based Design to first simulate and then to target a stereo rectification and undistortion algorithm to an embedded FPGA system on chip hardware target. In short, Model Based Design provides a mathematical and visual approach to simulating complex systems as well as the capability to generate executable code from those simulations. This work demonstrates what the future of FPGA-based algorithm development might look like by making use of closed loop design techniques with the aid of novel Model Based Design tool flows. We present the use of hardware-abstracted constructs such as ’virtual cameras’ and the capability of pulling images off the target system and feeding them back into the simulation model to help minimize discrepancies between simulation and hardware domains, ultimately reducing the total development time. The developed system is a deterministic real time and low latency implementation as it does not rely on external off chip memory as all necessary buffering is done in the internal FPGA block RAM structures. The design is capable of processing $752\times 480$ images at pixel line rates and is easily expandable to high resolution images. The algorithm being developed is not being touted as novel, rather that it is sufficiently complex in order to demonstrate the power of the Model Based Design tool flows.