A graphics and vision unified processor with 0.89µW/fps pose estimation engine for augmented reality

In many ways, 3D graphics and vision processing are inverse operations. Graphics processing generates pixels from descriptors, while vision generates descriptors from pixels [1]. Since augmented reality (AR) requires both graphics and vision abilities (Fig. 18.6.1), we report a unified processor for graphics, vision, and pose estimation for marker tracking. To process both graphics and vision simultaneously, we use 4 key features: (1) 6-way VLIW datapath design of processing elements, (2) reconfigurable processing elements for graphics and vision modes, (3) a pixel arranger for vision processing that has the inverse characteristic of a graphics rasterizer, and (4) a dedicated pose-estimation engine to generate graphics control data from vision processing. Using these methods, we achieve 371.9GOPS/W for full operation in a VGA image and 0.89µW/fps for pose estimation.