Realistic image synthesis of a deformable living thing based on motion understanding

This paper describes a system that can synthesize realistic sequential images of moving goldfish based on the image understanding result of real goldfish. To analyze and synthesize images in real-time, we have constructed a hardware system that consists of 32 paralell transputers with a high-speed visual-data interface called VIT (Visual Interface for Transpputer Network). The system is very flexible and powerful for various types of image processing because it can be extended according to the required computational cost. In the understanding process, we assume that the target object, a goldfish in this case, deforms its shape pliably in 3-D space and moves only in a two-dimensional direction. A modeling, called the Bone-Structured Solid Modeler, which is suitable for representing deformable objects such as living things, plays an important role in the understanding and synthesis processes of the deformable object. Three types of constraints for motion, namely, static, dynamic, and object, are utilized to verify the estimated pose and orientation of the object. In the motion synthesis process, realistic moving images are synthesized by controlling the model employing the motion understanding result. Simulation results are presented to show the effectiveness of the system. The technology discussed in this paper is expected to play a key role in the realization of future visual human interfaces.