Adaptive picture-in-picture technology based on visual saliency

Picture-in-picture (PiP) is a feature of some television receivers and video devices, which allows one main program to be displayed on the full screen while one or more subprogram displayed in inset windows. Currently most TV/video devices require users to specify where and how large to place the sub-program over the main program display. This process is however not user-friendly as it involves a manual process and once specified, the size and the location of the sub-program will be fixed even when they block some key visual information from the main program. We propose an automatic and adaptive PiP technology that makes use of computational modeling of visual saliency. For each frame of the main program, a saliency map is computed efficiently which quantifies how probable a display region of the main program contains useful information and will attract humans' attention/eyes. The sub-program can thus be adaptively resized and placed to the display region that contains the least useful information. Preliminary experiments show the effectiveness of the proposed technology.