Search

The image preprocessing optimization is a challenging task with numerous applications including classification and object detection on which this paper is oriented the most. Enhancing the performance in terms of processing time for image preprocessing is crucial to every researcher engaged into deep learning. A few common mistakes and practices are presented in this paper which can greatly impact training time, alongside practices and tools used for diagnosing potential pitfalls. In this paper, we evaluate several common Python libraries which are used for image preprocessing and analyze the impact of different augmentation ordering with respect to central processing unit (CPU) usage, Optimizacija predprocesiranja slike je izazovan zadatak koji zahvaća u niz područja, od kojih se ovaj rad fokusira primarno na klasifikaciju i prepoznavanje objekata na slici. Ubrzavanje performansi predprocesiranja slika je od iznimne važnosti istraživačima koji se bave područjem dubinskog učenja. Kroz ovaj rad prezentirano je nekoliko pogrešaka koje mogu utjecati na vrijeme treniranja modela, s naglaskom na metode dijagnosticiranja potencijalnih zamki. Evaluirano nekoliko poznatih Python biblioteka koje se koriste za predprocesiranje slika te je analiziran utjecaj pojedine popularne augmentacije i njihovog poretka u kontekstu maksimiziranja iskoristivosti resursa središnje procesorske jedinice (CPU).

The need to model visual information with compact representations has existed since the early days of computer vision. We implemented in the past a segmentation and model recovery method for range images which is unfortunately too slow for current size of 3D point clouds and type of applications. Recently, neural networks have become the popular choice for quick and effective processing of visual data. In this article we demonstrate that with a convolutional neural network we could achieve comparable results, that is to determine and model all objects in a given 3D point cloud scene. We started off with a simple architecture that could predict the parameters of a single object in a scene. Then we expanded it with an architecture similar to Faster R-CNN, that could predict the parameters for any number of objects in a scene. The results of the initial neural network were satisfactory. The second network, that performed also segmentation, still gave decent results comparable to the original method, but compared to the initial one, performed somewhat worse. Results, however, are encouraging but further experiments are needed to build CNNs that will be able to replace the state-of-the-art method.

We present a quantitative study of digital signage audience measurement using computer vision. We developed a camera-enhanced digital signage display that acquires audience measurement metrics with computer vision algorithms. Temporal metrics of a person’s dwell time, display in-view time and attention time are extracted. The system also determines demographic metrics of the gender and age group. The digital signage display was deployed in a real-world environment of a clothing boutique, where demographic and viewership data of 1294 store customers were recorded, manually verified and analysed. The analysis shows that 35% of customers specifically looked-at the display, having the average attention time of 0.7 s. Interestingly, the attention time was substantially higher for men (1.2 s) than for women (0.4 s). Age group comparison reveals that children (1–14 years) are the most responsive to the digital signage. Finally, the analysis shows that the average attention time is significantly higher when displaying the dynamic content (0.9 s) when compared with the static content (0.6 s).

The methods used in geology to determine colour and colour coverage are expensive, time consuming, and/ or subjective. Estimates of colour coverage can only be approximate since they are based on rough comparison- based measuring etalons and subjective estimation, which is dependent upon the skill and experience of the person performing the estimation. We present a method which accelerates, simpli es, and objecti es these tasks using a computer application. It automatically calibrates the colours of a digital photo, and enables the user to read colour values and coverage, even after returning from eld work. Colour identi cation is based on the Munsell colour system. For the purposes of colour calibration we use the X-Rite ColorChecker Passport colour chart placed onto the photographed scene. Our computer application detects the ColorChecker colour chart, and nds a colour space transformation to calibrate the colour in the photo. The user can then use the application to read colours within selected points or regions of the photo. The results of the computerised colour calibration were compared to the reference values of the ColorChecker chart. The values slightly deviate from the exact values, but the deviation is around the limit of human capability for visual comparison. We have devised an experiment, which compares the precision of the computerised colour analysis and manual colour analysis performed on a variety of rock samples with the help of geology students using Munsell Rock-color Chart. The analysis showed that the precision of manual comparative identi cation on multi- coloured samples is somewhat problematic, since the choice of representative colours and observation points for a certain part of a sample are subjective. The computer based method has the edge in veri ability and repeatability of the analysis since the application the original photo to be saved with colour calibration, and tagging of colour- analysed points and regions.

A simple and fast method of determining the position of the optical center without any specialized equipment is presented. The position of the optical center is a depth determining parameter in a panoramic depth imaging system [Peer, P., Solina, F., 2002. Panoramic depth imaging: single standard camera approach. Internat. J. Comput. Vision 47 (1/2/3), 149-160; Peer, P., Solina, F., 2005. Multiperspective panoramic depth imaging. In: Computer Vision and Robotics. Nova Science Publishers]. The reconstructed distances correspond well to the actual measured distances on the scene. ne.

In this paper we present a panoramic depth imaging system. The system is mosaic-based which means that we use a single rotating camera and assemble the captured images in a mosaic. Due to a setoff of the camera's optical center from the rotational center of the system we are able to capture the motion parallax effect which enables stereo reconstruction. The camera is rotating on a circular path with a step defined by the angle, equivalent to one pixel column of the captured image. The equation for depth estimation can be easily extracted from the system geometry. To find the corresponding points on a stereo pair of panoramic images the epipolar geometry needs to be determined. It can be shown that the epipolar geometry is very simple if we are doing the reconstruction based on a symmetric pair of stereo panoramic images. We get a symmetric pair of stereo panoramic images when we take symmetric pixel columns on the left and on the right side from the captured image center column. Epipolar lines of the symmetrical pair of panoramic images are image rows. The search space on the epipolar line can be additionaly constrained. The focus of the paper is mainly on the system analysis. Results of the stereo reconstruction procedure and quality evaluation of generated depth images are quite promissing. The system performs well for reconstruction of small indoor spaces. Our finall goal is to develop a system for automatic navigation of a mobile robot in a room.

A classical or static anamorphic image requires a specific, usually a highly oblique view direction, from which the observer can see the anamorphosis in its correct form. This paper explains dynamic anamorphosis which adapts itself to the changing position of the observer so that wherever the observer moves, he sees the same undeformed image. This dynamic changing of the anamorphic deformation in concert with the movement of the observer requires from the system to track the 3D position of the observer’s eyes and the re-computation of the anamorphic deformation in real time. This is achieved using computer vision methods which consist of face detection and tracking the 3D position of the selected observer. An application of this system of dynamic anamorphosis in the context of an interactive art installation is described. We show that anamorphic deformation is also useful for improving eye contact in videoconferencing. Other possible applications involve novel user interfaces where the user can freely move and observe perspectively undeformed images.

We present an approach to reliable and efficient recovery of part-descriptions in terms of superquadric models from range data. We show that superquadrics can directly be recovered from unsegmented data, thus avoiding any presegmentation steps (e.g. in terms of surfaces). The approach is based on the recover-and-select paradigm. We present several experiments on real and synthetic range images, where we demonstrate the stability of the results with respect to viewpoint and noise.

In this paper we present the development of an interactive, content-aware and cost-effective digital signage system. Using a monocular camera installed within the frame of a digital signage display, we employ real-time computer vision algorithms to extract temporal, spatial and demographic features of the observers, which are further used for observer-specific broadcasting of digital signage content. The number of observers is obtained by the Viola and Jones face detection algorithm, whilst facial images are registered using multi-view Active Appearance Models. The distance of the observers from the system is estimated from the interpupillary distance of registered faces. Demographic features, including gender and age group, are determined using SVM classifiers to achieve individual observer-specific selection and adaption of the digital signage broadcasting content. The developed system was evaluated at the laboratory study level and in a field study performed for audience measurement research. Comparison of our monocular localization module with the Kinect stereo-system reveals a comparable level of accuracy. The facial characterization module is evaluated on the FERET database with 95% accuracy for gender classification and 92% for age group. Finally, the field study demonstrates the applicability of the developed system in real-life environments.

Piano Crossing is an interactive art installation which turns a pedestrian crossing marked with white stripes into a piano keyboard so that pedestrians can generate music by walking over it. Matching tones are generated when a pedestrian is over a particular stripe or key. A digital camera is directed at the crossing from above. A special computer vision application was developed that maps the stripes of the pedestrian crossing to piano keys and which detects over which key is the center of gravity of every pedestrian in the image at any given moment. Special black stripes are added to the crossing, which represent also the black piano keys. The application consists of two parts: (1) initialization, where the model of the abstract piano keyboard is mapped to the image of the pedestrian crossing and (2) the detection of pedestrians on the crossing so that musical tones can be generated according to their locations. The art installation Piano crossing was presented to the public for the first time during the 51st Jazz Festival in Ljubljana in July 2010.

In this paper we are trying to establish a framework for natural and man-made scene differentiation based on perspective geometrical properties of the scenes. Although, we have not jet achieved expected results with built classifier, evidence of distinguishing attributes exists and some of them are introduced in this paper.

Segmentation with Volumetric Part Models. Volumetric models are top-level shape representation in computer vision applications. Volumetric models are especially suited for part-level representation on which manipulation, recognition and other reasoning can be based. The two most popular types of volumetric models in computer vision are generalized cylinders and superquadrics. This paper gives an overview of recovery and segmentation methods applying these two types of volumetric models. Methods of segmentation into parts are analyzed and advantageous properties of part-models discussed.

This paper proposes a technique for object recognition using superquadric built models. Superquadrics, which are three-dimensional models suitable for part-level representation of objects, are reconstructed from range images using the recover-and-select paradigm. Using interpretation trees, the presence of an object from the model database can be hypothesized. These hypotheses are verified by projecting and re-fitting the object model to the range image of the scene which at the same time enables a better localization of the object in the scene.

Computer vision is one out of many areas that wants to understand the process of human functionality and copy that process with intention to complement human life with intelligent machines. For better human–computer interaction it is necessary for the machine to see people. This can be achieved by employing face detection algorithms, like the one used in the installation “15 Seconds of Fame”. Mentioned installation unites the areas of modern art and technology. Its algorithm is based on skin colour detection. One of the problems this and similar algorithms have to deal with is sensitivity to the illumination conditions under which the input image is captured. Hence illumination sensitivity influences face detection results. One of the aspects from which we can observe illumination influence is the choice of the proper colour space. Since some colour spaces are designed to eliminate the influence of illumination (brightness) when describing colour of an object, an idea of using such a colour space for skin-colour detection has been taken under consideration and some of the methods have been researched and tested.

Computer vision is one out of many areas that want to understand the process of human functionality and copy that process with intention to complement human life with intelligent machines. For better human-computer interaction it is necessary for the machine to see people. This can be achieved by employing face detection algorithms, like the one used in the installation "15 Seconds of Fame" (F. Solina et al., 2002). Mentioned installation unites the areas of modern art and technology. Its algorithm is based on skin-color detection. One of the problems this and similar algorithms have to deal with is sensitivity to the illumination conditions under which the input image is captured. Hence illumination sensitivity influences face detection results. This problem is being more or less successfully solved by the use of color compensation and color constancy methods. In this work some of these methods are described, realized and tested. Their basic intention is to eliminate the influence of non-standard illumination from images. Tests that were performed showed that methods apply positive influence on face detection results.

We have designed a system with an intuitive user interface for remote camera control and image-based queries over the Internet. While searching for present solutions we realized the importance of a well-designed user interface. We developed a system, which enables remote observation and remote control of the JVC network camera over the Internet. The user interface is based on the combination of live video and a static panoramic view of a remote location. It provides a complete overview of a remote location and significantly simplifies the control over the Internet. By interactively moving a rectangular frame in the panoramic picture, the user locally selects the new direction of the camera. Visual summaries of activities at the observed location can be generated as well as custom queries with a simple user interface over the Internet.

— Cartesian moments are frequently used global geometrical features in computer vision for object pose estimation and recognition. In the paper we derive a closed form expression for 3D Cartesian moment of order p+q+r of a superellipsoid in its canonical coordinate system. We also show how 3D Cartesian moment of a globally deformed superellipsoid in general position and orientation can be computed as a linear combination of 3D Cartesian moments of the corresponding non-deformed superellipsoid in canonical coordinate system. Additionally, moments of objects that are compositions of superellipsoids can be computed as simple sums of moments of individual parts. To demonstrate practical application of the derived results we register pairs of range images based on moments of recovered compositions of superellipsoids. We use a standard technique to find centers of gravity and principal axes in pairs of range images while third-order moments are used to resolve the fourway ambiguity. Experimental results show expected improvement of recovered rigid transformation based on moments of recovered superellipsoids as compared to the registration based on moments of raw range image data. Beside object pose estimation the presented results can be directly used for object recognition with moments and/or moment invariants as object features.

An algorithm is proposed for the detection and separation of specularity from Lambertian reflectance using multiple image frames from different viewpoints. The algorithm is based on the physical model of the multiple reflection mechanism in which the appearance of specularity varies depending on viewing direction while that of Lambertian reflection does not. The approach is to move camera sensors actively and to obtain as many image frames as possible. Experimental results are presented to show the efficiency of the algorithm. >

In this article we present a panoramic depth imaging system. The system is mosaic-based which means that we use a single rotating camera and assemble the captured images in a mosaic. Due to a, setoff of the camera's optical center from the rotational center of the system we are able to capture the motion parallax effect which enables the stereo reconstruction. The camera is rotating on a circular path with the step defined by an angle, equivalent to one column of the captured image. The equation for depth estimation can be easily extracted from system geometry. To find the corresponding points on a stereo pair of panoramic images the epipolar geometry needs to be determined. It can be shown that the epipolar geometry is very simple if we are doing the reconstruction based on a symmetric pair of stereo panoramic images. We get a symmetric pair of stereo panoramic images when we take symmetric columns on the left and on the right side from the captured image center column. Epipolar lines of the symmetrical pair of panoramic images are image rows. We focused mainly on the system analysis. Results of the stereo reconstruction procedure and quality evaluation of generated depth images are quite promising. The system performs well in the reconstruction of small indoor spaces. Our final goal is to develop a system for automatic navigation of a mobile robot in a room.

This paper proposes a technique for object recognition using superquadric built models. Superquadrics, which are three dimensional models suitable for part-level representation of objects, are reconstructed from range images using the recover-and-select paradigm. Using an interpretation tree, the presence of an object in the scene from the model database can be hypothesized. These hypotheses are verified by projecting and re-fitting the object model to the range image which at the same time enables a better localization of the object in the scene.

"15 seconds of fame" is an interactive art installation, which elevates the face of a randomly selected gallery visitor for 15 seconds into a "work of art". The installation was inspired by Andy Warhol's statement that "In the future everybody will be world famous for fifteen minutes" as well as by the pop-art style of his works. The installation consists of a computer with a flat-panel monitor, a digital camera and a proprietary software that can detect human faces in images and graphically transform them. In this paper we present the technical background of the installation, in particular, how computer vision techniques were applied in this art installation.

In this article, we present a panoramic depth imaging system. The system is mosaic-based which means that we use a single rotating camera and assemble the captured images in a mosaic. Due to an offset of the camera's optical center from the rotational center of the system, we are able to capture the motion parallax effect which enables stereo reconstruction. The camera rotates on a circular path with the step defined by an angle equivalent to one column of the captured image. The equation for depth estimation can be easily extracted from the system geometry. To find the corresponding points on a stereo pair of panoramic images the epipolar geometry needs to be determined. We focused mainly on the system analysis. The system performs well in the reconstruction of small indoor spaces.

List of Figures. List of Tables. Preface. Contributing Authors. Introduction A. Leonardis, et al. 1. From images to virtual and augmented reality A. Zisserman, et al. 2. Surface reconstruction from multiple views using apparent contours and surface testure G. Cross, A. Zisserman. 3. Consistent projective reconstruction from multiple views M. Urban, et al. 4. Accurate natural surface reconstruction from polynocular stereo R. Sara. 5. Building models from sensor data: An application shared by the computer vision and the computer graphics community G. Roth. 6. Acquiring range images of objects with non-uniform reflectance using high dynamic scale radiance maps D. Skocaj, A. Leonardis. 7. Dynamic view interpolation without affine reconstruction R.A. Manning, C.R. Dyer. 8. Facial motion capturing using an explanation-based approach H. Tao, Th.S. Huang. 9. Image-based 3D modeling: Modeling from reality L. Van Gool, et al. 10. Computer vision and graphics techniques for modeling dressed humans N. Jojica, T.S. Huang. 11. Urban site models: Accurate, detailed, rapid and inexpensive F.W. Leberl, et al. 12. Medical visualisation, biomechanics, figure animation and robot teleoperation: Themes and links G.J. Clapworthy, et al. 13. Can virtual look real? A review of virtual studio techniques A. Wojdala. 14. Real-time 3D-teleimmersion K. Daniilidis, et al. 15. Augmented reality: A problem in need of many computer vision-based solutions G. Klinker. 16. Registration methods for harmoniousintegration of real world and computer generated objects G. Simon, et al. 17. 3D object tracking using analysis/synthesis techniques A. Gagalowicz, P. Gerard. 18. Augmented reality by integrating multiple sensory modalities for underwater scene understanding V. Murino, A. Fusiello. Index.

Reverse engineering of shape strives to construct CAD models of existing objects. Automatic construction of CAD models from range images requires the segmentation of the 3D data points into subsets that correspond to individual models. We present the results of our approach to segmentation and modeling of range images which enables the reconstruction of a compact geometric representation in the MDL sense which may consist of different types of models.

This paper presents the design and application of a system for live video transmission and remote camera control over the World Wide Web. Extensive testing of the Internet Video Server (IVS) prompted us to improve its user interface. The GlobalView extension of IVS was developed which enables the generation of panoramic images of the environment and a more intuitive control of the camera. The live video frame is superimposed on a 360° static panoramic picture. By interactively moving a rectangular frame in the panoramic picture, the user locally selects the new direction of the camera. Once the view is selected the users prompts the selection and the command is issued over the Internet to the remotely-controlled camera. The static panoramic image is constantly updated in areas where new live video information gets available. Two methods are described for static panoramic image generation: one uses geometric transformation and the other is the brute-force scanning approach. We discuss how visual summaries of activities on an observed location can be generated and custom queries made with a similar intuitive user interface.

A model-based approach to reconstruction of 3D human arm motion from a monocular image sequence taken under orthographic projection is presented. The reconstruction is divided into two stages. First, a 2D shape model is used to track the arm silhouettes and second-order curves are used to model the arm based on an iteratively reweighted least square method. As a result, 2D stick figures are extracted. In the second stage, the stick figures are backprojected into the scene. 3D postures are reconstructed using the constraints of a 3D kinematic model of the human arm. The motion of the arm is then derived as a transition between the arm postures. Applications of these results are foreseen in the analysis of human motion patterns.

Attentive Visual Motion Processing: Computations in the Log-Polar Plane.- Invariant Thinning and Distance Transform.- Recognition of Images Degraded by Linear Motion Blur without Restoration.- Symmetric Bi- and Trinocular Stereo: Tradeoffs between Theoretical Foundations and Heuristics.- Surface from Motion-without and with Calibration.- Properties of Pyramidal Representations.- A Robust Approach to Estimation of Parametric Models.- Computer Vision and Mathematical Morphology.- A Variational Approach to the Design of Early Vision Algorithms.- Banach Constructor and Image Compression.- Piecewise Linear Approximation of Planar Jordan Curves and Arcs: Theory and Applications.- Segmentation with Volumetric Part Models.- Theoretical Foundations of Anisotropic Diffusion in Image Processing.- Stability and Likelihood of Views of Three Dimensional Objects.

We present a novel approach to reliable and efficient recovery of part-descriptions from range images. We show that a set of superquadric models can be directly recovered from unsegmented range data, as opposed to methods which attempt the recovery of volumetric models only after the data has been pre-segmented using extensive pre-processing. The approach is based on the recover-and-select paradigm which consists of two intertwined stages: model-recovery and model-selection. At the model-recovery stage a redundant set of superquadrics is initiated in the image and allowed to grow, which involves an iterative procedure combining data classification and parameter estimation. All the recovered models are passed to the model-selection procedure where only the models resulting in the simplest overall description are selected.

Volumetric part models play an important part in robotic applications such as grasping, path planning, object avoidance, and modeling kinematic chains. The authors present a novel method for reliable and efficient recovery of part-descriptions in terms of superquadric models from range images. In contrast to usual approaches which perform the recovery of volumetric models in several steps (from curves, surfaces to volumes), the authors show that a direct recovery is possible. This is achieved by combining two existing methods: recover-and-select paradigm and recovery of superquadric models. A redundant set of superquadratics is initiated in the image and only the recovered models resulting in the simplest overall description are selected. The author show the results on several real range images. >

We present an approach to a typical task of active sensing, namely how to successively position the sensor in order to accomplish a given task. The approach is data driven since it is only the data that provides the information for planning the next views, and no a priori knowledge about the constituents of the scene is necessary. We applied our approach to the task of determining the 3-D coordinates of vertices of object silhouettes in the image under orthographic projection.

Volumetric models are the top-level representation in computer vision applications. Volumetric models are especially suited for part-level representation on which manipulation, recognition and other reasoning can be based. The two most popular types of volumetric models are generalized cylinders and superquadrics. This paper gives an overview of research and applications of volumetric models in computer vision and robotics with emphasis on superquadrics.

We present in this paper an approach to a typical task of active sensing, namely how to successively position the sensor in order to accomplish a given task. The approach is general in the sense that it does not require a priori information about the constituents of the scene, and data driven in the sense that it is the data that provides the information for driving the planning of the next views. We applied our approach to the task of determining the 3-D coordinates of the points of object silhouettes in the image under orthographic projection.

In this paper we discuss a possible use of camera translation to separate diffuse and specular component of image irradiance. For a moving observer specular reflection appears to “slide” on the object surface. The proposed method is based on the analysis of image sequence obtained during camera translation. By modeling image irradiance as a sum of specular and diffuse component and estimating the motion of the two components we can separate them by filtering the image sequence with two filters.

In this paper we propose an efficient architecture for selective image modeling. We give an example in which models of different scale are reconstructed in parallel. We show that this redundant representation can effectively be pruned using the criterion of {\sf Minimum Description Length}. Models that are selected in the final description indicate the appropriate scale of observation.

When vision is used for moving through the environment, for manipulating or for recognizing objects, it has to simplify the visual input to the level that is required for the specific task. To simplify means to partition images into entities that correspond to individual regions, objects and parts in the real world and to describe those entities only in detail sufficient for performing a required task. For visual discrimination, shape is probably the most important property. After all, line drawings of scenes and objects are usually sufficient for description and subsequent recognition. In computer vision literature this partitioning of images and description of individual parts is called segmentation and shape representation. Segmentation and shape representation appear to be distinct problems and are treated as such in most computer vision systems. In this paper we try to disperse this notion and show that there is no clear division between segmentation and shape representation. Solving any one of those two problems separately is very difficult. On the other hand, if any one of the two problems is solved first, the other one becomes much easier. For example, if the image is correctly divided into parts, the subsequent shape description of those parts gets easier. The opposite is also true when the shapes of parts are known, the partitioning of the image gets simpler.

We propose a modeling system for generic objects in order to recognize different objects from the same category with only one generic model. The representation consists of a prototype, represented by parts and their configuration. Parts are modeled by superquadric volumetric primitives which are combined via Boolean operations to form objects. Variations between objects within a category are described by allowable changes in structure and shape deformations of prototypical parts. Each prototypical part and relation has a set of associated features that can be recognized in the images. These features are used for selecting models from the model data base. The selected hypothetical models are then verified on the geometric level by deforming the prototype in allowable ways to match the data. We base our design of the modeling system upon the current psychological theories of categorization and of human visual perception.

"15 seconds of fame" is an interactive installation which every 15 seconds generates a new pop-art portrait of a randomly selected person from the audience. The installation was inspired by Andy Warhol’s ironical statement that "In the future everybody will be famous for 15 minutes". The installation detects human faces in digital images of people who are standing in front of the installation. Pop-art portraits are then generated from randomly chosen faces in the audience by applying randomly selected filters. These portraits are shown in 15 second intervals on the flat-panel computer monitor which is framed as a painting. Electronic copies of each displayed portrait can be ordered by e-mail.