Search

Light-field and plenoptic cameras are widely available today. Compared with monocular cameras, these cameras capture not only the intensity but also the direction of the light rays. Due to this specificity, light-field cameras allow for image refocusing and depth estimation using a single image. However, most of the existing depth estimation methods using light-field cameras require a prior complex calibration phase and raw data preprocessing before the desired algorithm is applied. We propose a homography-based method with plenoptic camera parameters calibration and optimization, dedicated to our homography-based micro-images matching algorithm. The proposed method works on debayerred raw images with vignetting correction. The proposed approach directly links the disparity estimation in the 2D image plane to the depth estimation in the 3D object plane, allowing for direct extraction of the real depth without any intermediate virtual depth estimation phase. Also, calibration parameters used in the depth estimation algorithm are directly estimated, and hence no prior complex calibration is needed. Results are illustrated by performing depth estimation with a focused light-field camera over a large distance range up to 4 m.

Light-Field (LF) cameras allow the extraction not only of the intensity of light but also of the direction of light rays in the scene, hence it records much more information of the scene than a conventional camera. In this paper, we present a novel method to detect key-points in raw LF images by applying key-points detectors on Pseudo-Focused images (PFIs). The main advantage of this method is that we don’t need to use complex key-points detectors dedicated to light-field images. We illustrate the method in two use cases: the extraction of corners in a checkerboard and the key-points matching in two view raw light-field images. These key-points can be used for different applications e.g. calibration, depth estimation or visual odometry. Our experiments showed that our method preserves the accuracy of detection by re-projecting the pixels in the original raw images.

This guest editorial introduces the Special Section on Quality Control by Artificial Vision VI.

This paper proposes a fully automated vision system to inspect the whole surface of crankshafts, based on the magnetic particle testing technique. Multiple cameras are needed to ensure the inspection of the whole surface of the crankshaft in real-time. Due to the very textured surface of crankshafts and the variability in defect shapes and types, defect detection methods based on deep learning algorithms, more precisely convolutional neural networks (CNNs), become a more efficient solution than traditional methods. This paper reviews the various approaches of defect detection with CNNs, and presents the advantages and weaknesses of each approach for real-time defect detection on crankshafts. It is important to note that the proposed visual inspection system only replaces the manual inspection of crankshafts conducted by operators at the end of the magnetic particle testing procedure.

During the last two decades the number of visual odometry algorithms has grown rapidly. While it is straightforward to obtain a qualitative result, if the shape of the trajectory is in accordance with the movement of the camera, a quantitative evaluation is needed to evaluate the performances and to compare algorithms. In order to do so, one needs to establish a ground truth either for the overall trajectory or for each camera pose. To this end several datasets have been created. We propose a review of the datasets created over the last decade. We compare them in terms of acquisition settings, environment, type of motion and the ground truth they provide. The purpose is to allow researchers to rapidly identifies the datasets that best fit their work. While the datasets cover a variety of techniques to establish a ground truth, we provide also the reader with techniques to create one that were not present among the reviewed datasets.

International audience; In the past few years, a new type of camera has been emerging on the market: a digital camera capable of capturing both the intensity of the light emanating from a scene and the direction of the light rays. This camera technology called a light-field camera uses an array of lenses placed in front of a single image sensor, or simply, an array of cameras attached together. An optical device is proposed: a four minilens ring that is inserted between the lens and the image sensor of a digital camera. This device prototype is able to convert a regular digital camera into a light-field camera as it makes it possible to record four subaperture images of the scene. It is a compact and cost-effective solution to perform both postcapture refocusing and depth estimation. The minilens ring makes also the plenoptic camera versatile; it is possible to adjust the parameters of the ring so as to reduce or increase the size of the projected image. Together with the proof of concept of this device, we propose a method to estimate the positions of each optical component depending on the observed scene (object size and distance) and the optics parameters. Real-world results are presented to validate our device prototype.

Multiple Object Association is a considerable and challenging process in highly cluttered environments. Its aim is to thoroughly relate known objects to new detected ones which is hard in such conditions. The recurrent occlusions and pose variation of objects can raise ambiguity in decision-making. The objective of this paper is to establish a multi-feature fusion approach based on two sources describing, distinctly, the position and the motion direction of considered objects. The proposition is based on Dempster-Shafer theory to model the uncertainty and unreliability of sources and to ensure an evidential combination. Experimental results with real data from the KITTI database are presented to evaluate the proposed solution.

This work shows the interest of combining polarimetric and light-field imaging. Polarimetric imaging is known for its capabilities to highlight and reveal contrasts or surfaces that are not visible in standard intensity images. This imaging mode requires to capture multiple images with a set of different polarimetric filters. The images can either be captured by a temporal or spatial multiplexing, depending on the polarimeter model used. On the other hand, light-field imaging, which is categorized in the field of computational imaging, is also based on a combination of images that allows to extract 3D information about the scene. In this case, images are either acquired with a camera array, or with a multi-view camera such as a plenoptic camera. One of the major interests of a light-field camera is its capability to produce different kind of images, such as sub-aperture images used to compute depth images, full focus images or images refocused at a specific distance used to detect defects for instance. In this paper, we show that refocused images of a light-field camera can also be computed in the context of polarimetric imaging. The 3D information contained in the refocused images can be combined with the linear degree of polarization and can be obtained with an unique device in one acquisition. An example illustrates how these two coupled imaging modes are promising, especially for the industrial control and inspection by vision.

This work shows the interests of the refocusing technics in the domain of industrial vision. A prototype of light field camera is used for computing refocused images, which are calibrated in depth. These images are computed with a method previously presented, using a multi-view camera modeled following the “variable homography” principles. The camera prototype is composed by 4 mini-lenses placed behind a single CCD sensor, calibrated and able to perform 3D measurements. As the device is calibrated in depth, we link refocused images to a selected depth. Contrary to the conventional imaging, with refocused depth calibrated images, it is possible to highlight planes of interest to facilitate vision inspection. Objects can be distinguished from others according to their depths. We also show that a pixel metric scale can be estimated at different depths, avoiding the use of other measurement devices. Two standard vision examples are presented to illustrate the interests of this approach.

International audience; This paper describes an efficient method for the detection of triangular traffic signs on grey-scale images. This method is based on the proposed RSLD (RANSAC Symmetric Lines Detection) algorithm which transforms triangle detection into a simple segment detection. A multi-scale approach allows the detection of any warning and yield traffic signs, whatever their distance to the vehicle. This algorithm is applied to a set of selected corners obtained with a coding gradient method. Baseline detection uses the scale of selected triangles to confirm the presence of traffic signs. The study demonstrates that RSLD is a low computation method compared to standard triangle detection. The performance of the method proposed is compared with recently published methods on road sign databases, which use colour information. An equivalent detection rate is obtained with this algorithm, working on grey-scale images. This algorithm is implemented and runs in real time at 30 frames per second.

In previous works, we have extended the principles of “variable homography”, defined by Zhang and Greenspan, for measuring height of emergent fibers on glass and non-woven fabrics. This method has been defined for working with fabric samples progressing on a conveyor belt. Triggered acquisition of two successive images was needed to perform the 3D measurement. In this work, we have retained advantages of homography variable for measurements along Z axis, but we have reduced acquisitions number to a single one, by developing an acquisition device characterized by 4 lenses placed in front of a single image sensor. The idea is then to obtain four projected sub-images on a single CCD sensor. The device becomes a plenoptic or light field camera, capturing multiple views on the same image sensor. We have adapted the variable homography formulation for this device and we propose a new formulation to calculate a depth with plenoptic cameras. With these results, we have transformed our plenoptic camera in a depth camera and first results given are very promising.

International audience; —This paper presents an object tracking algorithm using belief functions applied to vision-based traffic sign recognition systems. This algorithm tracks detected sign candidates over time in order to reduce false positives due to data fusion formalization. In the first stage, regions of interest (ROIs) are detected and combined using the transferable belief model semantics. In the second stage, the local pignistic probability algorithm generates the associations maximizing the belief of each pairing between detected ROIs and ROIs tracked by multiple Kalman filters. Finally, the tracks are analyzed to detect false positives. Due to a feedback loop between the multi-ROI tracker and the ROI detector, the solution proposed reduces false positives by up to 45%, whereas computation time remains very low.

International audience; — This paper tackles the problem of tracking-based Traffic Sign Recognition (TSR) systems. It presents an integrated object detection, association and tracking approach based on a spatio-temporal data fusion. This algorithm tracks detected sign candidates in order to reduce false positives. Regions Of Interest (ROIs) potentially containing traffic signs are determined from the vehicle-mounted camera images. An original corner detector associated to pixel coding ensures the detection efficiency. The ROIs are combined using the Transfer-able Belief Model semantics. The associations maximizing the pairwise belief between the detected ROIs and ROIs tracked by multiple Kalman filters are processed. The track evolution helps to detect false positives. Thanks to this solution and to a feedback loop between the tracking algorithm and the ROI detector, a false positive reduction of 45% is assessed.

International audience; Industrial textile fabric with functionalization always has a high standard requirement. Take polytetrafluoroethylene (PTFE) conveyor belt as an example, the base fabrics must be of first-rate quality, with no weaving faults or broken fibers on coated surface [1]. In a previous work, we have proposed an original method to measure the height of the fiber based on variable homography. This measurement is based on a single camera acquiring two successive frames. This scheme is working well, but measurement depends on conveyor speed used for scrolling fabric. In this paper, we propose an improvement by using a new acquisition device based on two mini-lenses assembled as a dual-objective. With this device, variable homography modeling leads to 3-D online inspection whatever fabric speed with a simplified calibration procedure compared to classic stereovision approaches.

During foreign operations, Improvised Explosive Devices (IEDs) are one of major threats that soldiers may unfortunately encounter along itineraries. Based on a vehicle-mounted camera, we propose an original approach by image comparison to detect signicant changes on these roads. The classic 2D-image registration techniques do not take into account parallax phenomena. The consequence is that the misregistration errors could be detected as changes. According to stereovision principles, our automatic method compares intensity proles along corresponding epipolar lines by extrema matching. An adaptive space warping compensates scale dierence in 3D-scene. When the signals are matched, the signal dierence highlights changes which are marked in current video.

International audience; Fabric's smoothness is a key factor to determine the quality of finished textile products and has great influence on the functionality of industrial textiles and high-end textile products. With popularization of the 'zero defect' industrial concept, identifying and measuring defective material in the early stage of production is of great interest for the industry. In the current market, many systems are able to achieve automatic monitoring and control of fabric, paper, and nonwoven material during the entire production process, however online measurement of hairiness is still an open topic and highly desirable for industrial applications1. In this paper we propose a computer vision approach to compute epipole by using variable homography, which can be used to measure emergent fiber length on textile fabrics. The main challenges addressed in this paper are the application of variable homography on textile monitoring and measurement, as well as the accuracy of the estimated calculation. We propose that a fibrous structure can be considered as a two-layer structure, and then show how variable homography combined with epipolar geometry can estimate the length of the fiber defects. Simulations are carried out to show the effectiveness of this method. The true length of selected fibers is measured precisely using a digital optical microscope, and then the same fibers are tested by our method. Our experimental results suggest that smoothness monitored by variable homography is an accurate and robust method of quality control for important industrial fabrics.

A fabric's smoothness is a key factor to determine the quality of textile finished products and has great influence on the functionality of industrial textiles and high-end textile products. With popularization of the 'zero defect' industrial concept, identifying and measuring defective material in the early stage of production is of great interest for the industry. In the current market, many systems are able to achieve automatic monitoring and control of fabric, paper, and nonwoven material during the entire production process, however online measurement of hairiness is still an open topic and highly desirable for industrial applications. In this paper we propose a computer vision approach, based on variable homography, which can be used to measure the emergent fiber's length on textile fabrics. The main challenges addressed in this paper are the application of variable homography to textile monitoring and measurement, as well as the accuracy of the estimated calculation. We propose that a fibrous structure can be considered as a two-layer structure and then show how variable homography can estimate the length of the fiber defects. Simulations are carried out to show the effectiveness of this method to measure the emergent fiber's length. The true lengths of selected fibers are measured precisely using a digital optical microscope, and then the same fibers are tested by our method. Our experimental results suggest that smoothness monitored by variable homography is an accurate and robust method for quality control of important industrially fabrics.

In this paper, we propose a new approach for calculating a user’s real point of regard (POR) using a head mounted binocular eye tracking system. We show that the coupling of one of the user’s eyes and the scene camera can be considered as a stereovision system. We then propose a new description for binocular eye trackers, in which we model them via pairs of stereovision systems having the scene camera in common. In this model, we use a hybrid fundamental matrix which allows to take into account the spherical shape of the eyeball model we use. Whatever the distance our model does not require any a priori information on the user’s morphology to calculate the POR.

Our research aims to know where a person is looking at. This information is also used for training purposes. Our eye tracker system can be used in various fields such as learning to drive, support for physically handicapped or as an interface command. Our system consists of a pair of glasses equipped with two cameras: one for the pupil and one for the scene. It can obtain the point of regard in the scene. Starting from the position of the center of the pupil it is possible via a calculation to estimate the point of regard (POR). We have developped a fast method of calibration with a single point. This allows to easily integrate this type of system in training sessions or audit since the calibration takes only a few seconds.

International audience; The main goal of this paper is to present recent improvements of the original optical method dedicated to 3D flows, named R VV (Rainbow Volumic Velocimetry). The authors propose a new way to evaluate the importance of the image quality in flow visualization. A method for generating synthetic images has been developed in order to simulate the RVV technique. With this approach, some factors can be tested such as image noise, image contrast, and spectrum characteristics. Moreover, the performance of image processing tools specifically developed for RVV applications can be also estimated. The principles of both PTV and PSV have been investigated synthetically.

We propose a model of depth camera based on a four-lens device. This device is used for validating alternate approaches for calibrating multi-view cameras and also for computing disparity or depth images. Calibration method arises from previous works, where principles of variable homography were extended for 3-D measurement. In this paper, calibration is performed between two contiguous views obtained on the same image sensor. This approach leads to propose a new approach for simplifying calibration, by using the properties of the variable homography. The second part of this paper addresses new principles for obtaining disparity images without any matching. A fast algorithm using a contour propagation algorithm is proposed without requiring structured or random pattern projection. These principles are proposed in a framework of quality control by vision, for inspection in natural illumination. By preserving scene photometry, some other standard controls, as for example calipers, shape recognition, or barcode reading, can be done conjointly with 3-D measurements. Approaches presented in this paper are evaluated. Firstly, we show that rapid calibration is relevant for devices mounted with multi lenses. Secondly, synthetic and real experimentations validate our method for computing depth images.

Recent studies on driver behaviour have shown that perception - mainly visual but also proprioceptive perception - plays a key role in the "driver-vehicle-road" system and so considerably affects the driver's decision making. The framework of research work presented here is the behaviour analysis and studies low-cost system (BASIL) based on the real time visual analysis tool called EyeAccessPilot (EAP) system (J. Baujon et al., 2001). This system, dedicated to driver's behaviour analysis, collects synchronously all the available embedded information: the visual perception via 2D eye's direction, the trajectory followed, accelerations, etc. In this framework, a new development is presented here in order to allow the analysis of focusing area of a driver in driving situations, via the automatic detection of regions of interest. This post-processing tool considers video sequence acquired with the EyeAccssPilot system. As the displacement of the head is effective during driving phase and is not measured, the aim of this work is to track automatically defined regions of interest (ROI) all along the stored video sequence. This automatic tracker is based on detection of singular points in images.

Recent studies on driver behaviour have shown that perception - mainly visual but also proprioceptive perception plays a key role in the "driver-vehicle-road" system and so considerably affects the driver's decision making. The framework of research work presented here is the behaviour analysis and studies low-cost system (BASIL) based on the real time visual analysis tool called EyeAccessPilot (EAP) system. This system, dedicated to driver's behaviour analysis, collects synchronously all the available embedded information: the visual perception via 2D eye's direction, the trajectory followed, accelerations... In this framework, a new development is presented here in order to allow the analysis of focusing area of a driver in driving situations, via the automatic detection of Regions of Interest. This post-processing tool considers video sequence acquired with the EyeAcessPilot system. As the displacement of the head is effective during driving phase and is not measured, the aim of this work is to track automatically defined Regions Of Interest (ROI) all along the stored video sequence. This automatic tracker is based on detection of singular points in images.

International audience; This paper presents the last development of an original method intended for experimental investigations of 3D flows: the Rainbow Volumic Velocimetry (RVV). In this technique, the flow is seeded with tracers whose positions are recorded by a 3CCD camera connected to a frame grabber. Insted of being lightened in a plan (classical tomography), the flow is lightened in a volume with a continuous spectrum illumination in order to introduce additional information in images. Thereby multicolored images are recorded. The first two spatial coordinates of a tracer are obtained like in classical PTV and the third coordinate is determined thanks to a wavelength coding, the tracer color being directly connected to its depth position.

Split Spectrum Processing (SSP) improves the detection of ultrasonic echoes. Generally, this method uses an algorithm named minimization. One limitation of this processing is the fact that the spectrum of echoes must be known and invariant. That restricts the field of SSP when ultrasound waves are attenuated. In this work, we propose a new scheme for the minimization process, allowing the ultrasonic investigation in absorbing material. We give an example of this method applied to a thermal spray polymer.

Proposes a robust algorithm for the reconstruction of blind strips happening in images generated by a revolutionary new quantum X-ray radiology system. These strips are due to the gaps between the detectors. Each blind strip is three defective columns wide. It may be wider because it happens that the neighbour channels were defective. A brief review of the literature shows that classical interpolation methods are not accurate enough for contour reconstruction. The authors developed a robust image reconstruction method which is contour preserving. Prior to the reconstruction, one must proceed to data preprocessing since quantum detectors related to the channels do not have the same response. Experimental results are presented and compared to polynomial fitting.

International audience; 3-D optical fluorescent microscopy becomes nowadays an efficient tool for volumic investigation of living biological samples. Using optical sectioning technique, a stack of 2-D images is obtained. However, due to the nature of the system optical transfer function and non-optimal experimental conditions, acquired raw data usually suffer from some distortions. In order to carry out biological analysis, raw data have to be restored by deconvolution. The system identification by the point-spread function is useful to obtain the knowledge of the actual system and experimental parameters, which is necessary to restore raw data. It is furthermore helpful to precise the experimental protocol. In order to facilitate the use of image processing techniques, a multi-platform-compatible software package called VIEW3D has been developed. It integrates a set of tools for the analysis of fluorescence images from 3-D wide-field or confocal microscopy. A number of regularisation parameters for data restoration are determined automatically. Common geometrical measurements and morphological descriptors of fluorescent sites are also implemented to facilitate the characterisation of biological samples. An example of this method concerning cytogenetics is presented.

International audience; In this paper a fusion algorithm for the detection of specular echoes blurred by coloured Gaussian noise is proposed. The phase and energy criteria, characteristic of these echoes, are handled separately. First, the detection capabilities of split spectrum (phase) and short time Fourier transform spectrogram (energy) are compared theoretically and experimentally. Second, the new algorithm based on mathematical morphology operators using both time-frequency representations is proposed. In the last part our method is shown to be more robust than absolute minimisation split spectrum processing.

International audience; A split spectrum processing technique using a novel moving bandwidth minimization (MBM) method was developed to detect multiple specular targets having different spectral characteristics. Mathematical morphology (MM) algorithms were also implemented in order to compare the results. An experimental approach to optimal parameter determination is described. These non-linear filtering methods are applied to medical in vivo imaging to illustrate specular detection and signal to noise ratio (SNR) enhancement.

International audience; Object detection in road scenes is an unavoidable task to develop autonomous vehicles and driving assistance systems. Deep neural networks have shown great performances using conventional imaging in ideal cases but they fail to properly detect objects in case of unstable scenes such as high reflections, occluded objects or small objects. Next to that, Polarized imaging, characterizing the light wave, can describe an object not only by its shape or color but also by its reflection properties. That feature is a reliable indicator of the physical nature of the object even under poor illumination or strong reflections. In this paper, we show how polarimetric images, combined with deep neural networks, contribute to enhance object detection in road scenes. Experimental results illustrate the effectiveness of the proposed framework at the end of this paper.

Session: Image Analysis II; International audience; In this research two methods for recognizing water droplets that are formed during dropwise condensation on the flat and pillared substrates are presented. The aim of these methods is to binarize the gray scale images of the droplets taken by a CCD camera in order to extract the information related to the droplets size and density.

This thesis deals with light-field cameras as cameras having 3D capacities. The raw images. acquired with these systems, are generally unusable directly. The main obstacle about their use lies in the complex processing of the recorded images. This thesis aims to overcome these limitations by focusing on multi-views and multi-camera devices. Morcover, as one of the application domains is the industrial vision, the images are acquired in natural lightning in order to conserve the possibility to make conventional treatments by vision on the images. The work is based on three axis: the study and'the optical desien of light-field systems, the calibration of these devices and the development of algorithms to show the intercsts of these cameras in various fields.; Cette thèse traite des caméras light-field en tant que caméra ayant des capacités 3D. Les images brutes, acquises avec ces systèmes, sont généralement inexploitables directement. L’obstacle majeur concernant l'utilisation de ces caméras réside dans la complexité du traitement des images acquises. Cette thèse vise à dépasser ces limitations en s'intéressant aux dispositifs multi-vues et multi-caméras. De plus, comme l'un des domaines d'application envisagé est la vision industrielle, les images sont acquises en lumière naturelle afin de conserver la possibilité d'effectuer des traitements conventionnels par vision sur les images. Le travail de thèse repose sur trois axes : l'étude et la conception optique de systèmes light-field multi-caméras et multi-vues, le calibrage de ces dispositifs et le développement d’algorithmes et enfin leur mise en application pour montrer les intérêts de ces caméras dans divers domaines.