Search

Continuous estimation the driver's take-over readiness is critical for safe and timely transfer of control during the failure modes of autonomous vehicles. In this article, we propose a data-driven approach for estimating the driver's take-over readiness based purely on observable cues from in-vehicle vision sensors. We present an extensive naturalistic driving dataset of drivers in a conditionally autonomous vehicle running on Californian freeways. We collect subjective ratings for the driver's take-over readiness from multiple human observers viewing the sensor feed. Analysis of the ratings in terms of intra-class correlation coefficients (ICCs) shows a high degree of consistency in the ratings across raters. We define a metric for the driver's take-over readiness termed the `Observable Readiness Index (ORI)' based on the ratings. Finally, we propose an LSTM model for continuous estimation of the driver's ORI based on a holistic representation of the driver's state, capturing gaze, hand, pose and foot activity. Our model estimates the ORI with a mean absolute error of 0.449 on a 5 point scale.

Reliable prediction of surround vehicle motion is a critical requirement for path planning for autonomous vehicles. In this paper, we propose a unified framework for surround vehicle maneuver classification and motion prediction that exploits multiple cues, namely, the estimated motion of vehicles, an understanding of typical motion patterns of freeway traffic and intervehicle interaction. We report our results in terms of maneuver classification accuracy and mean and median absolute error of predicted trajectories against the ground truth for real traffic data collected using vehicle mounted sensors on freeways. An ablative analysis is performed to analyze the relative importance of each cue for trajectory prediction. Additionally, an analysis of execution time for the components of the framework is presented. Finally, we present multiple case studies analyzing the outputs of our model for complex traffic scenarios.

A key step to driver safety is to observe the driver's activities with the face being a key step in this process to extracting information such as head pose, blink rate, yawns, talking to passenger which can then help derive higher level information, such as distraction, drowsiness, intent, and where they are looking. In the context of driving safety, it is important for the system perform robust estimation under harsh lighting and occlusion but also be able to detect when the occlusion occurs so that information predicted from occluded parts of the face can be taken into account properly. This paper introduces the Occluded Stacked Hourglass, based on the work of original Stacked Hourglass network for body pose joint estimation, which is retrained to process a detected face window and output 68 occlusion heat maps, each corresponding to a facial landmark. Landmark location, occlusion levels and a refined face detection score, to reject false positives, are extracted from these heat maps. Using the facial landmark locations, features such as head pose and eye/mouth openness can be extracted to derive driver attention and activity. The system is evaluated for face detection, head pose, and occlusion estimation on various datasets in the wild, both quantitatively and qualitatively, and shows state-of-the-art results.

Presents a listing of the senior associates editors for the IEEE Transactions on Intelligent Vehicles.

A mobile platform mounted with omnidirectional vision sensor (ODVS) can be used to monitor large areas and detect interesting events such as independently moving persons and vehicles. To avoid false alarms due to extraneous features, the image motion induced by the moving platform should be compensated. This paper describes a formulation and application of parametric egomotion compensation for an ODVS. Omni images give 360 $^\circ$ view of surroundings but undergo considerable image distortion. To account for these distortions, the parametric planar motion model is integrated with the transformations into omni image space. Prior knowledge of approximate camera calibration and camera speed is integrated with the estimation process using a Bayesian approach. Iterative, coarse-to-fine, gradient-based estimation is used to correct the motion parameters for vibrations and other inaccuracies in prior knowledge. Experiments with a camera mounted on various types of mobile platforms demonstrate successful detection of moving persons and vehicles.

In the future, intelligent rooms, with embedded multimodal sensory systems and semantic event databases, will support effective and efficient transactions of human activities and interactions. We are pursuing rigorous experimental investigations towards the development of such intelligent environments. In this paper we describe the overall system specification and general framework guiding our development. We also present details of the modules associated with the control and interpretation of video information acquired by a network of cameras and a novel semantic event database for characterization and recognition of activities. Evaluations of the system in a custom-built intelligent room are also presented.

The development and evaluation of texture synthesis algorithms is discussed. We present texture synthesis algorithms based on the gray-level co-occurrence (GLC) model of a texture field. These algorithms use a texture similarity metric, which is shown to have high correlation with human perception of textures. Synthesis algorithms are evaluated using extensive experimental analysis. These experiments are designed to compare various iterative algorithms for synthesizing a random texture possessing a given set of second-order probabilities as characterized by a GLC model. Three texture test cases are selected to serve as the targets for the synthesis process in the experiments. The three texture test cases are selected so as to represent three different types of primitive texture: disordered, weakly ordered, and strongly ordered. For each experiment, we judge the relative quality of the algorithms by two criteria. First, we consider the quality of the final synthesized result in terms of the visual similarity to the target texture as well as a numerical measure of the error between the GLC models of the synthesized texture and the target texture. Second, we consider the relative computational efficiency of an algorithm, in terms of how quickly the algorithm converges to the final result. We conclude that a multiresolution version of the “spin flip” algorithm, where an individual pixel's gray level is changed to the gray level that most reduces the weighted error between the images second order probabilities and the target probabilities, performs the best for all of the texture test cases considered. Finally, with the help of psychophysical experiments, we demonstrate that the results for the texture synthesis algorithms have high correlation with the texture similarities perceived by human observers. © 2001 Society of Photo-Optical Instrumentation Engineers.

We present an experimental framework for evaluating metrics for the search and discrimination of a natural texture pattern from its background. Such metrics could help identify preattentive cues and underlying models of search and discrimination, and evaluate and design camouflage patterns and automatic target recognition systems. Human observers were asked to view image stimuli consisting of various target patterns embedded within various background patterns. These psychophysical experiments provided a quantitative basis for comparison of human judgments to the computed values of target distinctness metrics. Two different experimental methodologies were utilized. The first methodology consisted of paired comparisons of a set of stimuli containing targets in a fixed location known to the observers. The observers were asked to judge the relative target distinctness for each pair of stimuli. The second methodology involved stimuli in which the targets were placed in random locations unknown to the observer. The observers were asked to search each image scene and identify suspected target locations. Using a prototype eye tracking testbed, the integrated testbed for eye movement studies, the observers’ fixation points during the experiment were recorded and analyzed. For both experiments, the level of correlation with the psychophysical data was used as the basis for evaluating target distinctness metrics. Overall, of the set of target distinctness metrics considered, a metric based on a model of image texture was the most strongly correlated with the psychophysical data. © 2001 Society of Photo-Optical Instrumentation Engineers.

Abstract.: An accurate analysis of a large dynamic system like our oceans requires spatially fine and temporally matched data collection methods. Current methods to estimate fish stock size from pelagic (marine) fish egg abundance by using ships to take point samples of fish eggs have large margins of error due to spatial and temporal undersampling. The real-time flow imaging and classification system (REFLICS) enhances fish egg sampling by obtaining continuous, accurate information on fish egg abundance as the ship cruises along in the area of interest. REFLICS images the dynamic flow with a progressive-scan area camera (60 frames/s) and a synchronized strobe in backlighting configuration. Digitization and processing occur on a dual-processor Pentium II PC and a pipeline-based image-processing board. REFLICS uses a segmentation algorithm to locate fish-egg-like objects in the image and then a classifier to determine fish egg, species, and development stage (age). We present an integrated system design of REFLICS and performance results. REFLICS can perform in real time (60 Hz), classify fish eggs with low false negative rates on real data collected from a cruise, and work in harsh conditions aboard ships at sea. REFLICS enables cost-effective, real-time assessment of pelagic fish eggs for research and management.

Texture is an important cue in region-based segmentation of images. In this paper, we provide insight into the development of a new set of distortion-invariant texture operators. These “circular-Mellin” operators are invariant to both scale and orientation of the target and represent the spectral decomposition of the image scene in the polar-log coordinate system. Coupled with the unique shift invariance property of the correlator architecture, we show that these circular-Mellin operators can be used for rotation-and scale-invariant feature extraction. We note that while these feature extractors have a functional form that is similar to the Gabor operators, they have distortion-invariant characteristics unlike the Gabor functions that make them more suitable for texture segmentation. A detailed analytical description of these operators and segmentation results to highlight their salient properties are presented.

The goal of this research is to find signature strength metrics that correspond closely to the distinctness of a target as perceived by humans, especially for camouflaged targets. We consider metrics that attempt to measure the strength of three perceptual cues: contrast, texture differences, and boundary strength. The contrast cue is measured with first-order metrics, differences in texture are measured with a second-order metric, and boundary strength is measured by computing contrast along the target-background boundary. We discuss a psychophysical experiment designed to generate quantitative measurements of perceived target distinctness for comparison with the target signature strength metrics. This experiment involves paired comparisons of image stimuli, each containing a single random target pattern embedded at a known location in a random background pattern. The data from the psychophysical experiment are compared with computed values of the target signature strength metrics, and a second-order image texture metric was found to exhibit the strongest correlation with the human data. © 1998 Society of Photo-Optical Instrumentation Engineers.

In the emerging paradigm of animate vision, the visual processes are not thought of as being independent of cognitive or motor processing, but as an integrated system within the context of visual behavior. Intimate coupling of sensory and motor systems have found to improve significantly the performance of behavior based vision systems. In order to study active vision systems one requires sensory-motor systems. Designing, building, and operating such a test bed is a challenging task. In this paper we describe the status of on-going work in developing a sensory-motor robotic system, R2H, with ten degrees of freedoms (DOF) for research in active vision. To complement the R2H system a Graphical Simulation and Animation (GSA) environment is also developed. The objective of building the GSA system is to create a comprehensive design tool to design and study the behavior of active systems and their interactions with the environment. GSA system aids the researchers to develop high performance and reliable software and hardware in a most effective manner. The GSA environment integrates sensing and motor actions and features complete kinematic simulation of the R2H system, it's sensors and it's workspace. With the aid of the GSA environment a Depth from Focus (DFF), Depth from Vergence, and Depth from Stereo modules are implemented and tested. The power and usefulness of the GSA system as a research tool is demonstrated by acquiring and analyzing images in the real and virtual worlds using the same software implemented and tested in the virtual world.

Abstract not available.

Abstract not available

Abstract not available

Abstract not available.