Search

In Europe, 20% of road crashes occur at intersections. In recent years, evolving communication technologies are making V2V and V2I faster and more reliable; with such advancements, these crashes, as well as their economic cost, can be partially reduced. In this work, we concentrate on straight path intersection collisions. Connectivity-based algorithms relying on 5G technology and smart sensors are presented and compared to a commercial radar AEB logic in order to evaluate performances and effectiveness in collision avoidance or mitigation. The aforementioned novel safety systems are tested in a blind intersection and low adherence scenario. The first algorithm proposed is obtained by incorporating connectivity information to the original control scheme, while the second algorithm proposed is a novel control logic fully capable of utilizing also adherence estimation provided by smart sensors. Test results show an improvement in terms of safety for both the architecture and high prospects for future developments.

Introduction. Automotive radars are the main tools for providing traffic safety. The development of such radars involve a number of technical difficulties due to the manufacture of high-precision extremely high-frequency (EHF) printed circuit boards. To facilitate the process of creating such devices, the existing algorithms for radar information processing should be debugged using prototypes from manufacturers of mm-band transceivers. However, the parameters of such boards are not known in advance, and the actual operating conditions of the as-produced automotive radars raise new challenges to target tracking algorithms. Therefore, checking the performance of such boards is a relevant research problem.Aim. To evaluate the performance of a millimeter-wave automotive radar prototype and to test target tracking algorithms using this prototype.Materials and methods. An original target tracking method was used, which considers the constraints on the use of additional data sources about the radar carrier movement.Results. An experimental performance evaluation of a 77 GHz automotive radar prototype was carried out. The effectiveness of primary processing for the target class “vehicle” in the millimetre range was checked. Original algorithms for target tracking were proposed and tested.Conclusion. The obtained results show that the prototype board of a transceiver chip is capable of testing tracking algorithms without creating an own automotive radar prototype. Thus, the developmental process can be significantly shortened. Moreover, after creating a hardware solution, the developer obtains a reference device to test and configure an own product without using extremely expensive and rare EHF equipment.

Safety-critical systems such as Advanced Driving Assistance Systems and Autonomous Vehicles require redundancy to satisfy their safety requirements and to be classified as fail-operational. Introducing redundancy in a system with high data rates and processing requirements also has a great impact on architectural design decisions. The current self-driving vehicle prototypes do not use a standardized system architecture but base their design on existing vehicles and the available components. In this work, we provide a novel analysis framework that allows us to qualitatively and quantitatively evaluate an in-vehicle architecture topology and compare it with others.With this framework, we evaluate different variants of two common topologies: domain and zone-based architectures. Each topology is evaluated in terms of total cost, failure probability, total communication cable length, communication load distribution, and functional load distribution.We introduce redundancy in selected parts of the systems using our automated process provided in the framework, in a safety-oriented design process that enables the ISO26262 Automotive Safety Integrity Level decomposition technique. After every design step, the architecture is re-evaluated. The advantages and disadvantages of the different architecture variants are evaluated to guide the designer towards the choice of correct architecture, with a focus on the introduction of redundancy.

Alzheimer's Disease Assessment Scale-Cognitive subscale (ADAS-Cog) is a neuropsychological tool that has been designed to assess the severity of cognitive symptoms of dementia. Personalized prediction of the changes in ADAS-Cog scores could help in the timing of therapeutic interventions in dementia and at-risk populations. In the present work, we compared single- and multi-task learning approaches to predict the changes in ADAS-Cog scores based on T1-weighted anatomical magnetic resonance imaging (MRI). In contrast to most machine learning-based methods to predict the changes in ADAS-Cog, we stratified the subjects based on their baseline diagnoses and evaluated the prediction performances in each group. Our experiments indicated a positive relationship between predicted and observed ADAS-Cog score changes in each diagnostic group suggesting that T1-weighted MRI has predictive value for evaluating cognitive decline in the whole AD continuum. We further studied whether correction of the differences in magnetic field strength of MRI would improve ADAS-Cog score prediction. The partial least square-based domain adaptation improved slightly prediction performance, but the improvement was marginal. In sum, this study demonstrated that ADAS-Cog change can be, to some extent, predicted based on anatomical MRI. Based on this study, the recommended method for learning the predictive models is a single-task regularized linear regression owing to its simplicity and good performance. It appears important to combine the training data across all subject groups for the most effective predictive models. publishedVersion

Strengthening road safety in the face of the enormous development of the automotive in recent decades is crucial. The safety benefits of automated vehicles are paramount.Automated vehicles have the potential to remove human error in road traffic, which will help protect drivers and passengers, as well as pedestrians and bicyclists.The carried-outforecasts are pioneering for Polish road traffic conditions.In England, studies have been carried out to determine the estimated impact of autonomous vehicles on road safety in simulated traffic conditions on the motorway.In Poland, preliminary forecasts of the reduction in the number of road accidents were made;however,they were based on other assumptions. Therefore, estimating the impact of using autonomous vehicles in order to increase the level of road safety is an innovative activity for Polish road conditions.For the purposes of this article, available statistical data on vehicles registered in Poland, their equipment with advanced driver-assistance systems as well asaccident data and their causes were analyzed.A diagnosis of Road Safety in Poland in 2018(base year for further estimations)was made, taking into account the trend of recent years together with an indication of the most common causes of road accidents.These data were compiled with statistical data from other countries about the influence of driver-support systems on traffic safety. Possible potential for increasing Road Safety in Poland by the year 2030 was estimated. The analyses were prepared assuming different types of processes related to traffic, road safety,and the recent development of the passenger car fleet in Poland. Presented results show four scenarios of road safety change, where the number of accidents is reduced with statistical average of 5000 reduction in the year 2030.These expectationsare based on various predictable factors connected with upgrade of car fleet quality and take into account changes in road safety observed in recent years. Based on the current trend of driving automation and rapid development of driver-support systems, the provided estimations were found reliable and likely.The conducted research shows the benefits and the need for the use of driver-assistance systems in vehicles as they can measurably affect the level of road safety.

The advent of machine learning (ML) methods for the industry has opened new possibilities in the automotive domain, especially for Advanced Driver Assistance Systems (ADAS). These methods mainly focus on specific problems ranging from traffic sign and light recognition to pedestrian detection. In most cases, the computational resources and power budget found in ADAS systems are constrained while most machine learning methods are computationally intensive. The usual solution consists in adapting the ML models to comply with the memory and real-time (RT) requirements for inference. Some models are easily adapted to resource-constrained hardware, such as Support Vector Machines, while others, like Neural Networks, need more complex processes to fit into the desired hardware. The ADAS hardware (HW platforms) are diverse, from complex MPSoC CPUs down to classical MCUs, DPSs and application-specific FPGAs and ASICs or specific GPU platforms (such as the NVIDIA families Tegra or Jetson). Therefore, there is a tradeoff between the complexity of the ML model implemented and the selected platform that impacts the performance metrics: function results, energy consumption and speed (latency and throughput). In this paper, a survey in the form of systematic review is conducted to analyze the scope of the published research works that embed ML models into resource-constrained implementations for ADAS applications and what are the achievements regarding the ML performance, energy and speed trade-off.

Providing ground truth supervision to train visual models has been a bottleneck over the years, exacerbated by domain shifts which degenerate the performance of such models. This was the case when visual tasks relied on handcrafted features and shallow machine learning and, despite its unprecedented performance gains, the problem remains open within the deep learning paradigm due to its data-hungry nature. Best performing deep vision-based object detectors are trained in a supervised manner by relying on human-labeled bounding boxes which localize class instances ( i.e. objects) within the training images. Thus, object detection is one of such tasks for which human labeling is a major bottleneck. In this article, we assess co-training as a semi-supervised learning method for self-labeling objects in unlabeled images, so reducing the human-labeling effort for developing deep object detectors. Our study pays special attention to a scenario involving domain shift; in particular, when we have automatically generated virtual-world images with object bounding boxes and we have real-world images which are unlabeled. Moreover, we are particularly interested in using co-training for deep object detection in the context of driver assistance systems and/or self-driving vehicles. Thus, using well-established datasets and protocols for object detection in these application contexts, we will show how co-training is a paradigm worth to pursue for alleviating object labeling, working both alone and together with task-agnostic domain adaptation.

The goal of this research is the development of a driver assistant feature, which can warn the driver in case a pedestrian is in a potential risk due to sudden intention to cross the road. The process of crossing pedestrian is defined as the changing of pedestrian orientation on the curb toward the road. We built a Convolutional Neural Network (CNN) model combined with depth sensing camera to estimate the pedestrian orientation and distance from the vehicle. The model detects the higher human body keypoints in 2D space while the depth info make it possible to translate the points into a 3D space. These info are tracked per pedestrian and any change in the pedestrian moving pattern toward the road is translated to a warning for the driver. The CNN model is end-end trained using different datasets presenting pedestrian in different configurations and scenes.

Safety critical systems in Advanced Driver Assistance Systems (ADAS) depend on multiple sensors to perceive the environment in which they operate. Radar sensors provide many advantages and complementary capabilities to other available sensors but are not without their own shortcomings. Performance of radar perception algorithms still pose many challenges, one of which is in object detection and classification. In order to increase redundancy in ADAS, the ability for a radar system to detect and classify objects independent of other sensors is desirable. In this paper, an investigation of a machine learning based radar perception algorithm for object detection is implemented, along with a novel, automated workflow for generating large-scale virtual datasets used for training and testing. Physics-based electromagnetic simulation of a complex scattering environment is used to create the virtual dataset. Objects are classified and localized within Doppler-Range results using a single channel 77 GHz FMCW radar system. Utilizing a fully convolutional network, the radar perception model is trained and tested. The training is performed using a wide range of environments and traffic scenarios. Model inference is tested on completely new environments and traffic scenarios. These simulated radar returns are highly scalable and offer an efficient method for dataset generation. These virtual datasets facilitate a simple method of introducing variability in training data, corner case evaluation and root cause analysis, amongst other advantages.

Pedestrian detection has never been an easy task for computer vision and the automotive industry. Systems like the advanced driver-assistance system (ADAS) highly rely on far-infrared (FIR) data captured to detect pedestrians at nighttime. The recent development of deep learning-based detectors has proven the excellent results of pedestrian detection in perfect weather conditions. However, it is still unknown what the performance in adverse weather conditions is. In this paper, we introduce a 16-bit thermal data dataset called ZUT (Zachodniopomorski Uniwersytet Technologiczny) as having the widest variety of fine-grained annotated images captured in the four biggest European Union countries captured during severe weather conditions. We also provide a synchronized Controller Area Network (CAN bus) data, including driving speed, brake pedal status, and outside temperature for future ADAS system development. Furthermore, we have tested and provided 16-bit depth modifications for the YOLOv3 deep neural network (DNN) based detector, reaching a mean Average Precision (mAP) up to 89.1%. The ZUT dataset is published and publicly available at IEEE Dataport and Github.

International audience; Shielded cables are an important mitigation for electromagnetic interference (EMI) in high-speed data systems. In the automotive domain, one use for them is to transmit image data from a front camera to an advanced driver assistance system (ADAS) controller. Some ADAS functions have implication for human safety and thus place extra requirements for the design of the transmission path including its resilience to EMI. This paper presents a case study of an automated lane centering (ALC) system with the above-mentioned shielded cable use case. The study starts from a National Highway Traffic Safety Administration (NHTSA) concept level assessment. Subsystem components are then separated and a physical realization derived. Goal Structuring Notation (GSN) is used to present EMI assurance scenarios over the safety requirements. First, the ability of a shielded cable reliability argument to cover the derived safety requirements during different operating scenarios is studied. It is found that relying on reliability alone, it is challenging to fulfil all the safety requirements. To overcome this challenge, an alternative systems safety based method is studied.

There are over four million miles of roads in the United States, and the prioritization of locations to perform maintenance activities typically relies on human inspection or semi-automated dedicated vehicles. Pavement markings are used to delineate the boundaries of the lane the vehicle is driving within. These markings are also used by original equipment manufacturers (OEM) for implementing advanced safety features such as lane keep assist (LKA) and eventually autonomous operation. However, pavement markings deteriorate over time due to the fact of weather and wear from tires and snowplow operations. Furthermore, their performance varies depending upon lighting (day/night) as well as surface conditions (wet/dry). This paper presents a case study in Indiana where over 5000 miles of interstate were driven and LKA was used to classify pavement markings. Longitudinal comparisons between 2020 and 2021 showed that the percentage of lanes with both lines detected increased from 80.2% to 92.3%. This information can be used for various applications such as developing or updating standards for pavement marking materials (infrastructure), quantifying performance measures that can be used by automotive OEMs to warn drivers of potential problems with identifying pavement markings, and prioritizing agency pavement marking maintenance activities.

In recent years, the automotive field has been changed by the accelerated rise of new technologies. Specifically, autonomous driving has revolutionized the car manufacturer's approach to design the advanced systems compliant to vehicle environments. As a result, there is a growing demand for the development of intelligent technology in order to make modern vehicles safer and smarter. The impact of such technologies has led to the development of the so-called Advanced Driver Assistance Systems (ADAS), suitable to maintain control of the vehicle in order to avoid potentially dangerous situations while driving. Several studies confirmed that an inadequate driver's physiological condition could compromise the ability to drive safely. For this reason, assessing the car driver's physiological status has become one of the primary targets of the automotive research and development. Although a large number of efforts has been made by researchers to design safety-assessment applications based on the detection of physiological signals, embedding them into a car environment represents a challenging task. These mentioned implications triggered the development of this study in which we proposed an innovative pipeline, that through a combined less invasive Neuro-Visual approach, is able to reconstruct the car driver's physiological status. Specifically, the proposed contribution refers to the sampling and processing of the driver PhotoPlethysmoGraphic (PPG) signal. A parallel enhanced low frame-rate motion magnification algorithm is used to reconstruct such features of the driver's PhotoPlethysmoGraphic (PPG) data when that signal is no longer available from the native embedded sensor platform. A parallel monitoring of the driver's blood pressure levels from the PPG signal as well as the driver's eyes dynamics completes the reconstruction of the driver's physiological status. The proposed pipeline has been tested in one of the major investigated automotive scenarios i.e., the detection and monitoring of pedestrians while driving (pedestrian tracking). The collected performance results confirmed the effectiveness of the proposed approach.

Vehicle automation and assistance technologies have been touted as a means by which to reduce traffic collisions by minimizing or eliminating “error-prone” and inefficient human operators. In concept, automation exists on a continuum that includes engaged driving by a human operator augmented by automated support features, vigilant driver monitoring of vehicle behavior with the possibility of driver take-over, to full automation with no active monitoring by a human operator. Moreover, the degree of automation varies by vehicle features (e.g., lane centering, emergency braking, adaptive cruise control, parking), by setting, meaning that automated features may or may not be available depending on specific attributes of the traffic environment (e.g., traffic volume, road geometry, etc), and by implementation (e.g., haptic vs auditory warnings). Thus, these automotive “transportation tools” are highly heterogeneous and pose unique challenges and opportunities for driver training. In this paper, we report the results of an experimental study (n=36) to determine if enhanced vehicle feedback influences driver trust, effort, frustration, and performance (indexed by reaction time) in a virtual driving environment. Results are contextualized in the extant literature on learning to operate motor vehicles and outline key research questions essential for understanding the processes by which skilled performance develops with respect to a real-world practical tool: the increasingly automated automobile.

Future automotive systems, with Advanced Driving Assistance Systems and Autonomous Driving functionalities, will require fail-operational electronic systems. To achieve that, redundancy is a necessary technique, like in many other fields such as aviation. Moreover the applications have different safety requirements, from safety-critical related applications, for example for the driver replacement domain, to QoS-oriented applications, for example for the infotainment domain. Redundancy in mixed-criticality systems can be solved by physically separating system resources or by using isolated virtualized environments with e.g. hypervisors. There are costs associated to both solutions. In this work we describe a novel model we use to characterize a mixed-criticality automotive system and the analysis steps to obtain quantified metrics. The quantified metrics include cost, failure probability, total functional and communication loads, and total cable length, to compare the different solutions from a system-level perspective. We analyse the same set of mixed-criticality applications that represent a simplified automotive system in four scenarios. The architecture topology is either domain-based or zone-based, and we use either physical separation or virtualization to provide isolation. The obtained results show how the model and the analysis allows us to understand the trade-offs between the different solutions in specific applications scenarios, and how to vary the metrics used in the analysis to adapt to a different applications scenario.

In contrast to previous works on data augmentation using LIDAR (Light Detection and Ranging), which mostly consider point clouds under good weather conditions, this paper uses point clouds which are affected by spray. Spray water can be a cause of phantom braking and understanding how to handle the extra detections caused by it is an important step in the development of ADAS (Advanced Driver Assistance Systems)/AV (Autonomous Vehicles) functions. The extra detections caused by spray cannot be safely removed without considering cases in which real solid objects may be present in the same region in which the detections caused by spray take place. As collecting real examples would be extremely difficult, the use of synthetic data is proposed. Real scenes are reconstructed virtually with an added extra object in the spray region, in a way that the detections caused by this obstacle match the characteristics a real object in the same position would have regarding intensity, echo number and occlusion. The detections generated by the obstacle are then used to augment the real data, obtaining, after occlusion effects are added, a good approximation of the desired training data. This data is used to train a classifier achieving an average F-Score of 92. The performance of the classifier is analyzed in detail based on the characteristics of the synthetic object: size, position, reflection, duration. The proposed method can be easily expanded to different kinds of obstacles and classifier types.

Top-performing computer vision models are powered by convolutional neural networks (CNNs). Training an accurate CNN highly depends on both the raw sensor data and their associated ground truth (GT). Collecting such GT is usually done through human labeling, which is time-consuming and does not scale as we wish. This data-labeling bottleneck may be intensified due to domain shifts among image sensors, which could force per-sensor data labeling. In this paper, we focus on the use of co-training, a semi-supervised learning (SSL) method, for obtaining self-labeled object bounding boxes (BBs), i.e., the GT to train deep object detectors. In particular, we assess the goodness of multi-modal co-training by relying on two different views of an image, namely, appearance (RGB) and estimated depth (D). Moreover, we compare appearance-based single-modal co-training with multi-modal. Our results suggest that in a standard SSL setting (no domain shift, a few human-labeled data) and under virtual-to-real domain shift (many virtual-world labeled data, no human-labeled data) multi-modal co-training outperforms single-modal. In the latter case, by performing GAN-based domain translation both co-training modalities are on par, at least when using an off-the-shelf depth estimation model not specifically trained on the translated images.

This paper presents a vision-based collision-warning system for ADAS in intelligent vehicles, with a focus on urban scenarios. In most current systems, collision warnings are based on radar, or on monocular vision using pattern recognition. Since detecting collisions is a core functionality of intelligent vehicles, redundancy is essential, so that we explore the use of stereo vision. First, our approach is generic and class-agnostic, since it can detect general obstacles that are on a colliding path with the ego-vehicle without relying on semantic information. The framework estimates disparity and flow from a stereo video stream and calculates stixels. Then, the second contribution is the use of the new asteroids concept as a consecutive step. This step samples particles based on a probabilistic uncertainty analysis of the measurement process to model potential collisions. Third, this is all enclosed in a Bayesian histogram filter around a newly introduced time-to-collision versus angle-of-impact state space. The evaluation shows that the system correctly avoids any false warnings on the real-world KITTI dataset, detects all collisions in a newly simulated dataset when the obstacle is higher than 0.4m, and performs excellent on our new qualitative real-world data with near-collisions, both in daytime and nighttime conditions.

Vanishing point (VP) provides extremely useful information related to roads in driving scenes for advanced driver assistance systems (ADAS) and autonomous vehicles. Existing VP detection methods for driving scenes still have not achieved sufficiently high accuracy and robustness to apply for real-world driving scenes. This paper proposes a robust motion-based road VP detection method to compensate for the deficiencies. For such purposes, three main processing steps often used in the existing road VP detection methods are carefully examined. Based on the analysis, stable motion detection, stationary point-based motion vector selection, and angle-based RANSAC (RANdom SAmple Consensus) voting are proposed. A ground-truth driving dataset including various objects and illuminations is used to verify the robustness and real-time capability of the proposed method. The experimental results show that the proposed method outperforms the existing motion-based and edge-based road VP detection methods for various illumination conditioned driving scenes.

There has been a significant change in trends in the automotive industry over the last ten years. Vehicles equipped with different Advanced Driver Assistance Systems (ADAS) are increasingly present on the roads, whereas the level of their autonomy increases over time. For autonomous vehicles to work properly, it is necessary to have reliable ADAS that process different input signals from distinct sensors. One of the most important ADAS algorithms is that intended for lane detection (LD) and lane departure warning (LDW). In this paper, a new algorithm for LD on the road and LDW, which processes only images obtained from the camera located at the front end of the vehicle, is proposed. The algorithm provides information on the number of detected lane lines in the image and their position on the image while marking the current driving lane and the first two adjacent lanes if they exist. If the vehicle is departing from the lane, a corresponding warning message is shown to the driver. The algorithm was tested on a set of 12 video sequences (17552 frames in total) recorded during day and night in different weather conditions. The results showed that the algorithm achieves high performance in most cases, while for some challenging cases there is room for further improvement.

Examining the blind spot of the vehicle while driving using the vehicle mirror and looking over the shoulder causes a critical time when the driver does not have an overview of the environment in front of the vehicle, which can be the cause of traffic accidents. For that reason, this paper presents the implementation of the solution for stitching multiple frames with the overlap area, obtained by vehicle cameras with the aim of increasing the visibility of the environment behind the vehicle and eliminating the need for a classic view over the shoulder. A real ADAS (Advanced Driver-Assistance System) development board and the three real automotive cameras were mounted on a small vehicle model and were used to create test videos. After frame stitching, correcting the brightness of the stitched frame is also implemented to achieve a higher quality of final output frame. The performed subjective quality evaluation results using 10 neutral viewers confirm that the proposed solution achieves high performance in terms of the quality of the output image. However, the results related to the measured execution time of the proposed solution show that it cannot achieve a real-time performance in its current form when implemented on real ADAS board. Therefore, for the proposed solution the intention is to use the Vehicle-to- Infrastructure (V2I) communication and remote data processing center which should process more demanding tasks of the solution.

Most of the tactic manoeuvres during driving require a certain understanding of the surrounding environment from which to devise our future behaviour. In this paper, a Convolutional Neural Network (CNN) approach is used to model the lane change behaviour to identify when a driver is going to perform this manoeuvre. To that end, a slightly modified CNN architecture adapted to both spatial (i.e., surrounding environment) and non-spatial (i.e., rest of variables such as relative speed to the front vehicle) input variables. Anticipating a driver&rsquo, s lane change intention means it is possible to use this information as a new source of data in wide range of different scenarios. One example of such scenarios might be the decision making process support for human drivers through Advanced Driver Assistance Systems (ADAS) fed with the data of the surrounding cars in an inter-vehicular network. Another example might even be its use in autonomous vehicles by using the data of a specific driver profile to make automated driving more human-like. Several CNN architectures have been tested on a simulation environment to assess their performance. Results show that the selected architecture provides a higher degree of accuracy than random guessing (i.e., assigning a class randomly for each observation in the data set), and it can capture subtle differences in behaviour between different driving profiles.

With the advancement of technology, there is a significant development in automotive industry, where the vehicle itself performs more and more driving functions. The number of Advanced Driver Assistance Systems (ADASs) has been rising significantly. In this paper, the solution for autonomous vehicle parking is proposed. Firstly, an algorithm that detects parking spaces (PSs) and classifies them only by processing frames received from the camera placed on the side of the ego vehicle was developed. Furthermore, an algorithm for performing the autonomous vehicle parking in different ways (forward/backward, left/right side of the road) at detected free PS is developed. The solution was developed using Python programing language and OpenCV library and tested in CARLA simulator. Test results show that the algorithm correctly detects and classifies PSs in 100% of test cases, whereas after that the success of the parking process itself was 88%. The algorithm is capable of processing 10 frames per second, thus allowing real-time operation for typical vehicle speed during parking process.

Nowadays, there is a trend to replace the physical vehicle mirrors with camera system. Camera systems bring several benefits to modern vehicles such as clear image in bad weather conditions, optimized brightness and contrast and no blind-spots. In this paper one such system is proposed which is based on the processing of signals received from five different cameras, using three wide and two narrow-angle cameras. The solution is developed by using the Texas Instruments VISION SDK 2 software framework and implemented on Advanced driver- assistance system (ADAS) board with Texas Instruments TDA2xx System on Chip. Four different use cases are developed. In use cases with two and three cameras, different views of the vehicle environment are presented to a driver. Such displays allow an overview of both blind spots around the vehicle and a clear overview of the rest of the surroundings. The use case with zooming of the desired part of the image allows the driver to select the part of the screen he wants to enlarge. The last use case detects the vehicle behind ego-vehicle, estimates the distance and speed of the vehicle and if necessary, warns driver about incoming vehicle. Verification of the proposed solution was performed by measuring the implementation processing speed and by comparing the obtained results with the actual ones in case of speed and distance estimation.

LiDARs are becoming one of the pillars for the environmental sensing required by advanced driver assistance system (ADAS). Driven by the automotive industry, many new manufacturers are continuously putting new LiDARs on the market, thus increasing their availability and, concomitantly, reducing prices. Accordingly, LiDARs are today finding many new applications also in other fields, such as agriculture and industrial automation. In this article, we describe and discuss a measurement procedure for the analysis and comparison of the performances of LiDARs and report an example of the results obtained from the characterization of one the most widespread LiDARs—the VLP 16 by Velodyne. The proposed measurement procedure and setup have been designed to allow quick and easy installation and analysis of most LiDARs. In particular, they have been designed to allow the straightforward investigation of performances such as the warm-up, stability, and errors in the measured coordinates and parameters such as the spots pattern, waist, and divergence of the laser beams, thus allowing to analyze probably the most relevant performances and parameters for scanning LiDARs. Errors in the measured coordinates have been estimated using an absolute interferometer, whereas the beam parameters have been analyzed using a camera system. As an example, the analysis of the performances of the VLP 16 revealed a warm-up time of about 42 min and errors of few millimeters over a measuring interval of about (2, 21) m. On the other hand, the analysis of the laser beams revealed the vertical and horizontal beam divergences of $\approx 1.2$ and $\approx 3.9$ mrad, respectively.

Avoiding vehicle-to-pedestrian crashes is a critical requirement for nowadays advanced driver assistant systems (ADAS) and future self-driving vehicles. Accordingly, detecting pedestrians from raw sensor data has a history of more than 15 years of research, with vision playing a central role. During the last years, deep learning has boosted the accuracy of image-based pedestrian detectors. However, detection is just the first step towards answering the core question, namely is the vehicle going to crash with a pedestrian provided preventive actions are not taken? Therefore, knowing as soon as possible if a detected pedestrian has the intention of crossing the road ahead of the vehicle is essential for performing safe and comfortable maneuvers that prevent a crash. However, compared to pedestrian detection, there is relatively little literature on detecting pedestrian intentions. This paper aims to contribute along this line by presenting a new vision-based approach which analyzes the pose of a pedestrian along several frames to determine if he or she is going to enter the road or not. We present experiments showing 750 ms of anticipation for pedestrians crossing the road, which at a typical urban driving speed of 50 km/h can provide 15 additional meters (compared to a pure pedestrian detector) for vehicle automatic reactions or to warn the driver. Moreover, in contrast with state-of-the-art methods, our approach is monocular, neither requiring stereo nor optical flow information.

Safety-critical advanced driver-assistance systems (ADAS) are expected to benefit from cellular vehicle-to-everything communication, providing both short-range and long-range wireless connectivity between vehicles and road/network infrastructure equipment. However, upon switching between adjacent coverage areas belonging to different network operators, the wireless connectivity may be discontinued. This issue prevents network service continuity and causes interruption of ADAS service availability. In this paper, we aim at assessing the average duration of service interruption due to such network reselection procedures. We provide performance evaluation results highlighting that there is a wide gap between the best case scenario and the worst case scenario. Moreover, we propose a simple heuristic to limit the downside of the lack of cooperation between network operators; our heuristic algorithm, which does not require a tight integration between the operators, performs much better than the worst case and it is fairly comparable with the best case scenario. Nevertheless the present work highlights the importance of a tighter integration between network operators to reduce the delay to the minimum.

An innovative solution named Annotation as a Service (AaaS) has been specifically designed to integrate heterogeneous video annotation workflows into containers and take advantage of a cloud native highly scalable and reliable design based on Kubernetes workloads. Using the AaaS as a foundation, the execution of automatic video annotation workflows is addressed in the broader context of a semi-automatic video annotation business logic for ground truth generation for Autonomous Driving (AD) and Advanced Driver Assistance Systems (ADAS). The document presents design decisions, innovative developments, and tests conducted to provide scalability to this cloud-native ecosystem for semi-automatic annotation. The solution has proven to be efficient and resilient on an AD/ADAS scale, specifically in an experiment with 25 TB of input data to annotate, 4000 concurrent annotation jobs, and 32 worker nodes forming a high performance computing cluster with a total of 512 cores, and 2048 GB of RAM. Automatic pre-annotations with the proposed strategy reduce the time of human participation in the annotation up to 80% maximum and 60% on average.

Modern cars use advanced electronic systems that help the driver with the driving process - so-called Advanced Driver-Assistance Systems (ADAS). ADAS systems are used to automate, customize and improve systems within a vehicle for greater safety and better driving experience. Since ADAS systems as such can have a significant impact on the driving process, the vehicle and the driver, they must be thoroughly tested and developed within many industry standards. The key factor in their work is communication between individual system components. This standardized communication is necessary to test, which is usually performed by developing AUTomotive Open System Architecture (AUTOSAR) communication tests. Since ADAS testing can be quite a complex and time-consuming process, automated testing is performed in an appropriate testing environment. In this paper, existing ADAS environment testing systems is presented, which generates a test environment for the simulation of communication in the middle layer (Middleware) of AUTOSAR architecture. Test Environment Generator (TEG) is a Python program for processing ARXML test files based on which it generates a test environment model in the form of separate components in the C programming language. The program consists of input data parsing, parsed data storing and components generation that build the test environment. Based on the detected disadvantages of the existing TEG, several modifications are proposed in order to accelerate its execution time and to introduce more robust and stable data storage methods in database form.

Many new technologies for ADAS are emerging in recent years. In this scenario, LiDARs are becoming one of the pillars for environmental sensing needed in ADAS.Driven by the automotive industry, nowadays many new manufacturers are continuously putting new LiDAR systems on the market, thus increasing LiDARs availability and concomitantly reducing prices. As a result, LiDARs are today finding many new applications in fields such as agriculture and industrial automation.In this paper, we describe and discuss a measurement procedure for the analysis of the performances of LiDARs and, we report the results obtained from the characterization of one the most widespread LiDAR systems — namely VLP 16 by Velodyne. In particular, we compare the estimates provided by the LiDAR with the estimates provided by an absolute interferometer. To the best of our knowledge, these are the first published data obtained comparing a LiDAR and an absolute interferometer.The proposed tests investigated both warm-up and stability and, errors in the measured coordinates.The obtained results indicate a warm-up times of about 42 min and errors of few millimeters over a measuring interval of about (2, 21) m.

International audience; In the recent years, automotive car industry is evolving towards a new generation of autonomous vehicles, where decision making is not fully perform by the driver but it partially relies on the technology of the car itself. Indeed, a CPU inside the car will process all information coming from the sensors, distinguishing different scenarios appearing in the real life and ultimately allowing decision making. Since the CPU will be confronted with plenty of information, tools like machine learning or big-data analysis will be a useful ally to separate data from information. These existing machine learning techniques, such as kernel Principal Component Analysis (k-PCA), Locally Linear Embedding (LLE) among many other techniques, are useful to unveil the latent parameters defining a given scenario. Indeed, these algorithms have been already used to perform real-time classification of signals appearing throughout the road. Selecting the modeling of the electromagnetic response of the radar plays an important role to achieve real time constraints. Even though Helmholtz equation represents accurately the physics, the computational cost of such simulation is not affordable for real-time applications due to high radar operating frequencies, resulting into a very fine finite element mesh. On the other hand, far field approaches are not so accurate when the objects are very close due to plane wave assumption. In the first part of this work, the Geometrical Optics method is investigated in this work as a possible route to fulfill both real-time and accuracy constraints. The main hypothesis under such model is that waves are treated as straight lines constrained to optical reflection laws. Therefore, there is no need to mesh the interior of the domain. However, the accuracy of such approach is compromised when the size of the objects inside the domain are comparable to the wave lengths or in the vicinity of angular points. The second part is mainly focused on of the application of manifold learning and big data analysis into a data set of precomputed scenarios. Indeed, the identification of an unknown scenario from electromagnetic signals is purchased. Nevertheless, current research lines are devoted to give an answer to questions such as how many receptors do we need to identify unequivocally the scenario, where to locate the receptors, or which parts of the scenario have a negligible impact in the electromagnetic response.

Mobility in smart cities is also becoming smart, promoting on the one hand transport modes based on zero emission electrical technologies and on the other providing vehicles with technological solutions that support the drivers in driving operations. In the cities, more and more cars today are equipped with autonomous driving systems, based on Sensor Fusion Perception platforms, which aim to improve road safety and reduce accidents, to halve accident deaths in the shortest possible time. On the other hand, the sustainability objectives also shift to increasingly intense use of public transport with zero emissions, such as that offered by trams. Trams, unlike the other rail transport systems, use the same road infrastructure as cars, motorbikes, bikes, and pedestrians and will soon find themselves interacting with vehicles with increasingly higher autonomous driving levels. It is clear, therefore, that even for trams, the time is ripe to accommodate driving support systems. In this document, an overview is provided of the potential for the diffusion of advanced driver assistance systems by the automotive sector, in order to then evaluate their porting to trams, with the ultimate goal of increasing their level of safety and automation.

In this paper we report our experiences in designing and implementing a digital virtual cockpit to be installed as a component within the software stack of an Advanced Driving Assisted System (ADAS). Since in next-generation automotive embedded platforms both autonomous driving related workloads and virtual cockpit rendering tasks will co-run in a hypervisor-mediated environment, they will share computational resources. For this purpose, our work has been developed by following a requirement-driven approach in which regulations, usability and visual attractiveness requirements have to be taken into account by balancing their impact in terms of computational resources of the embedded platform in which such graphics interfaces are deployed. The graphic interfaces we realized consist of a set of 2D frames for the instrument cluster (for displaying the tachometer and the speedometer) and a screen area in which a 3D representation of the vehicle surroundings is rendered alongside driving directions and the point-cloud obtained through a LIDAR. All these components are able to alert the driver of imminent and/or nearby driving hazards.

Automotive industry has made significant progress with the introduction of semi- autonomous features in the form of Advanced Driver Assistance Systems (ADAS). Development of ADAS algorithms requires vast amounts of reliable data which accurately represents information available to the vehicle during its operation. Usually, cameras operate and capture everything only while ADAS is operational, meaning that all captured data is only used by the ADAS, and not preserved for later analysis. Storing all recorded information for future use in development of ADAS algorithms presents a significant issue and is the main motivation behind this work. This paper introduces a system capable of simultaneous video recording from up to 9 cameras on an external PC using PCI Express and reproduction of stored video materials using up to 9 output ports. The system connects to the cameras in vehicle and can be easily detached, thus allowing it to be moved to an algorithm development facility, where it can reproduce all recorded data. Using the developed system application, user can easily record selective data from the vehicle and then reproduce it in a development facility without any risk of information loss. System application also supports test pattern mode and frame extraction functionalities along with several tools for easier system configuration.

Classical approaches for testing of automated driving systems (ADS) of SAE levels 1 and 2 were based on defined scenarios with specific maneuvers, depending on the function under test. For ADS of SAE level 3+, the scenario space is infinite and calling for virtual testing and verification. The biggest challenge for virtual testing methods lies in the realistic representation of the virtual environment where the ADS is tested. Such an environment shall provide the possibility to model and develop vehicles, objects, control algorithms, traffic participants and environment elements in order to generate valid and representative test data. An important and crucial aspect of such environments is the testing of vehicles in a complex traffic environment with a stochastic and realistic traffic representation. For this research we used a microscopic traffic flow simulation software (TFSS) PTV Vissim and the vehicle simulation software IPG CarMaker to test ADS. Although the TFSS provides realistic and stochastic behavior of traffic participants, the occurrence of safety-critical scenarios (SCS) is not guaranteed. To generate and increase such scenarios, a novel stress testing method (STM) is introduced. With this method, traffic participants are manipulated in the vicinity of the vehicle under test in order to provoke SCS derived from statistical accident data on motorways in Austria. Using the co-simulation between IPG CarMaker, PTV Vissim and external driver models in Vissim are used to imitate human driving errors, resulting in an increase of SCS.

open access article Detecting and classifying driver distractions is crucial in the prevention of road accidents. These distractions impact both driver behavior and vehicle dynamics. Knowing the degree of driver distraction can aid in accident prevention techniques, including transitioning of control to a level 4 semi- autonomous vehicle, when a high distraction severity level is reached. Thus, enhancement of Advanced Driving Assistance Systems (ADAS) is a critical component in the safety of vehicle drivers and other road users. In this paper, a new methodology is introduced, using an expert knowledge rule system to predict the severity of distraction in a contiguous set of video frames using the Naturalistic Driving American University of Cairo (AUC) Distraction Dataset. A multi-class distraction system comprises the face orientation, drivers’ activities, hands and previous driver distraction, a severity classification model is developed as a discrete dynamic Bayesian (DDB). Furthermore, a Mamdani-based fuzzy system was implemented to detect multi- class of distractions into a severity level of safe, careless or dangerous driving. Thus, if a high level of severity is reached the semi-autonomous vehicle will take control. The result further shows that some instances of driver’s distraction may quickly transition from a careless to dangerous driving in a multi-class distraction context.

Background and Objectives The increasing number of senior drivers may introduce new road risks due to age-related declines in physical and cognitive abilities. Advanced driver assistance systems (ADAS) have been proposed as solutions to minimize age-related declines, thereby increasing both senior safety and mobility. This study examined factors that influence seniors’ attitudes toward adopting ADAS after significant exposure to the technology in naturalistic settings. Research Design and Methods This study recruited 18 senior drivers aged 70–79 to drive vehicles equipped with ADAS for 6 weeks in their own environments. Afterward, each participant was enrolled in 1 of the 3 focus group sessions to discuss their changes in attitude toward ADAS based on their driving experiences. We applied structural topic modeling (STM) on the focus group transcripts to reveal key topics deemed important to seniors. Results STM revealed 5 topics of importance for seniors. In order of prevalence, these were (i) safety, (ii) confidence concerning ADAS, (iii) ADAS functionality, (iv) user interface/usability, and (v) non-ADAS–related features. Based on topics and associated keywords, seniors perceived safety improvement with ADAS but expressed concerns about its limitations in coping with adverse driving conditions. Experience and training were suggested for improving seniors’ confidence in ADAS. Blind spot alert and adaptive cruise control received the most discussion regarding perceived safety and comfort. Discussion and Implications This study indicated that promoting road safety for senior drivers through ADAS is feasible. Acceptance and appropriate use of ADAS may be supported through intuitive and senior-friendly user interfaces, in-depth training programs, and owner’s manuals specifically designed and tested for senior drivers.

The growing functionalities implemented on vehicles have increased the importance of simulation in the design process. This complexity is mainly driven by the introduction of electrified powertrains, Advanced Driver Assistance Systems (ADAS) and Automated Driving Systems (ADS). Additionally, the automotive industry must reduce development times and cost, while keeping flexible development capabilities and fulfilling demanding regulation standards for safety-critical systems. Existing testing frameworks based on simulation implement typically analytical models to ensure real-time performance, and provide limited flexibility to perform Hardware in the Loop (HiL) setup based tests. In this work a vehicle modelling approach which guarantees high accuracy and real-time capabilities is proposed. Moreover, the proposed approach is validated firstly with real vehicle data, demonstrating that it can fairly reproduce the behaviour of the vehicle tested; and secondly, in a HiL setup to demonstrate the real-time execution capabilities of the approach.

The increasing complexity of the Advanced Driver Assistance Systems (ADAS) is making more difficult to perform the Hazard Analysis and Risk Assessment (HARA). These items require high-performance Electronic Control Units (ECU) with extensive software functionalities. To correctly operate they interact with the driver, environment and other vehicle functions through high-speed in-vehicle networks, as well as a wide range of sensors and actuators. As a result, they implement complex behaviors whose outcome in presence of faults is not trivial to identify and classify as requested by the concept phase included in the most recent functional safety standards. In this paper we present a simulation-based methodology to perform the HARA of a vehicle function by mixing the usual industrial approach, based on the designers' knowledge, with one that makes use of a vehicle-level simulator. The simulation-based approach provides an automatic and systematic method to assess the complex interaction of the item under analysis with other vehicle functions in possibly complex operational situations, thus making the prediction of hazards easier. We choose to demonstrate the approach by applying it to a well-known automotive industry case study: an Advanced Emergency Braking System (AEBS). In this way, it is possible to analyze the effects of the function provided by the item, keeping into account the simulations results and comparing them to similar situations analysis available in literature. Thanks to the obtained simulation-based results, safety engineers can formulate a more objective hypothesis, in particular during the hazard classification subphase.

The detection of pedestrians plays an essential part in the development of automated driver assistance systems. Many of the currently available datasets for pedestrian detection focus on urban environments. State-of-the-art neural networks trained on these datasets struggle in generalizing their predictions from one environment to a visually dissimilar one, limiting the use case to urban scenes. Commercial working machines like tractors or excavators make up a substantial share of the total number of motorized vehicles and are often situated in fundamentally different surroundings, e.g. forests, meadows, construction sites or farmland. In this paper, we present a dataset for pedestrian detection which consists of 1018 stereo-images showing varying numbers of persons in differing non-urban environments and comes with manually annotated pixel-level segmentation masks and bounding boxes.

International audience; Vision-based advanced driver assistance systems (ADAS), appeared in the 2000s, are increasingly integrated on-board mass-produced vehicles, as off-the-shelf low-cost cameras are now available. But ADAS implement most of the time-specific and basic functionalities such as lane departure or control of the distance to other vehicles. Integrating accurate localization and mapping functionalities meeting the constraints of ADAS (high-throughput, low-consumption, and small-design footprint) would pave the way towards obstacle detection, identification and tracking on-board vehicles at potential high speed. While the SLAM problem has been widely addressed by the robotics community, very few embedded operational implementations can be found, and they do not meet the ADAS-related constraints. In this paper, we implement the first 3D monocular EKF-SLAM chain on a heterogeneous architecture, on a single System on Chip (SoC), meeting these constraints. In order to do so, we picked up a standard co-design method (Shaout et al. Specification and modeling of hw/sw co-design for heterogeneous embedded systems, 2009) and adapted it to the implementation of potentially any of such complex processing chains. The refined method encompasses a hardware-in-the-loop approach allowing to progressively integrate hardware accelerators on the basis of a systematic rule. We also have designed original hardware accelerators for all the image processing functions involved, and for some algebraic operations involved in the filtering process.

Increasing literature has shown the usefulness of a dimensional approach to mental disorders, particularly when exploring subjects exposed to traumatic experiences such as a severe illness in one's child. Recent evidence suggests an increased vulnerability in subjects with autism spectrum symptoms to develop post-traumatic stress symptoms. The aim of the present study was to evaluate the presence of adult autism subthreshold spectrum in a sample of parents of children with epilepsy and its impact on post-traumatic stress spectrum symptoms in the same study sample. Results revealed noteworthy correlations between post-traumatic stress symptoms and adult autism subthreshold spectrum (AdAS Spectrum) only in the subgroup of the fathers. In particular, were evidenced correlations between AdAS Spectrum domain of rumination and narrow interests and some TALS-SR nuclear domains: reaction to traumatic events, reexperiencing and arousal. These findings corroborate the hypothesis that subthreshold autistic features may influence the possible psychopathological reaction to trauma. Keywords: PTSD, AdAS, Gender, Caregiver, Parents, Epileptic children

Nowadays, advanced driver assistance systems (ADAS) has been incorporated with a distinct type of progressive and essential features. One of the most preliminary and significant features of the ADAS is lane marking detection, which permits the vehicle to keep in a particular road lane itself. It has been detected by utilizing high-specialized, handcrafted features and distinct post-processing approaches lead to less accurate, less efficient, and high computational framework under different environmental conditions. Hence, this research proposed a simple encode-decode deep learning approach under distinguishing environmental effects like different daytime, multiple lanes, different traffic condition, good and medium weather conditions for detecting the lane markings more accurately and efficiently. The proposed model is emphasized on the simple encode-decode Seg-Net framework incorporated with VGG16 architecture that has been trained by using the inequity and cross-entropy losses to obtain more accurate instant segmentation result of lane markings. The framework has been trained and tested on a vast public dataset named Tusimple, which includes around 3.6K training and 2.7 k testing image frames of different environmental conditions. The model has noted the highest accuracy, 96.61%, F1 score 96.34%, precision 98.91%, and recall 93.89%. Also, it has also obtained the lowest 3.125% false positive and 1.259% false-negative value, which transcended some of the previous researches. It is expected to assist significantly in the field of lane markings detection applying deep neural networks.

Advanced Driver-Assistance Systems (ADASs) are currently gaining particular attention in the automotive field, as enablers for vehicle energy consumption, safety, and comfort enhancement. Compelling evidence is in fact provided by the variety of related studies that are to be found in the literature. Moreover, considering the actual technology readiness, larger opportunities might stem from the combination of ADASs and vehicle connectivity. Nevertheless, the definition of a suitable control system is not often trivial, especially when dealing with multiple-objective problems and dynamics complexity. In this scenario, even though diverse strategies are possible (e.g., Equivalent Consumption Minimization Strategy, Rule-based strategy, etc.), the Model Predictive Control (MPC) turned out to be among the most effective ones in fulfilling the aforementioned tasks. Hence, the proposed study is meant to produce a comprehensive review of MPCs applied to scenarios where ADASs are exploited and aims at providing the guidelines to select the appropriate strategy. More precisely, particular attention is paid to the prediction phase, the objective function formulation and the constraints. Subsequently, the interest is shifted to the combination of ADASs and vehicle connectivity to assess for how such information is handled by the MPC. The main results from the literature are presented and discussed, along with the integration of MPC in the optimal management of higher level connection and automation. Current gaps and challenges are addressed to, so as to possibly provide hints on future developments.

Among the reasons for traffic accidents, distractions are the most common. Although there are many traffic signs on the road that contribute to safety, variable message signs (VMSs) require special attention, which is transformed into distraction. ADAS (advanced driver assistance sys-tem) devices are advanced systems that perceive the environment and provide assistance to the driver for his comfort or safety. This project aims to develop a prototype of a VMS (variable mes-sage sign) reading system using machine learning techniques, which are still not used, especially in this aspect. The assistant consists of two parts: a first one that recognizes the signal on the street and another one that extracts its text and transforms it into speech. For the first one, a set of im-ages were labeled in PASCAL VOC format by manual annotations, scraping and data augmenta-tion. With this dataset, the VMS recognition model was trained, a RetinaNet based off of Res-Net50 pretrained on the dataset COCO. Firstly, in the reading process, the images were prepro-cessed and binarized to achieve the best possible quality. Finally, the extraction was done by the Tesseract OCR model in its 4.0 version, and the speech was done by the cloud service of IBM Watson Text to Speech. Plan Nacional para la Investigación PN I+D+i (PID2019-104793RB-C32) Comunidad de Madrid. SEGVAUTO-4.0-CM (P2018/EMT-4362) 3.576 JCR (2020) Q1, 14/64 Instruments & Instrumentation 0.636 SJR (2020) Q2, 46/121 Analytical Chemistry No data IDR 2019 UEM

The aim of the paper was to analyze effective solutions for accurate lane detection on the roads. We focused on effective detection of airport runways and taxiways in order to drive a light-measurement trailer correctly. Three techniques for video-based line extracting were used for specific detection of environment conditions: (i) line detection using edge detection, Scharr mask and Hough transform, (ii) finding the optimal path using the hyperbola fitting line detection algorithm based on edge detection and (iii) detection of horizontal markings using image segmentation in the HSV color space. The developed solutions were tuned and tested with the use of embedded devices such as Raspberry Pi 4B or NVIDIA Jetson Nano.

New trends in the automotive industry such as autonomous driving and Car2X require a large amount of data to be exchanged between different devices. Radar sensors are key components in developing vehicles of the future, therefore these devices are used in a large spectrum of applications, where data traffic is of paramount importance. As a result, communication traffic volumes have become more complex, leading to the research of optimization approaches to be applied at the AUTOSAR level. Our paper offers such an optimization solution at the AUTOSAR communication level. The radar sensor is accessed in a remote manner, and the experiments aimed at performance measurements revealed that our solution is superior to the Full AUTOSAR implementation in terms of memory usage and runtime measurements.

Reliable vision in challenging illumination conditions is one of the crucial requirements of future autonomous automotive systems. In the last decade, thermal cameras have become more easily accessible to a larger number of researchers. This has resulted in numerous studies which confirmed the benefits of the thermal cameras in limited visibility conditions. In this paper, we propose a learning-based method for visible and thermal image fusion that focuses on generating fused images with high visual similarity to regular truecolor (red-green-blue or RGB) images, while introducing new informative details in pedestrian regions. The goal is to create natural, intuitive images that would be more informative than a regular RGB camera to a human driver in challenging visibility conditions. The main novelty of this paper is the idea to rely on two types of objective functions for optimization: a similarity metric between the RGB input and the fused output to achieve natural image appearance, and an auxiliary pedestrian detection error to help defining relevant features of the human appearance and blending them into the output. We train a convolutional neural network using image samples from variable conditions (day and night) so that the network learns the appearance of humans in the different modalities and creates more robust results applicable in realistic situations. Our experiments show that the visibility of pedestrians is noticeably improved especially in dark regions and at night. Compared to existing methods we can better learn context and define fusion rules that focus on the pedestrian appearance, while that is not guaranteed with methods that focus on low-level image quality metrics.

Most approaches for moving object detection (MOD) based on computer vision are limited to stationary camera environments. In advanced driver assistance systems (ADAS), however, ego-motion is added to image frames owing to the use of a moving camera. This results in mixed motion in the image frames and makes it difficult to classify target objects and background. In this paper, we propose an efficient MOD algorithm that can cope with moving camera environments. In addition, we present a hardware design and implementation results for the real-time processing of the proposed algorithm. The proposed moving object detector was designed using hardware description language (HDL) and its real-time performance was evaluated using an FPGA based test system. Experimental results demonstrate that our design achieves better detection performance than existing MOD systems. The proposed moving object detector was implemented with 13.2K logic slices, 104 DSP48s, and 163 BRAM and can support real-time processing of 30 fps at an operating frequency of 200 MHz.