Search

Autonomous driving poses numerous challenging problems, one of which is perceiving and understanding the environment. Since self-driving is safety critical and many actions taken during driving rely on the outcome of various perception algorithms (for instance all traffic participants and infrastructural objects in the vehicle's surroundings must reliably be recognized and localized), thus the perception might be considered as one of the most critical subsystems in an autonomous vehicle. Although the perception itself might further be decomposed into various sub-problems, such as object detection, lane detection, traffic sign detection, environment modeling, etc. In this paper the focus is on fusion models in general (giving support for multisensory data processing) and some related automotive applications such as object detection, traffic sign recognition, end-to-end driving models and an example of taking decisions in multi-criterial traffic situations that are complex for both human drivers and for the self-driving vehicles as well.

We demonstrate a working functional prototype of a cooperative perception system that maintains a real-time digital twin of the traffic environment, providing a more accurate and more reliable model than any of the participant subsystems—in this case, smart vehicles and infrastructure stations—would manage individually. The importance of such technology is that it can facilitate a spectrum of new derivative services, including cloud-assisted and cloud-controlled ADAS functions, dynamic map generation with analytics for traffic control and road infrastructure monitoring, a digital framework for operating vehicle testing grounds, logistics facilities, etc. In this paper, we constrain our discussion on the viability of the core concept and implement a system that provides a single service: the live visualization of our digital twin in a 3D simulation, which instantly and reliably matches the state of the real-world environment and showcases the advantages of real-time fusion of sensory data from various traffic participants. We envision this prototype system as part of a larger network of local information processing and integration nodes, i.e., the logically centralized digital twin is maintained in a physically distributed edge cloud.

The paper discusses the identification of the empirical white noise processes generated by deterministic numerical algorithms.The introduced fuzzy-random complementary approach can identify the inner hidden correlational patterns of the empirical white noise process if the process has a real hidden structure of this kind. We have shown how the characteristics of auto-correlated white noise processes change as the order of autocorrelation increases. Although in this paper we rely on random number generators to get approximate white noise processes, in our upcoming research we are planning to turn the focus on physical white noise processes in order to validate our hypothesis.

A spectacular measurement campaign was carried out on a real-world motorway stretch of Hungary with the participation of international industrial and academic partners. The measurement resulted in vehicle based and infrastructure based sensor data that will be extremely useful for future automotive R&D activities due to the available ground truth for static and dynamic content. The aim of the measurement campaign was twofold. On the one hand, road geometry was mapped with high precision in order to build Ultra High Definition (UHD) map of the test road. On the other hand, the vehicles—equipped with differential Global Navigation Satellite Systems (GNSS) for ground truth localization—carried out special test scenarios while collecting detailed data using different sensors. All of the test runs were recorded by both vehicles and infrastructure. The paper also showcases application examples to demonstrate the viability of the collected data having access to the ground truth labeling. This data set may support a large variety of solutions, for the test and validation of different kinds of approaches and techniques. As a complementary task, the available 5G network was monitored and tested under different radio conditions to investigate the latency results for different measurement scenarios. A part of the measured data has been shared openly, such that interested automotive and academic parties may use it for their own purposes.

The paper discusses the identification of the empirical, partially intelligible white noise processes generated by deterministic numerical algorithms. The introduced fuzzy‐random complemen...

Autonomous driving functions increasingly incorporate machine learning methods for perception, environment modeling and decision making. The ever-present danger of unintended behavior of neural networks has given cause for concern in safety-critical applications and has spurred research towards the establishment of certified properties. In our work we present a general approach for exploring guaranteed properties of neural networks through a specific use case. We first methodically specified several desired properties for an ideal traffic sign classifier. Then we applied a mixed integer linear programming verification method to certify (or conversely, falsify) that our pre-trained rectified linear unit (ReLU) activation network will satisfy given properties under all conditions. We established through relaxations of our initial specification that our model does in fact happen to guarantee certain (but not all) properties at a different threshold. We thus showed that it is possible to establish the degree to which non-trivial and humanly meaningful properties are inherently guaranteed by an arbitrary pixel space ReLU network. We believe this approach can be a helpful addition in comprehensively evaluating the reliability of a network via quantifying its degree of compliance to an ideal specification.

In real time evaluation systems the decision should be made in time to avoid the serious consequences. In the literature there are several techniques which aim to reduce the computational complexity of these kinds of systems. Among others the Higher Order Singular Value Decomposition (HOSVD) based reduction method is a useful tool for handling this problem. In the case when only non-exact reduction can be performed, the error calculation is also essential, because the reduction degree is based on the calculated error in these systems. In this paper the authors present the HOSVD reduction error calculation in hierarchical fuzzy systems using Mamdani-type inference and define a general formula which includes the propagated error from the previous levels of the hierarchy; in which the error bound, calculated by different ways, can be used and takes into account the case when not all the inputs contain a propagated error but only a few of them. Furthermore a greedy algorithm is presented for reduction optimization.

The paper focuses on the problem of raw data fusion in neural networks based 3D object detection architectures. Here we consider the case of autonomous driving with data from camera and LiDAR sensors. Understanding the vehicle surroundings is a crucial task in autonomous driving since any subsequent action taken is strongly dependent on it. In this paper we present an alternative method of fusing camera image information with LiDAR poinclouds at a close to raw level of abstraction. Our results suggest that our approach improves the average precision of 3D bounding box detection of cyclists (and possibly other objects) in sparse point clouds compared to the baseline architecture without low-level fusion. The proposed approach has been evaluated on the KITTI dataset containing driving scenes with corresponding camera and LiDAR data. The long-term goal of our research is to develop a neural network architecture for environment perception that fuses multi-sensor data at the earliest stages possible, thus leveraging the full benefits of possible inter-sensor synergies.

Photo sharing websites have become extremely popular with GPS-enabled digital cameras and camera phones. Seeing autumn leaves, cherry blossoms, etc. is a traditional seasonal activity. Tremendous photos about leaves and blossoms have been uploaded. The goal of our paper is to detect and rank the most scenic leaves and blossoms viewing places based on social and image features. Methods for social interestingness, color percentage, edge rate, sharpness, ranking score have been provided. The proposed methods have been validated and verified by experimental results for maple leaves in Kyoto.

A time series collected by thermal cameras yields many application possibilities for security monitoring applications. The character of features involved in thermal images is different compared to the images acquired by conventional capturing devices; many methods applicable for conventional image processing cannot directly be applied for thermal images. New methods are introduced in HOSVD based representation of thermal images, and in application of resolution enhancement and data compression. The retrieval process of thermal image databases and useful feature descriptors were analyzed, as well as the special semi-automatic fusion techniques of thermal image sequences.

With GPS-enabled digital cameras and camera phones photo sharing websites have become exceptionally popular recently. Traditional seasonal activities are to admire leaves and blossoms. Tremendous photos about them have been uploaded. The goal of our paper is to detect the most scenic leaves and blossoms viewing places from Flickr based on social and image features as well as to perform their time trend analysis based on statistical and control engineering methods. Firstly, methods for social interestingness, color percentage, edge rate, sharpness, ranking score have been proposed. Secondly, methods for time trend analysis based on chi square test of independence and subspace identification have been provided. The proposed methods have been validated and verified with experimental results for maple leaves in Kyoto.