Search

Cellular Vehicle-to-Everything (C-V2X) communications is a technology that enables intelligent vehicles to exchange information and thus coordinate with other vehicles, road users, and infrastructure. However, despite advancements in cellular technology for V2X applications, significant challenges remain regarding the ability of the system to meet stringent Quality-of-Service (QoS) requirements when deployed at scale. Thus, smaller-scale V2X use case deployments may embody a necessary stepping stone to address these challenges. This work assesses network architectures for an Intelligent Perception System (IPS) blind road junction or blind corner scenarios. Measurements were collected using a private 5G NR network with Sub-6GHz and mmWave connectivity, evaluating the feasibility and trade-offs of IPS network configurations. The results demonstrate the feasibility of the IPS as a V2X application, with implementation considerations based on deployment and maintenance costs. If computation resources are co-located with the sensors, sufficient performance is achieved. However, if the computational burden is instead placed upon the intelligent vehicle, it is questionable as to whether an IPS is achievable or not. Much depends on image quality, latency, and system performance requirements.

As the global production of waste continues to rise, there is a growing demand for more effective waste management strategies to handle this expanding problem. Recycling rates in the United States for recyclable materials are below 35%, resulting in elevated levels of waste being sent to landfills. This situation has alarming consequences, contributing to rising pollution in both soil and aquatic ecosystems, and is a significant source of concern for environmental scientists and the general population alike. The presence of contamination in recycling collection trucks is a root cause impacting recycling rates, leading to the rejection of entire loads from recycling processing sites. This problem can be alleviated by automatically detecting contamination in recyclable waste that is loaded into the collection truck hopper before compaction. In this paper, we have used different state-of-the-art computer vision-based models such as Faster-RCNN, Cascade-RCNN, Retinanet, YOLOv8 and Mask-RCNN to identify contamination within a densely cluttered waste environment. We further investigate the viability of transfer learning, comparing it to the models trained from scratch. The YOLOv8-x model attained a mean-average precision of 0.395 without using transfer learning, whereas with the incorporation of transfer learning, its performance increased to 0.463.

Graph Neural Networks (GNNs) offer a promising direction for medical image analysis, particularly due to their ability to capture complex relationships and handle non-Euclidean data structures often encountered in this domain. This study aims to demonstrate the potential of GNNs in medical image classification, showcasing their comparable performance to leading CNNs while using significantly fewer parameters. We introduce a novel GNN model, Graph Convolution Neural Network Enhanced Connectivity (GCNN-EC), incorporating an enhanced connectivity technique that leverages interrelationships between RGB channel features. Evaluated on the MedMNIST dataset, a standardized benchmark for medical image analysis, GCNN-EC performs competitively against pre-trained DNNs, highlighting its potential as an efficient and effective approach for medical image analysis. Importantly, we do not claim that GCNN-EC outperforms all CNNs in every scenario, but rather aim to showcase the potential of GNNs as a valuable tool in the medical imaging toolkit that can be used on its own or in conjuction with CNNs to further advance the state of the art. Additionally, our model outperforms traditional GNNs on standard image datasets like MNIST and CIFAR-10, mitigating the common issue of over-smoothing. These findings underscore the effectiveness of GCNs with dynamic filtering and encourage further research into GNN-based approaches for medical imaging and broader computer vision tasks.

Supervised deep learning methods have produced state-of-the-art results with large labeled datasets. However, accessing large labeled datasets is difficult in medical image analysis because of a shortage of medical experts, expensive annotations, and privacy constraints in the healthcare domain. Self-supervised learning is a branch of machine learning that exploits unlabeled data to encourage network weights toward a valid latent representation of the data during a so-called pretext task. The features learned by the model while solving pretext tasks are transferred to a downstream task where limited annotations are available. In this work, we propose PatchLoc, a novel pretext task whose objective is to find the location of a given patch from an image as a source of supervision. We validated the effectiveness of PatchLoc on a downstream segmentation task using three different medical datasets. PatchLoc yields substantial improvements compared to U-Net trained from scratch and other pretext task-based approaches in a low data regime.

Camera-based object detection is integral to advanced driver assistance systems (ADAS) and autonomous vehicle research, and RGB cameras remain indispensable for their spatial resolution and color information. This study investigates exposure time optimization for such cameras, considering image quality in dynamic ADAS scenarios. Exposure time, the period during which the camera sensor is exposed to light, directly influences the amount of information captured. In dynamic scenarios, such as those encountered in typical driving scenarios, optimizing exposure time becomes challenging due to the inherent trade-off between Signal-to-Noise Ratio (SNR) and motion blur, i.e., extending exposure time to maximize information capture increases SNR, but also increases the risk of motion blur and overexposure, particularly in low-light conditions where objects may not be fully illuminated. The study introduces a comprehensive methodology for exposure time optimization under various lighting conditions, examining its impact on image quality and computer vision performance. Traditional image quality metrics show a poor correlation with computer vision performance, highlighting the need for newer metrics that demonstrate improved correlation. The research presented in this paper offers guidance into the enhancement of single-exposure camera-based systems for automotive applications. By addressing the balance between exposure time, image quality, and computer vision performance, the findings provide a road map for optimizing camera settings for ADAS and autonomous driving technologies, contributing to safety and performance advancements in the automotive landscape.

In recent years, significant advances have been made in the development of Advanced Driver Assistance Systems (ADAS) and other technology for autonomous vehicles. Automated object detection is a crucial component of autonomous driving; however, there are still known issues that affect its performance. For automotive applications, object detection algorithms are required to perform at a high standard in all lighting conditions; however, a major problem for object detection is poor performance in low-light conditions due to objects being less visible. This study considers the impact of training data composition on object detection performance in low-light conditions. In particular, this study evaluates the effect of different combinations of images of outdoor scenes, from different times of day, on the performance of deep neural networks, and considers the different challenges encountered during the training of a neural network. Through experiments with a widely used public database, as well as a number of commonly used object detection architectures, we show that more robust performance can be obtained with an appropriate balance of classes and illumination levels in the training data. The results also highlight the potential of adding images obtained in dusk and dawn conditions for improving object detection performance in day and night.

Vehicle-to-Everything (V2X) communication technology aims to enhance intelligent vehicles’ coordination and perception by enabling interactions with other vehicles, road users, and infrastructure. Although 4th and 5th generation cellular technologies (4G LTE and 5G NR) can support non-safety V2X communication, their design, rooted in multimedia and telephony, doesn’t align well with safety-critical applications. Despite the advancements of 5G NR over 4G LTE, ensuring reliable Quality-of-Service (QoS) metrics remains a notable concern. The process of handover, a core cellular network function, significantly contributes to the challenges in cellular communication reliability. To assess handover’s impact on latency in V2X communication scenarios, a driving-focused measurement campaign was executed on an early 5G NR-enabled commercial cellular network. Findings reveal that, in terms of its effect on end-user latency, the more mature 4G LTE network displays on average 63% (intra-freq.) and 36% (inter-freq.) lower latency costs when performing handovers compared to early 5G NR deployments. Notably, inter-generation handovers (4G to 5G and vice versa) exhibit 24% (intra-freq.) and 10% (inter-freq.) higher average latency costs with respect to other handover types. Furthermore, discernible variations in network reliability emerge between distinct V2X scenarios such as urban and rural environments. The prevailing reliability concerns tied to present handover algorithms hold substantial weight, as they undermine intelligent vehicles’ trust in network-received information. Consequently, these outcomes underscore the urgency for designing and implementing handover strategies better suited to the dynamic mobility characteristics of intelligent vehicles across diverse V2X scenarios.

Advancements in intelligent vehicles and Intelligent Transport Systems (ITS) have shown that they are now feasible in both technology and commerce. However, there are still significant challenges to overcome, particularly regarding the perception and coordination of intelligent vehicles in unfavourable conditions. Vehicle-to-Everything (V2X) communications is a technology that aims to enable intelligent vehicles to communicate with other road users and infrastructure to increase their range of perception and coordination capabilities. While the 4th generation of cellular technology (4G LTE) is capable of supporting V2X communications to some extent, its multimedia and telephony-centric design does not translate well to safety-critical applications. As a result, the 5th generation of cellular technology (5G NR) is being developed to improve V2X communications. To investigate the effectiveness of 5G NR in V2X communications, a driving-based measurement campaign of a commercial cellular network with early 5G NR deployments was conducted. Results showed that the existing 4G LTE network is limited in its capability, and early 5G NR deployments can in fact outperform it. However, neither 4G LTE nor 5G NR can reliably support advanced V2X applications. Early 5G NR deployments suffer from significant reliability issues compared to existing 4G LTE deployments. These reliability issues are of particular concern, as they impact the vehicle’s ability to trust the information it receives. These findings highlight the need for further design and implementation of intelligent vehicles and future 5G NR networks to address these reliability concerns and ensure the safe and efficient operation of intelligent vehicles in all conditions.

Automated vehicles rely heavily on image data from visible spectrum cameras to perform a wide range of tasks from object detection, classification, and avoidance to path planning. The availability and reliability of these sensors in adverse weather is therefore of critical importance to the safe and continuous operation of an automated vehicle. This review paper presents a data communication-inspired Image Formation Framework that characterizes the data flow from object through channel to sensor, and subsequent processing of the data. This framework is used to explore the degree to which adverse weather conditions affect the cameras used in automated vehicles for sensing and perception. The effects of rain on each element of the model are reviewed. Furthermore, the prevalence of these rain-induced changes in publicly available open-source datasets is reviewed. The degree to which synthetic rain generation techniques can accurately capture these changes is also examined. Finally, this paper offers some suggestions on how future adverse weather automotive datasets should be collected.

In the field of autonomous driving, cameras are crucial sensors for providing information about a vehicle’s environment. Image quality refers to a camera system’s ability to capture, process, and display signals to form an image. Historically, “good quality” in this context refers to images that have been processed by an Image Signal Processor (ISP) designed with the goal of providing the optimal experience for human consumption. However, image quality perceived by humans may not always result in optimal conditions for computer vision. In the context of human consumption, image quality is well documented and understood. Image quality for computer vision applications, such as those in the autonomous vehicle industry, requires more research. Fully autonomous vehicles inevitably encounter constraints concerning data storage, transmission speed, and energy consumption. This is a result of enormous amounts of data being generated by the vehicle from suites made up of multiple different sensors. We propose a potential optimization along the computer vision pipeline, by completely bypassing the ISP block for a class of applications. We demonstrate that doing so has a negligible impact on the performance of Convolutional Neural Network (CNN) object detectors. The results also highlight the benefits of using raw pre-ISP data, in the context of computation and energy savings achieved by removing the ISP.

Fully-supervised object detection and instance segmentation models have accomplished notable results on large-scale computer vision benchmark datasets. However, fully-supervised machine learning algorithms’ performances are immensely dependent on the quality of the training data. Preparing computer vision datasets for object detection and instance segmentation is a labor-intensive task requiring each instance in an image to be annotated. In practice, this often results in the quality of bounding box and polygon mask annotations being suboptimal. This paper quantifies empirically the ground truth annotation quality and COCO’s mean average precision (mAP) performance by introducing two separate noise measures, uniform and radial, into the ground truth bounding box and polygon mask annotations for the COCO and Cityscapes datasets. Mask-RCNN models are trained on various levels of noise measures to investigate the performance of each level of noise. The results showed degradation of mAP as the level of both noise measures increased. For object detection and instance segmentation respectively, using the highest level of noise measure resulted in a mAP degradation of 0.185 & 0.208 for uniform noise with reductions of 0.118 & 0.064 for radial noise on the COCO dataset. As for the Cityscapes datasets, reductions of mAP performance of 0.147 & 0.142 for uniform noise and 0.101 & 0.033 for radial noise were recorded. Furthermore, a decrease in average precision is seen across all classes, with the exception of the class motorcycle. The reductions between classes vary, indicating the effects of annotation uncertainty are class-dependent.

As the amount of waste being produced globally is increasing, there is a need for more efficient waste management solutions to accommodate this expansion. The first step in waste management is the collection of bins or containers. Each bin truck in a fleet is assigned a collection route. As the bin trucks have a finite amount of storage for waste, accepting overfilled bins may result in filling this storage before the end of the collection route. This creates inefficiencies as a second bin truck is needed to finish the collection route if the original becomes full. Currently, the recording and tracking of overfilled bins is a manual process, requiring the bin truck operator to undertake this task, resulting in longer collection route durations. To create a more efficient and automated process, computer vision methods are considered for the task of detecting the bin status. Video footage from a commercial collection route for two bin types, automated side loader (ASL) and front-end loader (FEL), was utilized to create appropriate computer vision datasets for the task of fully supervised object detection and instance segmentation. Selected state-of-the-art object detection and instance segmentation algorithms were used to investigate their performances on this proprietary dataset. A mean average precision (mAP) score of 0.8 or greater was achieved with each model, reflecting the effectiveness of using computer vision as a tool to automate the process of recording overfilled bins.

In advanced driver assistance systems (ADAS) or autonomous vehicle research, acquiring semantic information about the surrounding environment generally relies heavily on camera-based object detection. Image signal processors (ISPs) in cameras are generally tuned for human perception. In most cases, ISP parameters are selected subjectively and the resulting image differs depending on the individual who tuned it. While the installation of cameras on cars started as a means of providing a view of the vehicle’s environment to the driver, cameras are increasingly becoming part of safety-critical object detection systems for ADAS. Deep learning-based object detection has become prominent, but the effect of varying the ISP parameters has an unknown performance impact. In this study, we analyze the performance of 14 popular object detection models in the context of changes in the ISP parameters. We consider eight ISP blocks: demosaicing, gamma, denoising, edge enhancement, local tone mapping, saturation, contrast, and hue angle. We investigate two raw datasets, PASCALRAW and a custom raw dataset collected from an advanced driver assistance system (ADAS) perspective. We found that varying from a default ISP degrades the object detection performance and that the models differ in sensitivity to varying ISP parameters. Finally, we propose a novel methodology that increases object detection model robustness via ISP variation data augmentation.

Interacting with other roads users is a challenge for an autonomous vehicle, particularly in urban areas. Existing vehicle systems behave in a reactive manner, warning the driver or applying the brakes when the pedestrian is already in front of the vehicle. The ability to anticipate a pedestrian’s crossing intention ahead of time will result in safer roads and smoother vehicle maneuvers. The problem of crossing intent forecasting at intersections is formulated in this paper as a classification task. A model that predicts pedestrian crossing behaviour at different locations around an urban intersection is proposed. The model not only provides a classification label (e.g., crossing, not-crossing), but a quantitative confidence level (i.e., probability). The training and evaluation are carried out using naturalistic trajectories provided by a publicly available dataset recorded from a drone. Results show that the model is able to predict crossing intention within a 3-s time window.

Image quality is a well understood concept for human viewing applications, particularly in the multimedia space, but increasingly in an automotive context as well. The rise in prominence of autonomous driving and computer vision brings to the fore research in the area of the impact of image quality in camera perception for tasks such as recognition, localization and reconstruction. While the definition of “image quality” for computer vision may be ill-defined, what is clear is that the configuration of the image signal processing pipeline is the key factor in controlling the image quality for computer vision. This paper is partly review and partly positional with demonstration of several preliminary results promising for future research. As such, we give an overview of what is an