Search

Deep probabilistic forecasting is gaining attention across numerous applications. From weather prognosis, electricity consumption estimation, traffic flow prediction, to autonomous vehicle trajectory prediction. However, existing approaches focus on improving on average metrics without addressing the long-tailed distribution of errors. This thesis identifies long tail behavior in the error distribution of state-of-the-art deep learning methods for probabilistic forecasting. We analyze potential sources and explanations for this behavior. Further, we present two loss augmentation methods to mitigate tailedness pf error distributions: Pareto Loss and Kurtosis Loss. Both methods modify the loss using the concept of moments to penalize higher loss samples. Kurtosis Loss uses a symmetric measure, the fourth moment, while Pareto Loss uses an asymmetric measure of right-tailedness and models loss using a Generalized Pareto Distribution (GPD). We demonstrate the performance of our methods on several real-world datasets, including time series and spatiotemporal trajectories, achieving significant improvements on tail error metrics, while maintaining and even improving upon average error metrics.

In medical applications, deep learning methods are designed to automate diagnostic tasks. However, a clinically relevant question that practitioners usually face, is how to predict the future trajectory of a disease (prognosis). Current methods for such a problem often require domain knowledge, and are complicated to apply. In this paper, we formulate the prognosis prediction problem as a one-to-many forecasting problem from multimodal data. Inspired by a clinical decision-making process with two agents — a radiologist and a general practitioner, we model a prognosis prediction problem with two transformer-based components that share information between each other. The first block in this model aims to analyze the imaging data, and the second block leverages the internal representations of the first one as inputs, also fusing them with auxiliary patient data. We show the effectiveness of our method in predicting the development of structural knee osteoarthritis changes over time. Our results show that the proposed method outperforms the state-of-the-art baselines in terms of various performance metrics. In addition, we empirically show that the existence of the multi-agent transformers with depths of 2 is sufficient to achieve good performances. Our code is publicly available at https://github.com/MIPT-Oulu/CLIMAT.

In hopes of lessening the reliance on fossil fuels, Plug-in Hybrid Electric Vehicles (PHEVs) have become an attractive option as an alternative fuel vehicle due to their larger electric motors and energy storage systems (ESS). PHEVs can propel themself relying solely on their internal combustion engine (ICE), electric motor (EM), and or a hybrid of both. To improve their fuel efficiency, many studies have been done to investigate the use of a priori route information to optimize the use of a PHEV’s ICE and EM. This study introduces a real-time machine learning application of a control strategy known as Trajectory Forecasting (TF). TF takes a priori knowledge of a PHEV’s pre-planned route to determine when the vehicle will use its different forms of propulsion in the form of propulsion mode scheduling. However, it assumes constant route data such as traffic and resulting driving speed for its scheduling to be applicable. To automatically account for changing traffic as well as choose better alternative routes, this study looks at the use of a Convolutional Neural Network (CNN) to simulate a PHEV’s operation along available routes beforehand according to the rules of TF to choose a route that best satisfies a driver’s want, better fuel efficiency and possibly lower emissions. This new real-time TF-based machine learning control strategy is evaluated and compared to common PHEV control strategies such as Charge Sustaining (CS) and Charge Depletion (CD) using National Renewable Energy Laboratory’s vehicle simulator ADVISOR. Results show possible increases in Mpgge from 1.72%-130%, decreases in emitted hydrocarbons (HC), carbon monoxide (CO), and nitrous oxides (NOx) from 0.05%-70%, and 0.05%-90.35% reduction in gasoline consumption depending on overall route length and PHEV configuration.

In hopes of lessening the reliance on fossil fuels, Plug-in Hybrid Electric Vehicles (PHEVs) have become an attractive option as an alternative fuel vehicle due to their larger electric motors and energy storage systems (ESS). PHEVs can propel themself relying solely on their internal combustion engine (ICE), electric motor (EM), and or a hybrid of both. To improve their fuel efficiency, many studies have been done to investigate the use of a priori route information to optimize the use of a PHEV’s ICE and EM. This study introduces a real-time machine learning application of a control strategy known as Trajectory Forecasting (TF). TF takes a priori knowledge of a PHEV’s pre-planned route to determine when the vehicle will use its different forms of propulsion in the form of propulsion mode scheduling. However, it assumes constant route data such as traffic and resulting driving speed for its scheduling to be applicable. To automatically account for changing traffic as well as choose better alternative routes, this study looks at the use of a Convolutional Neural Network (CNN) to simulate a PHEV’s operation along available routes beforehand according to the rules of TF to choose a route that best satisfies a driver’s want, better fuel efficiency and possibly lower emissions. This new real-time TF-based machine learning control strategy is evaluated and compared to common PHEV control strategies such as Charge Sustaining (CS) and Charge Depletion (CD) using National Renewable Energy Laboratory’s vehicle simulator ADVISOR. Results show possible increases in Mpgge from 1.72%-130%, decreases in emitted hydrocarbons (HC), carbon monoxide (CO), and nitrous oxides (NOx) from 0.05%-70%, and 0.05%-90.35% reduction in gasoline consumption depending on overall route length and PHEV configuration.

Spatio-temporal data is a commonly used source of information. Using machine learning to analyse this kind of data can lead to many interesting and useful insights. In this thesis project, a novel public transportation spatio-temporal dataset is explored and analysed. The dataset contains 282 GB of positional events, spanning two weeks of time, from all public transportation vehicles in Östergötland county, Sweden. From the data exploration, three high-level problems are formulated: bus stop detection, GPS variance estimation, and arrival time prediction, also called trajectory forecasting. The bus stop detection problem is briefly discussed and solutions are proposed. Gaussian process regression is an effective method for solving regression problems. The GPS variance estimation problem is solved via the use of a mixture of Gaussian processes. A mixture of Gaussian processes is also used to predict the arrival time for public transportation buses. The arrival time prediction is from one bus stop to the next, not for the whole trajectory. The result from the arrival time prediction is a distribution of arrival times, which can easily be applied to determine the earliest and latest expected arrival to the next bus stop, alongside the most probable arrival time. The naïve arrival time prediction model implemented has a root mean square error of 5 to 19 seconds. In general, the absolute error of the prediction model decreases over time in each respective segment. The results from the GPS variance estimation problem is a model which can compare the variance for different environments along the route on a given trajectory.

Spatio-temporal data is a commonly used source of information. Using machine learning to analyse this kind of data can lead to many interesting and useful insights. In this thesis project, a novel public transportation spatio-temporal dataset is explored and analysed. The dataset contains 282 GB of positional events, spanning two weeks of time, from all public transportation vehicles in Östergötland county, Sweden. From the data exploration, three high-level problems are formulated: bus stop detection, GPS variance estimation, and arrival time prediction, also called trajectory forecasting. The bus stop detection problem is briefly discussed and solutions are proposed. Gaussian process regression is an effective method for solving regression problems. The GPS variance estimation problem is solved via the use of a mixture of Gaussian processes. A mixture of Gaussian processes is also used to predict the arrival time for public transportation buses. The arrival time prediction is from one bus stop to the next, not for the whole trajectory. The result from the arrival time prediction is a distribution of arrival times, which can easily be applied to determine the earliest and latest expected arrival to the next bus stop, alongside the most probable arrival time. The naïve arrival time prediction model implemented has a root mean square error of 5 to 19 seconds. In general, the absolute error of the prediction model decreases over time in each respective segment. The results from the GPS variance estimation problem is a model which can compare the variance for different environments along the route on a given trajectory.

Air transport has been growing over the last few years, and for this reason, maintaining the safety and improving navigation systems support is a constant challenge in congested airspace. An alternative that has been accepted worldwide because of its advantageous costs and benefits, involves the integration of satellite navigation systems with autonomous broadcasting systems on aircraft, which have allowed to extend situational awareness into areas without coverage of CNS/ATM systems, through an automatic dependent surveillance aircraft system called ADS-B network which allows flight profile sharing with nearby aircraft and ground stations indifferently. This system is being integrated with the monitoring and control systems of many countries, in response to the need of maintaining the air safety in increasingly trafficked airspaces, allowing better use of routes and decreasing operational costs of companies. The Colombian Air Force, through its Technological Development Center for Defense CETAD, is working on understanding and adapting this technology to utilize it in military operations., El transporte aéreo ha venido creciendo en los últimos años, motivo por el cual mantener la seguridad y mejorar el sistema de apoyo a la navegación son retos constantes en el espacio aéreo más congestionado. Una alternativa que ha sido aceptada, a nivel mundial, como ventajosa por sus costos y sus prestaciones, implica la integración de los sistemas satelitales de navegación con sistemas de radiodifusión autónomos en las aeronaves, que han permitido extender la alerta situacional a áreas en las que no se cuenta con cobertura de sistemas CNS/ATM a través de un sistema ADS-B de vigilancia automático dependiente de la aeronave que permite compartir su perfil de vuelo, con aeronaves cercanas y con estaciones de tierra indistintamente. Este sistema se está integrando a los sistemas de vigilancia y control de muchos países, como respuesta a la necesidad de mantener la seguridad aérea en espacios aéreos cada vez más transitados, lo que ha permitido una mejor utilización de las rutas y la disminución de costos operacionales a las empresas. La Fuerza Aérea Colombiana a través del Centro de Desarrollo Tecnológico para la Defensa - CETAD, está trabajando en entender y adaptar esta tecnología para poder explotarla en operaciones militares.

Air transport has been growing over the last few years, and for this reason, maintaining the safety and improving navigation systems support is a constant challenge in congested airspace. An alternative that has been accepted worldwide because of its advantageous costs and benefits, involves the integration of satellite navigation systems with autonomous broadcasting systems on aircraft, which have allowed to extend situational awareness into areas without coverage of CNS/ATM systems, through an automatic dependent surveillance aircraft system called ADS-B network which allows flight profile sharing with nearby aircraft and ground stations indifferently. This system is being integrated with the monitoring and control systems of many countries, in response to the need of maintaining the air safety in increasingly trafficked airspaces, allowing better use of routes and decreasing operational costs of companies. The Colombian Air Force, through its Technological Development Center for Defense CETAD, is working on understanding and adapting this technology to utilize it in military operations., El transporte aéreo ha venido creciendo en los últimos años, motivo por el cual mantener la seguridad y mejorar el sistema de apoyo a la navegación son retos constantes en el espacio aéreo más congestionado. Una alternativa que ha sido aceptada, a nivel mundial, como ventajosa por sus costos y sus prestaciones, implica la integración de los sistemas satelitales de navegación con sistemas de radiodifusión autónomos en las aeronaves, que han permitido extender la alerta situacional a áreas en las que no se cuenta con cobertura de sistemas CNS/ATM a través de un sistema ADS-B de vigilancia automático dependiente de la aeronave que permite compartir su perfil de vuelo, con aeronaves cercanas y con estaciones de tierra indistintamente. Este sistema se está integrando a los sistemas de vigilancia y control de muchos países, como respuesta a la necesidad de mantener la seguridad aérea en espacios aéreos cada vez más transitados, lo que ha permitido una mejor utilización de las rutas y la disminución de costos operacionales a las empresas. La Fuerza Aérea Colombiana a través del Centro de Desarrollo Tecnológico para la Defensa - CETAD, está trabajando en entender y adaptar esta tecnología para poder explotarla en operaciones militares.