Search

We introduce the first audio-visual dataset for traffic anomaly detection taken from real-world scenes, called MAVAD, with a diverse range of weather and illumination conditions. In addition, we propose a novel method named AVACA that combines visual and audio features extracted from video sequences by means of cross-attention to detect anomalies. We demonstrate that the addition of audio improves the performance of AVACA by up to 5.2%. We also evaluate the impact of image anonymization, showing only a minor decrease in performance averaging at 1.7%.

This paper presents a Keyword-driven and N-gram Graph based approach for Image Captioning (KENGIC). Most current state-of-the-art image caption generators are trained end-to-end on large scale paired image-caption datasets which are very laborious and expensive to collect. Such models are limited in terms of their explainability and their applicability across different domains. To address these limitations, a simple model based on N-Gram graphs which does not require any end-to-end training on paired image captions is proposed. Starting with a set of image keywords considered as nodes, the generator is designed to form a directed graph by connecting these nodes through overlapping n-grams as found in a given text corpus. The model then infers the caption by maximising the most probable n-gram sequences from the constructed graph. To analyse the use and choice of keywords in context of this approach, this study analysed the generation of image captions based on (a) keywords extracted from gold standard captions and (b) from automatically detected keywords. Both quantitative and qualitative analyses demonstrated the effectiveness of KENGIC. The performance achieved is very close to that of current state-of-the-art image caption generators that are trained in the unpaired setting. The analysis of this approach could also shed light on the generation process behind current top performing caption generators trained in the paired setting, and in addition, provide insights on the limitations of the current most widely used evaluation metrics in automatic image captioning., Comment: Published in the Digital Image Computing: Techniques and Applications, 2022 (DICTA 2022)

Current image description generation models do not transfer well to the task of describing human faces. To encourage the development of more human-focused descriptions, we developed a new data set of facial descriptions based on the CelebA image data set. We describe the properties of this data set, and present results from a face description generator trained on it, which explores the feasibility of using transfer learning from VGGFace/ResNet CNNs. Comparisons are drawn through both automated metrics and human evaluation by 76 English-speaking participants. The descriptions generated by the VGGFace-LSTM + Attention model are closest to the ground truth according to human evaluation whilst the ResNet-LSTM + Attention model obtained the highest CIDEr and CIDEr-D results (1.252 and 0.686 respectively). Together, the new data set and these experimental results provide data and baselines for future work in this area., Comment: 7 pages, 5 figures, 4 tables, to appear in LREC 2022 (P-VLAM workshop)

The emergence of new variants of SARS-CoV-2 is a major concern given their potential impact on the transmissibility and pathogenicity of the virus as well as the efficacy of therapeutic interventions. Here, we predict the mutability of all positions in SARS-CoV-2 protein domains to forecast the appearance of unseen variants. Using sequence data from other coronaviruses, pre-existing to SARS-CoV-2, we build statistical models that do not only capture amino-acid conservation but more complex patterns resulting from epistasis. We show that these models are notably superior to conservation profiles in estimating the already observable SARS-CoV-2 variability. In the receptor binding domain of the spike protein, we observe that the predicted mutability correlates well with experimental measures of protein stability and that both are reliable mutability predictors (ROC AUC ~0.8). Most interestingly, we observe an increasing agreement between our model and the observed variability as more data become available over time, proving the anticipatory capacity of our model. When combined with data concerning the immune response, our approach identifies positions where current variants of concern are highly overrepresented. These results could assist studies on viral evolution, future viral outbreaks and, in particular, guide the exploration and anticipation of potentially harmful future SARS-CoV-2 variants., Comment: 21 pages + supplementary information

Convolutional Gridding is a technique (algorithm) extensively used in Radio Interferometric Image Synthesis for fast inversion of functions sampled with irregular intervals on the Fourier plane. In this thesis, we propose some modifications to the technique to execute faster on a GPU. These modifications give rise to \textit{Hybrid Gridding} and \textit{Pruned NN Interpolation}, which take advantage of the oversampling of the Gridding Convolutional Function in Convolutional Gridding to try to make gridding faster with no reduction in the quality of the output. Our experiments showed that given the right conditions, Hybrid Gridding executes up to $6.8\times$ faster than Convolutional Gridding, and Pruned NN Interpolation is generally slower than Hybrid Gridding. The two new techniques feature the downsampling of an oversampled grid through convolution to accelerate the Fourier inversion. It is a well-known approximate technique which suffers from aliasing. In this thesis, we are re-proposing the technique as a \textit{Convolution-Based FFT Pruning} algorithm able to suppress aliasing below arithmetic noise. The algorithm uses the recently discovered least-misfit gridding functions, which through our experiments gave promising results, although not as good as expected from the related published work on the stated gridding functions. Nevertheless, our experiments showed that, given the right conditions, Convolutional-Based Pruning reduces the Fourier inversion execution time on a GPU by approximately a factor of $8\times$., Comment: This is a PhD Thesis read at the University of Malta

Propagation Phase Contrast Synchrotron Microtomography (PPC-SR${\mu}$CT) is the gold standard for non-invasive and non-destructive access to internal structures of archaeological remains. In this analysis, the virtual specimen needs to be segmented to separate different parts or materials, a process that normally requires considerable human effort. In the Automated SEgmentation of Microtomography Imaging (ASEMI) project, we developed a tool to automatically segment these volumetric images, using manually segmented samples to tune and train a machine learning model. For a set of four specimens of ancient Egyptian animal mummies we achieve an overall accuracy of 94-98% when compared with manually segmented slices, approaching the results of off-the-shelf commercial software using deep learning (97-99%) at much lower complexity. A qualitative analysis of the segmented output shows that our results are close in terms of usability to those from deep learning, justifying the use of these techniques.

Vision and language tasks such as Visual Relation Detection and Visual Question Answering benefit from semantic features that afford proper grounding of language. The 3D depth of objects depicted in 2D images is one such feature. However it is very difficult to obtain accurate depth information without learning the appropriate features, which are scene dependent. The state of the art in this area are complex Neural Network models trained on stereo image data to predict depth per pixel. Fortunately, in some tasks, its only the relative depth between objects that is required. In this paper the extent to which semantic features can predict course relative depth is investigated. The problem is casted as a classification one and geometrical features based on object bounding boxes, object labels and scene attributes are computed and used as inputs to pattern recognition models to predict relative depth. i.e behind, in-front and neutral. The results are compared to those obtained from averaging the output of the monodepth neural network model, which represents the state-of-the art. An overall increase of 14% in relative depth accuracy over relative depth computed from the monodepth model derived results is achieved., Comment: 9 pages, 2 figures

When dealing with multi-class classification problems, it is common practice to build a model consisting of a series of binary classifiers using a learning paradigm which dictates how the classifiers are built and combined to discriminate between the individual classes. As new data enters the system and the model needs updating, these models would often need to be retrained from scratch. This work proposes three learning paradigms which allow trained models to be updated without the need of retraining from scratch. A comparative analysis is performed to evaluate them against a baseline. Results show that the proposed paradigms are faster than the baseline at updating, with two of them being faster at training from scratch as well, especially on larger datasets, while retaining a comparable classification performance., Comment: 10 pages, 6 figures

Maltese, the national language of Malta, is spoken by approximately 500,000 people. Speech processing for Maltese is still in its early stages of development. In this paper, we present the first spoken Maltese corpus designed purposely for Automatic Speech Recognition (ASR). The MASRI-HEADSET corpus was developed by the MASRI project at the University of Malta. It consists of 8 hours of speech paired with text, recorded by using short text snippets in a laboratory environment. The speakers were recruited from different geographical locations all over the Maltese islands, and were roughly evenly distributed by gender. This paper also presents some initial results achieved in baseline experiments for Maltese ASR using Sphinx and Kaldi. The MASRI-HEADSET Corpus is publicly available for research/academic purposes., Comment: 8 pages, 2 figures, 4 tables, 1 appendix. Appears in Proceedings of the 12th edition of the Language Resources and Evaluation Conference (LREC'20)

In this paper we describe $Vja\dot{g}\dot{g}$, a battery-aware journey detection algorithm that executes on the mobile device. The algorithm can be embedded in the client app of the transport service provider or in a general purpose mobility data collector. The thick client setup allows the customer/participant to select which journeys are transferred to the server, keeping customers in control of their personal data and encouraging user uptake. The algorithm is tested in the field and optimised for both accuracy in registering complete journeys and battery power consumption. Typically the algorithm can run for a full day without the need of recharging and more than 88% of journeys are correctly detected from origin to destination, whilst 12% would be missing part of the journey.

The Reynolds-Averaged Navier-Stokes equations and the Large-Eddy Simulation equations can be coupled using a transition function to switch from a set of equations applied in some areas of a domain to the other set in the other part of the domain. Following this idea, different time integration schemes can be coupled. In this context, we developed a hybrid time integration scheme that spatially couples the explicit scheme of Heun and the implicit scheme of Crank and Nicolson using a dedicated transition function. This scheme is linearly stable and second-order accurate. In this paper, an extension of this hybrid scheme is introduced to deal with a temporal adaptive procedure. The idea is to treat the time integration procedure with unstructured grids as it is performed with Cartesian grids with local mesh refinement. Depending on its characteristic size, each mesh cell is assigned a rank. And for two cells from two consecutive ranks, the ratio of the associated time steps for time marching the solutions is $2$. As a consequence, the cells with the lowest rank iterate more than the other ones to reach the same physical time. In a finite-volume context, a key ingredient is to keep the conservation property for the interfaces that separate two cells of different ranks. After introducing the different schemes, the paper recalls briefly the coupling procedure, and details the extension to the temporal adaptive procedure. The new time integration scheme is validated with the propagation of 1D wave packet, the Sod's tube, and the transport of a bi-dimensional vortex in an uniform flow.

This paper addresses how two time integration schemes, the Heun's scheme for explicit time integration and the second-order Crank-Nicolson scheme for implicit time integration, can be coupled spatially. This coupling is the prerequisite to perform a coupled Large Eddy Simulation / Reynolds Averaged Navier-Stokes computation in an industrial context, using the implicit time procedure for the boundary layer (RANS) and the explicit time integration procedure in the LES region. The coupling procedure is designed in order to switch from explicit to implicit time integrations as fast as possible, while maintaining stability. After introducing the different schemes, the paper presents the initial coupling procedure adapted from a published reference and shows that it can amplify some numerical waves. An alternative procedure, studied in a coupled time/space framework, is shown to be stable and with spectral properties in agreement with the requirements of industrial applications. The coupling technique is validated with standard test cases, ranging from one-dimensional to three-dimensional flows.

Visual Relationship Detection is defined as, given an image composed of a subject and an object, the correct relation is predicted. To improve the visual part of this difficult problem, ten preprocessing methods were tested to determine whether the widely used Union method yields the optimal results. Therefore, focusing solely on predicate prediction, no object detection and linguistic knowledge were used to prevent them from affecting the comparison results. Once fine-tuned, the Visual Geometry Group models were evaluated using Recall@1, per-predicate recall, activation maximisations, class activation maps, and error analysis. From this research it was found that using preprocessing methods such as the Union-Without-Background-and-with-Binary-mask (Union-WB-and-B) method yields significantly better results than the widely used Union method since, as designed, it enables the Convolutional Neural Network to also identify the subject and object in the convolutional layers instead of solely in the fully-connected layers.

Image caption generation systems are typically evaluated against reference outputs. We show that it is possible to predict output quality without generating the captions, based on the probability assigned by the neural model to the reference captions. Such pre-gen metrics are strongly correlated to standard evaluation metrics., Comment: 13 pages, 6 figures This publication will appear in the Proceedings of the First Workshop on Shortcomings in Vision and Language (2018). DOI to be inserted later

The past few years have witnessed renewed interest in NLP tasks at the interface between vision and language. One intensively-studied problem is that of automatically generating text from images. In this paper, we extend this problem to the more specific domain of face description. Unlike scene descriptions, face descriptions are more fine-grained and rely on attributes extracted from the image, rather than objects and relations. Given that no data exists for this task, we present an ongoing crowdsourcing study to collect a corpus of descriptions of face images taken `in the wild'. To gain a better understanding of the variation we find in face description and the possible issues that this may raise, we also conducted an annotation study on a subset of the corpus. Primarily, we found descriptions to refer to a mixture of attributes, not only physical, but also emotional and inferential, which is bound to create further challenges for current image-to-text methods., Comment: Proceedings of the 11th edition of the Language Resources and Evaluation Conference (LREC'18)

Automatic description generation from natural images is a challenging problem that has recently received a large amount of interest from the computer vision and natural language processing communities. In this survey, we classify the existing approaches based on how they conceptualize this problem, viz., models that cast description as either generation problem or as a retrieval problem over a visual or multimodal representational space. We provide a detailed review of existing models, highlighting their advantages and disadvantages. Moreover, we give an overview of the benchmark image datasets and the evaluation measures that have been developed to assess the quality of machine-generated image descriptions. Finally we extrapolate future directions in the area of automatic image description generation., Comment: Journal of Artificial Intelligence Research 55, 409-442, 2016

An equilateral set (or regular simplex) in a metric space $X$, is a set $A$ such that the distance between any pair of distinct members of $A$ is a constant. An equilateral set is standard if the distance between distinct members is equal to $1$. Motivated by the notion of frame-functions, as introduced and characterized by Gleason in \cite{Gl}, we define an equilateral weight on a metric space $X$ to be a function $f:X\to \mathbb R$ such that $\sum_{i\in I}f(x_i)=W$, for every maximal standard equilateral set $\{x_i:i\in I\}$ in $X$, where $W\in\mathbb R$ is the weight of $f$. In this paper we characterize the equilateral weights associated with the unit ball $B^n$ of $\mathbb R^n$ as follows: For $n\ge 2$, every equilateral weight on $B^n$ is constant., Comment: Any comments are welcome

A radio interferometer indirectly measures the intensity distribution of the sky over the celestial sphere. Since measurements are made over an irregularly sampled Fourier plane, synthesising an intensity image from interferometric measurements requires substantial processing. Furthermore there are distortions that have to be corrected. In this thesis, a new high-performance image synthesis tool (imaging tool) for radio interferometry is developed. Implemented in C++ and CUDA, the imaging tool achieves unprecedented performance by means of Graphics Processing Units (GPUs). The imaging tool is divided into several components, and the back-end handling numerical calculations is generalised in a new framework. A new feature termed compression arbitrarily increases the performance of an already highly efficient GPU-based implementation of the w-projection algorithm. Compression takes advantage of the behaviour of oversampled convolution functions and the baseline trajectories. A CPU-based component prepares data for the GPU which is multi-threaded to ensure maximum use of modern multi-core CPUs. Best performance can only be achieved if all hardware components in a system do work in parallel. The imaging tool is designed such that disk I/O and work on CPU and GPUs is done concurrently. Test cases show that the imaging tool performs nearly 100$\times$ faster than another general CPU-based imaging tool. Unfortunately, the tool is limited in use since deconvolution and A-projection are not yet supported. It is also limited by GPU memory. Future work will implement deconvolution and A-projection, whilst finding ways of overcoming the memory limitation., Comment: This is a Masters Thesis read at the University of Malta

Despite great technological importance and many investigations, a material with measured hardness comparable to that of diamond or cubic boron nitride has yet to be identified. Combined theoretical and experimental investigations led to the discovery of a new polymorph of titanium dioxide with titanium nine-coordinated to oxygen in the cotunnite (PbCl2) structure. Hardness measurements on the cotunnite-structured TiO2 synthesized at pressures above 60 GPa and temperatures above 1000 K reveal that this material is the hardest oxide yet discovered. Furthermore, it is one of the least compressible (with a measured bulk modulus of 431 GPa) and hardest (with a microhardness of 38 GPa) polycrystalline materials studied thus far., Comment: This is full version of the paper published as Brief Communications in Nature, 410, 653-654

This paper presents the measurement of the scattering of 172 MeV/c muons in assorted materials, including liquid hydrogen, motivated by the need to understand ionisation cooling for muon acceleration. Data are compared with predictions from the Geant 4 simulation code and this simulation is used to deconvolute detector effects. The scattering distributions obtained are compared with the Moliere theory of multiple scattering and, in the case of liquid hydrogen, with ELMS. With the exception of ELMS, none of the models are found to provide a good description of the data. The results suggest that ionisation cooling will work better than would be predicted by Geant 4.7.0p01., Comment: pdfeTeX V 3.141592-1.21a-2.2, 30 pages with 22 figures