Search

New chemical entities that humans and animals are likely to be exposed to are tested using various in vitro and in vivo toxicological assays to understand the potential for harm. The increasing power of in silico analysis in combination with in vitro techniques is being exploited for the ability to predict in vivo outcomes such as toxicity, thereby reducing the requirement for animal testing. Previous studies in our laboratory have found that dietary phenobarbital treatment of rats results in dysregulation of the hepatic miRNAome, including prominent alteration of the expression of the miR-200a/b/429 cluster and miR-182/96 cluster. In this thesis I explore the use of a cell culture system, rat hepatocyte-like B13/H cells, to interrogate the underlying mechanisms of phenobarbital-mediated toxicity. Through this model I explore the function of phenobarbital-dysregulated microRNAs to determine their roles as markers of phenobarbital treatment in the rat. We find that the B13/H cells provide a useful system for studying the phenobarbital response in vitro, and how these responses relate to the effects of oral exposure to phenobarbital in vivo in the rat. In this respect, the B13/H cells appear representative of the rat response to phenobarbital, which is different to that of the mouse. With a focus on the phenobarbital-mediated dysregulation of the miR-182/96 cluster, I have explored the consequences of phenobarbital treatment on the B13/H cell model and related this to the in vivo effects. The data suggest that these microRNAs appear to control phenobarbital-mediated perturbation of glycolysis, cell transformation and methylation patterns.

We develop a family of Bayesian algorithms built around Gaussian processes for various problems posed by sensor networks. We firstly introduce an iterative Gaussian process for multi-sensor inference problems, and show how our algorithm is able to cope with data that may be noisy, missing, delayed and/or correlated. Our algorithm can also effectively manage data that features changepoints, such as sensor faults. Extensions to our algorithm allow us to tackle some of the decision problems faced in sensor networks, including observation scheduling. Along these lines, we also propose a general method of global optimisation, Gaussian process global optimisation (GPGO), and demonstrate how it may be used for sensor placement. Our algorithms operate within a complete Bayesian probabilistic framework. As such, we show how the hyperparameters of our system can be marginalised by use of Bayesian quadrature, a principled method of approximate integration. Similar techniques also allow us to produce full posterior distributions for any hyperparameters of interest, such as the location of changepoints. We frame the selection of the positions of the hyperparameter samples required by Bayesian quadrature as a decision problem, with the aim of minimising the uncertainty we possess about the values of the integrals we are approximating. Taking this approach, we have developed sampling for Bayesian quadrature (SBQ), a principled competitor to Monte Carlo methods. We conclude by testing our proposals on real weather sensor networks. We further benchmark GPGO on a wide range of canonical test problems, over which it achieves a significant improvement on its competitors. Finally, the efficacy of SBQ is demonstrated in the context of both prediction and optimisation.

The time value of money (TVM) equation is a key equation in economics and finance. It takes the form of an nth order polynomial having n roots. It is usual to calculate and use only one root (interest rate). The remaining (n-1) roots are mostly complex or negative, and are usually discarded. In this thesis it is shown that the unorthodox roots (interest rates) should not be ignored because they have utility and meaning. New expressions are developed for existing concepts in economics and finance. The concepts include duration in bond mathematics, net present value (NPV) in capital budgeting, the value of a stock of capital in capital theory, and the total charge for credit in loan analysis. The new expressions for these concepts employ all possible interest rates as components. The new expressions provide solutions to puzzles. In bond mathematics, the new equation for duration delivers what previous formulas for duration fail to provide: an accurate measure of the impact of a change in interest rate on the price of a bond. In capital budgeting, the new equation for NPV offers a resolution to the debate about the relative merits of NPV and internal rate of return (IRR) as investment criteria. In economics, a solution is proposed to the reswitching debate in the Cambridge capital controversies. In credit analysis, a new relationship is developed between the total charge for credit and the orthodox measure of the cost of a loan, the annual percentage rate (APR). The result implies that the complicated APR need no longer be the focus of consumer credit legislation; the total charge for credit and its variants suffice. The new analysis not only employs all interest rates, it also endows them with meaning. The suggested interpretation of a complete cluster of interest rates sheds new light on the meaning of orthodox rates such as IRR and APR.

An investigation into the oceanographic influences affecting the surface mixed layer and underlying water column at Ocean Weather Station Lima (57°N, 20°W) in the Northeast Atlantic is described. Data from an intensive field programme in 1987 are presented. The seasonal thermocline and structure of the underlying water column was influenced by the presence and admixture of various water masses. North Atlantic Water (NAW) was observed to enter the region from the south. Modified NAW (fresher and cooler) entered the region at the surface from the north, probably due to Ekman drift. During the summer months, this fresher water was 'trapped' in the surface mixed layer - thus increasing the stability of the upper water column. Below a depth of 450 m, Sub-Arctic Intermediate Water entered the region from the west. Distributions of density and dynamic height indicate the presence of baroclinic instability and the considerable local influence of the Hatton Bank on circulation and hydrography of the region. Calculation of the annual heat budget indicates an annual loss of around 80 Wm⁻¹; equivalent to an overall heat input by advection of 0.02°C per day and to a spatial meridional temperature gradient of -1.2 x 10⁻⁶°Cm⁻¹. The open nature of the heat budget has serious consequences for the application of one-dimensional mixed layer models. Comparisons between observed and modelled sea surface temperature, mixed layer depths and heat content show good agreement at times of positive net surface heat flux, when there is little or no input of heat by advection. However, during a period of cooling, the model grossly over-predicted the depth of the surface mixed layer.

Subject to several assumptions, sufficiently advanced reinforcement learners would likely face an incentive and likely have an ability to intervene in the provision of their reward, with catastrophic consequences. In this thesis, I develop a theory of pessimism and show how it can produce safe advanced artificial agents. Not only do I demonstrate that the assumptions mentioned above can be avoided; I prove theorems which demonstrate safety. First, I develop an idealized pessimistic reinforcement learner. For any given novel event that a mentor would never cause, a sufficiently pessimistic reinforcement learner trained with the help of that mentor would probably avoid causing it. This result is without precedent in the literature. Next, on similar principles, I develop an idealized pessimistic imitation learner. If the probability of an event when the demonstrator acts can be bounded above, then the probability can be bounded above when the imitator acts instead; this kind of result is unprecedented when the imitator learns online and the environment never resets. In an environment that never resets, no one has previously demonstrated, to my knowledge, that an imitation learner even exists. Finally, both of the agents above demand more efficient algorithms for high-quality uncertainty quantification, so I have developed a new kernel for Gaussian process modelling that allows for log-linear time complexity and linear space complexity, instead of a naïve cubic time complexity and quadratic space complexity. This is not the first Gaussian process with this time complexity-inducing points methods have linear complexity-but we do outperform such methods significantly on regression benchmarks, as one might expect given the much higher dimensionality of our kernel. This thesis shows the viability of pessimism with respect to well-quantified epistemic uncertainty as a path to safe artificial agency.

A fundamental concern in the deployment of artificial agents in real-life is their capacity to quickly adapt to their surroundings. Traditional reinforcement learning (RL) struggles with this requirement in two ways. Firstly, iterative exploration of unconstrained environment dynamics yields numerous uninformative updates and consequently slow adaptation. Secondly, final policies have no capacity to adapt to future observations and have to either slowly learn indefinitely or retrain entirely as observations occur. This thesis explores two formulations aimed at addressing these issues. The consideration of entire task distributions in meta-RL evolves policies quickly adapting to specific instances on their own. By forcing agents to specifically request feedback, Active RL enforces selective observations and updates. Both of these formulations reduce to a Bayes-Adaptive setting in which a probabilistic belief over possible environments is maintained. Many existing solutions only provide asymptotic guarantees that are of limited use in practical contexts. We develop a variational approach to approximate belief management and support its validity empirically through a broad range of ablations. We then consider recently successful planning approaches but uncover and discuss obstacles in their application to the discussed settings. An important factor influencing the data requirements and stability of RL systems is the choice of appropriate hyperparameters. We develop a Bayesian optimisation approach exploiting the iterative structure of training processes whose empiric performance exceeds that of existing baselines. A final contribution of this thesis concerns increasing the scalability and expressiveness of Gaussian Processes (GPs). While we make no direct use of the presented framework, GPs have been used to model probabilistic beliefs in closely related settings.

Both human and algorithmic decision making can be complex. To truly intertwine the two, algorithms need to understand the human decision making process and humans need to have transparent understanding of algorithmic decision analytics. In this thesis, we leverage Bayesian Gaussian processes to better understand people as well as offering a framework for people to better understand algorithms. Key to such transparency on both sides is the robust and principled reporting of uncertainty and careful consideration of preference: the latter being the foundational basis of human decision making. We consider pairwise preference modelling in which the strength of preference (e.g. strong or weak) can be seamlessly taken into account in the model, as well as confidence (uncertainty). We contribute to probabilistic functional regression by enabling the functional predictor space to be interpretable. We develop a principled, robust tool for understanding black-box algorithms, leading to a novel method for the interpretability of features in local domains. Finally, using our heteroscedastic ordinal regression model, we analyse the relationship between the features of occupations (such as their requirement for manual dexterity) and their expected share of US employment in 2030. Together, the approaches and algorithms we provide create a suite of models that increase our decision making capability with human-centric data.

Bayesian machine learning (ML) models have long been advocated as an important tool for safe artificial intelligence. Yet, little is known about their vulnerability against adversarial attacks. Such attacks aim to cause undesired model behaviour (e.g. misclassification) by crafting small perturbations to regular inputs which appear to be insignificant to humans (e.g. slight blurring of image data). This fairly recent phenomenon has undermined the suitability of many ML models for deployment in safety critical applications. In this thesis, we investigate how robust Bayesian ML models are against adversarial attacks, focussing on Gaussian process (GP) and Bayesian neural network (BNN) classification models. In particular, for GP classification models, we derive guarantees for the robustness against adversarial attacks that facilitate the evaluation of their suitability for a given safety critical application. Furthermore we investigate whether better posterior approximations benefit the adversarial robustness of BNNs, comparing the adversarial robustness resulting from high quality posterior approximations to that resulting from deterministic approximations in a range of experiments. We find that well approximated BNNs tend to be empirically more susceptible to adversarial attacks than deterministic neural network models for most popular priors. This calls for caution when deploying such BNNs in safety critical applications. Lastly, we show that using GPs in a Bayesian optimisation framework, it is possible to craft successful adversarial perturbations in black-box scenarios (where the attacked model is a priori unknown and can only be studied by querying it) with fewer model queries than previously needed. This facilitates a better evaluation of the adversarial robustness of a model in black-box scenarios.

The design of models which are appropriate for specific tasks is an important activity in machine learning. This thesis considers multiple ways in which knowledge about the task at hand can be incorporated into the design of a machine learning model: (i) by using Bayesian models, which incorporate prior knowledge in probability distributions; (ii) by designing models to respect the symmetries of the task; (iii) by adapting models in a practical setting. For Bayesian models, we propose a probabilistic numeric method for computing integrals which arise in the context of Bayesian inference. Respecting the symmetries of a task comes under the framework of group equivariance and invariance. We provide a theoretical investigation of Deep Sets, a prominent permutation-invariant model, and propose an extension of the SE(3)-Transformer, an SE(3)-equivariant model. Finally, we present an investigation of the application of machine learning techniques to problems in atmospheric physics, in which the relevant models are adapted to specific known features of the problem.

We propose a novel family of Bayesian learning algorithms for online portfolio selection that overcome many of the shortcomings of traditional techniques, including selection bias (the failure to cover a broad universe of assets), data-snooping bias (the risk that a trading strategy's performance on past data is inflated due to hyperparameter overfitting) and a lack of robustness to transaction costs. As the basis for this novel family, we develop a Bayesian treatment of the online passive-aggressive and gradient descent algorithms, some of the most popular algorithms in the literature. Our approach starts from a probabilistic interpretation of the underlying objective functions and enables uncertainty modelling, probabilistic predictions as well as automatic, data-dependent hyperparameter tuning. We conclude by testing our proposals on real-world financial data. We further benchmark our framework on a wide range of canonical test problems, over which it achieves a significant improvement on its competitors. Beyond online portfolio selection, our algorithms contribute to the theory of adaptive gradient methods by equipping these with uncertainty estimates and a self-tuning mechanism for the learning rate parameter, which constitutes a major milestone in the area of Bayesian inference for neural networks.

In order to integrate machine learning into human decision-making in a useful way, we must trust machine learning systems enough in our reasoning processes. To evaluate a system's trustworthiness, humans naturally seek interpretable causal systems to understand outcomes, make decisions, and integrate feedback. This thesis presents four explorations into interpretable causal systems, progressing from associational interpretability up the "Ladder of Causation" to counterfactual representation. In the first contribution, I introduce a Bayesian nonparametric method for calculating the expected value and volatility of gradients in the data; this helps inform further experiments for deriving causal effects and interpreting changes when faced with decision-making scenarios. In the second, I show how latent expert knowledge can be collected to produce an interpretable model with policy-relevant consequences. In the third, I interpret time series treatment effect inference as a problem naturally handled by multitask Gaussian processes, addressing several shortcomings in a popular approach. In the fourth, I consider twin networks, a novel but understudied method of representing counterfactual questions; I show that one can sometimes improve on computational cost, error, and insight by representing counterfactual problems as twin networks. I then introduce a prototype probabilistic programming engine that natively leverages the efficiencies gained from twin networks. I close by considering the future of how interpretability and causality might integrate into machine learning-driven decision-making and speculate that causality is a key component in achieving efficient, generalisable intelligence in a variety of applications.

For decades, the holy grail of many fields of science, mathematics and statistics has been the ability to predict the future or, more quantitatively, to forecast the behaviours of quantities of interest. Examples range from predicting the value of financial products to forecasting energy consumption and several other applications. While historically practitioners have focused on designing handcrafted methods for specific prediction tasks, the tools of Machine Learning (ML) promise to automate data analyses that extract patterns and structures and leverage this information in future forecasts. The addition of the probabilistic framework of Bayesian Machine Learning enables the inclusion of useful prior knowledge and provides predictive uncertainty quantification. This thesis focuses on generative modeling, an important class of algorithms that stands at the core of Bayesian ML: by making structural assumptions on the generative processes that produced the data, generative models intend to explain the observed variability and yield a deep, interpretable understanding of data. We present three original contributions, which leverage model-based machine learning to gain insights about the data-generating process and subsequently leverage them to provide reliable predictions. First, we propose a novel probabilistic framework based on constrained Gaussian Processes that incorporates derivative information in the form of ordinary differential equations. Second, we propose a novel noise model for Gaussian Processes that, in conjunction with state-of-the-art moment matching and adversarial techniques, addresses the problem of parameter inference in systems of stochastic differential equations. Finally, we move our attention to the optimization and sampling algorithms employed for Bayesian inference. In particular, we use Gaussian Process latent variable models to learn lower-dimensional space embeddings wherein optimizers and samplers can operate more effectively, thereby producing a meta-algorithm that can be applied to these inference methods to improve efficiency.

This thesis studies, through the prism of image classification, the influence of the input data on learning deep representations. Indeed, data is abundant but also multifaceted: samples might come from a single or multiple domains, or they can also be more or less labelled. We show in this thesis that methods to learn deep network representations should be designed according to the type of input data at hand. Our work covers different training data configurations. We start with the multiple visual domains setting. We tackle this challenge by modifying the network architecture with residual adapters, domain specific modules which lead to a high degree of parameter sharing across domains with better transfer learning performance than standard fine-tuning. Second, we consider the settings where a part of the data is unlabelled. This thesis spans different levels of supervision but, in all cases, we promote the systematic use of self-supervision pre-training. First, we consider the semi-supervised setting with few labels per class for which we propose an alternating optimisation scheme to avoid overfitting the scarse labelled data. We then learn to discover new classes given some known classes by proposing a two-headed network which jointly learns to cluster the unlabelled data and to classify both the labelled and unlabelled samples. To this end, we propose a clustering loss which learns to connect pairs of samples based on their similarity at the feature level. We show that this loss, proposed for novel class discovery, can also be used to perform deep clustering by choosing a nearest neighbours based similarity. From another perspective, we look at the influence of the data on the deep representations through the lens of saliency. To interpret which parts of the image contribute the most to the data representation, we introduce a unifying saliency framework based on the learning mechanism of convolutional neural networks. This framework can then be used to derive new or existing backpropagation based saliency methods.

This thesis studies the problem of tracking the extremum of an objective function that is latent, noisy and expensive to evaluate. This problem is notable because many large-scale learning systems with complex models operating on non-stationary data have meta-problems whose solutions require the tracking of an evolving extremum. We start by describing dynamic optimization problems and model them using spatiotemporal Gaussian process priors. We construct an intelligent search mechanism that uses the learnt insights to skillfully guide the search by dynamically modifying the feasible search region as a device to keep track of the evolution. We also show that this mechanism induces a natural approximation scheme for cases where the number of samples for the model becomes too expensive for inference. We test the resulting method on synthetic and real-world problems. In the next part of the thesis, we demonstrate the utility of the method on pertinent real-world meta-problems occurring in essential and widely-used large-scale learning systems. We start by proposing intelligent meta-heuristics, which are derived from our resulting method, for the learning rate adaptation meta-problem that is part of many large-scale learning systems trained by stochastic gradient descent. We incorporate hyper-gradients into the model for the meta-problem and use the additional insights to aid the intelligent meta-heuristic search. We test the resulting adaptation scheme on training various types of neural network models to demonstrate its utility through the significant performance improvements. We also exploit the wealth of prior art to construct a configuration oracle that adapts various types of system parameters in online trading algorithms as they interact with noisy and nonstationary financial data. We test the oracle on multiple algorithms operating using widely-accepted trading principles and demonstrate notable performance gains. The importance of this study is that it proposes a mechanism for intelli- gently exploiting vast amounts of prior knowledge that exists for these meta- problems. The intelligent use of this knowledge significantly improves the performance of the primary learning systems, which are vital for solving many practical problems, operating in dynamic environments.

We are concerned primarily with improving the practical applicability of Bayesian optimization. We make contributions in three key areas. We develop an intuitive online stopping criterion, allowing only as many steps as necessary to achieve the desired target to be taken. By combining this with intelligent online switching between acquisition functions and pure local optimization we are also able to substantially improve convergence to the local minimum associated with our final solution. In cases where a continuum of reduced cost, but also reduced accuracy, evaluations are available we develop a Bayesian Optimization acquisition function to select both the location and cost of each evaluation. We achieve this with lower overheads than previous methods, translating to a real increase in performance. Part of this improvement is achieved by way of a new, more efficient, method for generating support points to sample the minimum of a Gaussian process. Further, in the case that the reduced cost estimates are unbiased we show that a practical solution cannot exist in most cases without also taking into consideration both computational overheads and a restriction on available resources. Given this knowledge we then develop a method which provides a viable solution in this setting. Finally, we outline practical implementation details for Bayesian optimization which allow substantial reductions in the overhead costs without changing the theoretical properties of optimization. This is primarily achieved by use of adaptive quadrature to marginalize Gaussian process hyperparameters in place of the more common slice sampling approach.

In this thesis we present a variety of new, continuous, Bayesian Gaussian-process-driven Cox process models. These are used to model sparse event data distributed on a continuous domain, where the events may have a tendency to cluster. These find direct use in application areas ranging from disease incidence modelling through to statistical cosmology, where the distribution of galaxies in the universe is weakly clustered due to the effects of dark matter. They may also be deployed in a more abstract sense, for example as a structured prior for network communications. In previous work, the difficulty of performing inference in Gaussian-process-driven Cox processes has hindered their application to large, high-dimensional datasets. We develop novel and computationally efficient inference schemes for these models as well as our own extensions to them, demonstrating an improvement on the existing state of the art using real data. In particular, we present the first known variational inference scheme for such models, which scales linearly with the size of the dataset. Spurred on to consider the problem of computationally efficient Bayesian inference in general, we tackle model evidence estimation. Arriving at an accurate measure of model evidence quickly allows for the objective measure of model fit, and ensures we select a set of assumptions which most closely embody the data-generating process. We deviate from the traditional core Monte Carlo estimator, and instead present a computationally efficient general Bayesian quadrature scheme for model evidence computation. This is the first such scheme which can be shown to be demonstrably wall-clock competitive with state of the art Monte Carlo approaches.

This thesis presents novel functional regression methods for non-linear data and develops core Gaussian Process theory for arc lengths. The need for richer functional regression methods and theory around Gaussian Process arc lengths is established in the introduction. Chapter 1 commences with the requisite background material and Chapter 2 works through the fundamentals of probability, highlighting key messages with a worked example. Gaussian Processes are introduced in Chapter 3, developing the core theory needed to extend the functional methods outlined in Chapter 4 and to prepare for considering the arc length dis- tribution of Gaussian Processes. Three novel Gaussian Process functional regression models are introduced in Chapter 5, building upon existing models and furthering the probabilistic approach to functional regression, namely the Gaussian Process Functional Index Model, the Gaussian Process Functional Additive Model and the Gaussian Process Functional Generalised Additive Model. In each model a kernel function is derived whilst outlining inference and prediction. It is clearly demonstrated that the functional regression models out perform their competitors on synthetic examples. In Chapter 6, the arc length distribution of a Gaussian Process is de- rived. By tackling the arc length integrand, a novel approach to the one dimensional arc length mean is presented. This equips us with the tools to approximate the arc length of a vector Gaussian Process. We derive the arc length distribution for a prior and posterior Gaussian Process in terms of its kernel functions and hyperparamaters. Chapter 7 signals a return to functional regression where two challenging real world functional data sets are tackled. The novel Gaussian Process methods provide competitive results in a number of cases, supporting the potential for the new methods. The thesis concludes with a vision for future work.

Robots need to be able to adapt to their surroundings. Robots whose core function relates to the rehabilitation and assistance of humans, need to be able to adapt to humans. But not all humans, one human in particular: their user. This is an adaptive control problem which speaks of the need for powered prostheses to have anthropomorphic adaptive capabilities. However, it is inconceivable that all possible movements, dynamics and tasks can be preprogrammed into such a system. Prostheses, like robots, need to be able to learn and improve, either by themselves unsupervised, or with the help of human supervision. The United States alone is home to more than two million amputees, where below-knee amputations is the most common form. The same country is also home to 4.7 million stroke survivors, many of whom could make use of rehabilitation robots. Vascular diseases, such as diabetes, cause 54% of all amputations and the number of diabetes and pre-diabetes cases in the United States currently exceeds 100 million and growing. Whilst this thesis does not attack the root causes, it does present a three pronged approach for framing the adaptive control problem required for effective rehabilitation. We develop methodology that structures the problem into one of learning a control framework for lower extremity active prosthetics. First, we consider incidence detection as a key element of any control strategy. We study the dynamics of falling under the aegis of classification and state-space modelling. We use the standard Kalman smoother to inpaint missing observations. Then dimensionally reduce the feature space to demonstrate increased classification accuracy, compared with our reference study. Secondly, we move to the temporal segmentation of similar time-series. This is done using the Bayesian nonparametric paradigm within the framework of state-space modelling and probabilistic programming. Models are rarely correct for real world data. Rather than comparing models that vary in complexity, this approach fits a single model that can adapt its complexity to the observations. This unbounded analysis of the state-space is required given the need for the proposed control framework to grow with new observations. Finally, we investigate multiple incidents of walking velocity, and propose a learned control strategy able to smoothly transition between the incidents. We combine Gaussian process regression with impedance control, and gait-cycle regression to form a locomotion envelope. The learned control capabilities allow the wearer to smoothly transition between self-selected velocities.

Imagine a world where computational simulations can be inverted as easily as running them forwards, where data can be used to refine models automatically, and where the only expertise one needs to carry out powerful statistical analysis is a basic proficiency in scientific coding. Creating such a world is the ambitious long-term aim of probabilistic programming. The bottleneck for improving the probabilistic models, or simulators, used throughout the quantitative sciences, is often not an ability to devise better models conceptually, but a lack of expertise, time, or resources to realize such innovations. Probabilistic programming systems (PPSs) help alleviate this bottleneck by providing an expressive and accessible modeling framework, then automating the required computation to draw inferences from the model, for example finding the model parameters likely to give rise to a certain output. By decoupling model specification and inference, PPSs streamline the process of developing and drawing inferences from new models, while opening up powerful statistical methods to non-experts. Many systems further provide the flexibility to write new and exciting models which would be hard, or even impossible, to convey using conventional statistical frameworks. The central goal of this thesis is to improve and extend PPSs. In particular, we will make advancements to the underlying inference engines and increase the range of problems which can be tackled. For example, we will extend PPSs to a mixed inference-optimization framework, thereby providing automation of tasks such as model learning and engineering design. Meanwhile, we make inroads into constructing systems for automating adaptive sequential design problems, providing potential applications across the sciences. Furthermore, the contributions of the work reach far beyond probabilistic programming, as achieving our goal will require us to make advancements in a number of related fields such as particle Markov chain Monte Carlo methods, Bayesian optimization, and Monte Carlo fundamentals.

X-ray crystallography has been the workhorse behind most 3D protein structures, which are crucial in the understanding of their biological function and interaction with other molecules. However, a major rate-limiting step in X-ray crystallography remains obtaining suitable protein crystals. The best approach to crystallise a protein can be summarized as a random-screen-and-wait procedure, with little to no readout from experiments that do not produce crystals. This project aims to make the most out of present screening practice, by establishing objective analyses of sparse-matrix screening experiments that extract informative readouts from this standard front-line experiment, independent of whether it yields visible crystals or not. We have developed methods to objectively characterize crystallization outcome based on image analysis, enabling several things. Firstly, the ranking of droplets based on their likelihood of crystallinity to increase the efficiency and accuracy of human visual identification of crystals. Secondly, fingerprints of the collective precipitation behaviour of a protein across standard sparse-matrix can be generalised, and compared objectively to fingerprints of historical experiments, to assess crystallizability and infer optimization strategies based on past successes. Thirdly, clear drops can be automatically identified, and mapped to chemical components in a sparse-matrix screen to suggest alternative buffers for protein formulation. Fourthly, TeXRank, a user interface could be developed to present and make all algorithm output accessible for daily use. Fifthly, the associated data mining led us to evaluate the strategies for setting up screening experiments with limited protein samples, based on over ten years of crystallization data at the Structural Genomics Consortium, Oxford. Our methods capitalizes on present day standard screening procedure and hardware to extract useful information, bypassing laborious and subjective evaluation of each droplet.

The demand for automated decision making, learning and inference in uncertain, risk sensitive and dynamically changing situations presents a challenge: to design computational approaches that promise to be widely deployable and flexible to adapt on the one hand, while offering reliable guarantees on safety on the other. The tension between these desiderata has created a gap that, in spite of intensive research and contributions made from a wide range of communities, remains to be filled. This represents an intriguing challenge that provided motivation for much of the work presented in this thesis. With these desiderata in mind, this thesis makes a number of contributions towards the development of algorithms for automated decision-making and inference under uncertainty. To facilitate inference over unobserved effects of actions, we develop machine learning approaches that are suitable for the construction of models over dynamical laws that provide uncertainty bounds around their predictions. As an example application for conservative decision-making, we apply our learning and inference methods to control in uncertain dynamical systems. Owing to the uncertainty bounds, we can derive performance guarantees of the resulting learning-based controllers. Furthermore, our simulations demonstrate that the resulting decision-making algorithms are effective in learning and controlling under uncertain dynamics and can outperform alternative methods. Another set of contributions is made in multi-agent decision-making which we cast in the general framework of optimisation with interaction constraints. The constraints necessitate coordination, for which we develop several methods. As a particularly challenging application domain, our exposition focusses on collision avoidance. Here we consider coordination both in discrete-time and continuous-time dynamical systems. In the continuous-time case, inference is required to ensure that decisions are made that avoid collisions with adjustably high certainty even when computation is inevitably finite. In both discrete-time and finite-time settings, we introduce conservative decision-making. That is, even with finite computation, a coordination outcome is guaranteed to satisfy collision-avoidance constraints with adjustably high confidence relative to the current uncertain model. Our methods are illustrated in simulations in the context of collision avoidance in graphs, multi-commodity flow problems, distributed stochastic model-predictive control, as well as in collision-prediction and avoidance in stochastic differential systems. Finally, we provide an example of how to combine some of our different methods into a multi-agent predictive controller that coordinates learning agents with uncertain beliefs over their dynamics. Utilising the guarantees established for our learning algorithms, the resulting mechanism can provide collision avoidance guarantees relative to the a posteriori epistemic beliefs over the agents' dynamics.

The purpose of the study was to identify factors affecting student retention at King Saud University in Saudi Arabia. It has been estimated that 35% of university students leave higher education before completing their studies (Al-Saud, 2006). This study was guided by Tinto’s (1975) Student Integration Theory. Berger and Braxton (1998, p. 104) have stated that Tinto’s integration model ‘has been the focus of much empirical research and has near-paradigmatic status in the study of the college student departure.’ This theory is longitudinal and dynamic and views student retention decisions largely as the results of interactions between the student and the academic and social systems of the institution (Tinto, 1975, 1993). This study used a mixed methods approach. Using the terminology of Creswell (2003), the appropriate description of the overall design of this study is a mixed methods concurrent triangulation strategy. This means that ‘qualitative and quantitative data are collected and analyzed at the same time. Priority is usually equal and given to both forms of data. Data analysis is usually separate, and integration usually occurs at the data interpretation stage’ (Hanson et al., 2005, p. 229). This strategy was selected because it allows the findings to be confirmed, cross-validated, and corroborated within a single study (Creswell, 2003). This strategy consisted of two phases. The first phase was the quantitative approach. Quantitative data were collected from 414 freshman students using two questionnaires administered on two occasions and from the university admission office. The quantitative data were analysed using a structural equation modelling (SEM) technique using the AMOS software package. The results of the SEM indicated that Tinto’s model were not useful in predicting the Saudi freshman student retention process. The variables in the model explained only 30 percent of the variance in student retention. The results of the SEM indicated that four of the nine hypotheses proposed in Tinto’s model were supported by statistically significant results. Moreover, only three variables had direct effects on retention. The largest direct effect on retention was accounted for by initial goal and institutional commitment (0.49), followed by later goal and institutional commitment and pre-college schooling as measured by high school scores (0.10). The second phase of this study utilised a qualitative approach. Qualitative data were obtained from three sources: non-persister students, persister students, and staff members. Seventeen non-persister students were interviewed over the phone; 15 persister students were interviewed using a focus group technique; while staff members were asked to complete a survey. Of the 200 surveys distributed, 37 were returned including responses from 16 lecturers, 12 administrators, 5 librarians and 4 academic advisors. A comparison was made between those students who persisted and those who dropped out using constructs from Tinto’s theory. In relation to students’ levels of goal and institutional commitment, it was found that persister students appeared to be more motivated and to have higher levels of goal commitment than non-persister students. Similarly, persister students appeared to have higher levels of institutional commitment than non-persister students, in part it is suggested, due to the fact that the majority of persister students had been able to select their desired majors whereas the majority of non-persister students had not. In relation to the students’ levels of academic integration, there was no significant difference between both groups of students. Persister and non-persister students both exhibited low levels of academic integration into the university system. In addition, there was no significant difference between both groups of students in terms of social integration. Both groups of students indicated low levels of social integration into the university system. In addition, the participants (persister students, non-persister students, and staff members) were all asked to indicate what they perceived to be the major factors affecting student retention at King Saud University. The findings from the qualitative data not only help to explain and confirm the quantitative findings but also identify why Saudi freshman students leave the university before completing their studies. The most important factors were: difficulties of selecting majors, difficulties of transferring between subjects, lack of academic advice and irregularity of monthly reward.

Through case study research, two pioneering schools in Canada that developed whole-school approaches to education for sustainability were investigated to illuminate how conceptual root metaphors resonate with ecological philosophy and educational practices. The study considers philosophy, policy formation, organization/ management structures, buildings/ grounds and resources, curriculum development, and teaching and learning practices at each of these schools. The findings are highlighted and further informed by what the administrators, teachers, community volunteers, parents, and students perceive to be the successes, obstacles and needs they faced in trying to establish their pioneering approaches. These insights provided methodological triangulation as they reinforced the literature review and analysis of findings. The case study includes an Independent school founded and designed specifically around bioregionalism so as to promote sustainability, and a government-run elementary school that decided to teach and model sustainability. The analysis reveals differences in the underlying conceptual metaphors and the significant extent to which these metaphors resonated with practice. This research suggests that root conceptual metaphors are significant and can be associated with various intentions and enactments of the whole curriculum. Ecological and mechanistic metaphorical perspectives have been found to be associated with policy formation, organization and management structures, decision-making and communication; curriculum development; community involvement; changes to the buildings and grounds; and teaching/ learning practices. Although this research suggests that where ecological metaphors were in play school practices were more strongly associated with an ecological model in education for sustainability, it has also shown that this may not be sufficient. Being aware of the underlying conceptual root metaphors in all aspects of the educational approach is also a critical step. The context within which a school operates may preclude or act as a strong obstacle to change. Simply grafting a sustainability program imbued with ecological metaphors onto the accepted educational system, one founded on contradictory ‘mechanistic’ metaphors, may not be as effective as intended as metaphors seem to seep into the school’s culture and systems. This, however, implies that there needs to be freedom and room to challenge significant systemic obstacles. There would need to be noteworthy changes in the socio-political structure that is in play. Accordingly, for schools to lead the change towards an ecological worldview or paradigm shift, schools would need to be free to adopt an alternative vision of education, ethos and particular organizational structures.