Search

Many real-world applications require artificial agents to compete and coordinate with other agents in complex environments. As a stepping stone to this goal, the domain of StarCraft has emerged as an important challenge for artificial intelligence research, owing to its iconic and enduring status among the most difficult professional esports and its relevance to the real world in terms of its raw complexity and multi-agent challenges. Over the course of a decade and numerous competitions1-3, the strongest agents have simplified important aspects of the game, utilized superhuman capabilities, or employed hand-crafted sub-systems4. Despite these advantages, no previous agent has come close to matching the overall skill of top StarCraft players. We chose to address the challenge of StarCraft using general-purpose learning methods that are in principle applicable to other complex domains: a multi-agent reinforcement learning algorithm that uses data from both human and agent games within a diverse league of continually adapting strategies and counter-strategies, each represented by deep neural networks5,6. We evaluated our agent, AlphaStar, in the full game of StarCraft II, through a series of online games against human players. AlphaStar was rated at Grandmaster level for all three StarCraft races and above 99.8% of officially ranked human players.

The CRISPR/Cas system could provide an efficient and reliable means of editing the human genome and has the potential to revolutionize modern medicine; however, rapid developments are raising complex ethical issues. There has been significant scientific debate regarding the acceptability of some applications of CRISPR/Cas, with leaders in the field highlighting the need for the lay public's views to shape expert discussion. As such, we sought to determine the factors that influence public opinion on gene editing. We created a 17-item online survey translated into 11 languages and advertised worldwide. Topic modeling was used to analyze textual responses to determine what factors influenced respondents' opinions toward human somatic or embryonic gene editing, and how this varied among respondents with differing attitudes and demographic backgrounds. A total of 3,988 free-text responses were analyzed. Respondents had a mean age of 32 (range, 11-90) years, and 37% were female. The most prevalent topics cited were Future Generations, Research, Human Editing, Children, and Health. Respondents who disagreed with gene editing for health-related purposes were more likely to cite the topic Better Understanding than those who agreed to both somatic and embryonic gene editing. Respondents from Western backgrounds more frequently discussed Future Generations, compared with participants from Eastern countries. Religious respondents did not cite the topic Religious Beliefs more frequently than did nonreligious respondents, whereas Christian respondents were more likely to cite the topic Future Generations. Our results suggest that public resistance to human somatic or embryonic gene editing does not stem from an inherent mistrust of genome modification, but rather a desire for greater understanding. Furthermore, we demonstrate that factors influencing public opinion vary greatly amongst demographic groups. It is crucial that the determinants of public attitudes toward CRISPR/Cas be well understood so that the technology does not suffer the negative public sentiment seen with previous genetic biotechnologies.

The dominant view in neuroscience is that changes in synaptic weights underlie learning. It is unclear, however, how the brain is able to determine which synapses should change, and by how much. This uncertainty stands in sharp contrast to deep learning, where changes in weights are explicitly engineered to optimize performance. However, the main tool for that, backpropagation, has two problems. One is neuro-science related: it is not biologically plausible. The other is inherent: networks trained with this rule tend to forget old tasks when learning new ones. Here we introduce the Dendritic Gated Network (DGN), a variant of the Gated Linear Network, which offers a biologically plausible alternative to backpropagation. DGNs combine dendritic ‘gating’ (whereby interneurons target dendrites to shape neuronal responses) with local learning rules to yield provably efficient performance. They are significantly more data efficient than conventional artificial networks, and are highly resistant to forgetting. Consequently, they perform well on a variety of tasks, in some cases better than backpropagation. Importantly, DGNs have structural and functional similarities to the cerebellum, a link that we strengthen by usingin vivotwo-photon calcium imaging to show that single interneurons suppress activity in individual dendritic branches of Purkinje cells, a key feature of the model. Thus, DGNs leverage targeted dendritic inhibition and local learning – two features ubiquitous in the brain – to achieve fast and efficient learning.

The current design trend in large scale machine learning is to use distributed clusters of CPUs and GPUs with MapReduce-style programming. Some have been led to believe that this type of horizontal scaling can reduce or even eliminate the need for traditional algorithm development, careful parallelization, and performance engineering. This paper is a case study showing the contrary: that the benefits of algorithms, parallelization, and performance engineering, can sometimes be so vast that it is possible to solve "cluster-scale" problems on a single commodity multicore machine. Connectomics is an emerging area of neurobiology that uses cutting edge machine learning and image processing to extract brain connectivity graphs from electron microscopy images. It has long been assumed that the processing of connectomics data will require mass storage, farms of CPU/GPUs, and will take months (if not years) of processing time. We present a high-throughput connectomics-on-demand system that runs on a multicore machine with less than 100 cores and extracts connectomes at the terabyte per hour pace of modern electron microscopes., National Science Foundation (U.S.) (grant IIS-1447786), National Science Foundation (U.S.) (grant CCF1563880), United States. Intelligence Advanced Research Projects Activity (grant 138076-5093555)

Population Based Training (PBT) is a recent approach that jointly optimizes neural network weights and hyperparameters which periodically copies weights of the best performers and mutates hyperparameters during training. Previous PBT implementations have been synchronized glass-box systems. We propose a general, black-box PBT framework that distributes many asynchronous "trials" (a small number of training steps with warm-starting) across a cluster, coordinated by the PBT controller. The black-box design does not make assumptions on model architectures, loss functions or training procedures. Our system supports dynamic hyperparameter schedules to optimize both differentiable and non-differentiable metrics. We apply our system to train a state-of-the-art WaveNet generative model for human voice synthesis. We show that our PBT system achieves better accuracy, less sensitivity and faster convergence compared to existing methods, given the same computational resource., 9 pages

We present a framework for data-driven robotics that makes use of a large dataset of recorded robot experience and scales to several tasks using learned reward functions. We show how to apply this framework to accomplish three different object manipulation tasks on a real robot platform. Given demonstrations of a task together with task-agnostic recorded experience, we use a special form of human annotation as supervision to learn a reward function, which enables us to deal with real-world tasks where the reward signal cannot be acquired directly. Learned rewards are used in combination with a large dataset of experience from different tasks to learn a robot policy offline using batch RL. We show that using our approach it is possible to train agents to perform a variety of challenging manipulation tasks including stacking rigid objects and handling cloth., Comment: Project website: https://sites.google.com/view/data-driven-robotics/

Ongoing breakthroughs with CRISPR/Cas-based editing could potentially revolutionize modern medicine, but there are many questions to resolve about the ethical implications for its therapeutic application. We conducted a worldwide online survey of over 12,000 people recruited via social media to gauge attitudes toward this technology and discuss our findings here.

The London District Surveyors Association guidance notes for the design of straight shafted bored piles in London Clay recommend an adhesion factor of 0·5 for shaft friction pile design. This value, based on the back-analysis of many maintained load pile tests to failure in London Clay, was selected to cover all design scenarios irrespective of site location, pile construction method, pile geometry and whether or not pile load testing is carried out. This ‘one size fits all’ approach to pile design is potentially over conservative. For sites where maintained load tests (MLTs) to failure on representative preliminary test piles are carried out, the results can be back-analysed to determine the appropriate adhesion factor for the site, which may then be used to improve the efficiency of the pile design. This paper discusses the pile design philosophy, pile load testing and construction sequence for a large development in Whitechapel, London, where MLT results were used to justify an increased adhesion factor of 0·6. This increased adhesion factor ensured that piles were ‘dry’ bored and founded above the water-bearing strata of the Lambeth Group, leading to lower project costs and risks.

Physically realistic simulated environments are powerful platforms for enabling measurable, replicable, and statistically robust investigation of complex robotic systems. Such environments are epitomized by the RoboCup (RC) simulation leagues, which have been successfully utilized to conduct massively parallel experiments on a variety of topics, including optimization of bipedal locomotion, self-localization from noisy perception data, and planning complex multiagent strategies without direct agent-to-agent communication. Many of these systems are later transferred to physical robots, making the simulation leagues invaluable beyond the scope of simulated soccer matches.

Modeling biology as classical problems in computer science allows researchers to leverage the wealth of theoretical advancements in this field. Despite countless studies presenting heuristics that report improvement on specific benchmarking data, there has been comparatively little focus on exploring the theoretical bounds on the performance of practical (polynomial-time) algorithms. Conversely, theoretical studies tend to overstate the generalizability of their conclusions to physical biological processes. In this article we provide a fresh perspective on the concepts of NP-hardness and inapproximability in the computational biology domain, using popular sequence assembly and alignment (mapping) algorithms as illustrative examples. These algorithms exemplify how computer science theory can both (a) lead to substantial improvement in practical performance and (b) highlight areas ripe for future innovation. Importantly, we discuss caveats that seemingly allow the performance of heuristics to exceed their provable bounds.

Background Characterising programs of gene regulation by studying individual protein-DNA and protein-protein interactions would require a large volume of high-resolution proteomics data, and such data are not yet available. Instead, many gene regulatory network (GRN) techniques have been developed, which leverage the wealth of transcriptomic data generated by recent consortia to study indirect, gene-level relationships between transcriptional regulators. Despite the popularity of such methods, previous methods of GRN inference exhibit limitations that we highlight and address through the lens of information theory. Results We introduce new model-free and non-linear information theoretic measures for the inference of GRNs and other biological networks from continuous-valued data. Although previous tools have implemented mutual information as a means of inferring pairwise associations, they either introduce statistical bias through discretisation or are limited to modelling undirected relationships. Our approach overcomes both of these limitations, as demonstrated by a substantial improvement in empirical performance for a set of 160 GRNs of varying size and topology. Conclusions The information theoretic measures described in this study yield substantial improvements over previous approaches (e.g. ARACNE) and have been implemented in the latest release of NAIL (Network Analysis and Inference Library). However, despite the theoretical and empirical advantages of these new measures, they do not circumvent the fundamental limitation of indeterminacy exhibited across this class of biological networks. These methods have presently found value in computational neurobiology, and will likely gain traction for GRN analysis as the volume and quality of temporal transcriptomics data continues to improve.

The selection of an appropriate competition format is critical for both the success and credibility of any competition, both real and simulated. In this paper, the automated parallelism offered by the RoboCupSoccer 2D simulation league is leveraged to conduct a 28,000 game round-robin between the top 8 teams from RoboCup 2012 and 2013. A proposed new competition format is found to reduce variation from the resultant statistically significant team performance rankings by 75 % and 67 %, when compared to the actual competition results from RoboCup 2012 and 2013 respectively. These results are statistically validated by generating 10,000 random tournaments for each of the three considered formats and comparing the respective distributions of ranking discrepancy.

‘Reproducible research’ has received increasing attention over the past few years as bioinformatics and computational biology methodologies become more complex. Although reproducible research is progressing in several valuable ways, we suggest that recent increases in internet bandwidth and disk space, along with the availability of open-source and free-software licences for tools, enable another simple step to make research reproducible. In this article, we urge the creation of minimal virtual reference environments implementing all the tools necessary to reproduce a result, as a standard part of publication. We address potential problems with this approach, and show an example environment from our own work.

Predictive modelling of gene expression provides a powerful framework for exploring the regulatory logic underpinning transcriptional regulation. Recent studies have demonstrated the utility of such models in identifying dysregulation of gene and miRNA expression associated with abnormal patterns of transcription factor (TF) binding or nucleosomal histone modifications (HMs). Despite the growing popularity of such approaches, a comparative review of the various modelling algorithms and feature extraction methods is lacking. We define and compare three methods of quantifying pairwise gene-TF/HM interactions and discuss their suitability for integrating the heterogeneous chromatin immunoprecipitation (ChIP)-seq binding patterns exhibited by TFs and HMs. We then construct log-linear and ϵ-support vector regression models from various mouse embryonic stem cell (mESC) and human lymphoblastoid (GM12878) data sets, considering both ChIP-seq- and position weight matrix- (PWM)-derived in silico TF-binding. The two algorithms are evaluated both in terms of their modelling prediction accuracy and ability to identify the established regulatory roles of individual TFs and HMs. Our results demonstrate that TF-binding and HMs are highly predictive of gene expression as measured by mRNA transcript abundance, irrespective of algorithm or cell type selection and considering both ChIP-seq and PWM-derived TF-binding. As we encourage other researchers to explore and develop these results, our framework is implemented using open-source software and made available as a preconfigured bootable virtual environment.

Humans have the subconscious ability to create simple abstractions from observations of their physical environment. The ability to consider the colour of an object in terms of “red” or “blue”, rather than spatial distributions of reflected light wavelengths, is vital in processing and communicating information about important features within our local environment. The real-time identification of such features in image processing necessitates the software implementation of such a process; segmenting an image into regions of salient colour, and in doing so reducing the information stored and processed from 3-dimensional pixel values to a simple colour class label. This paper details a method by which colour segmentation may be performed offline and stored in a static look-up table, allowing for constant time dimensionality reduction in an arbitrary environment of coloured features. The machine learning framework requires no human supervision, and its performance is evaluated in terms of feature classification performance within a RoboCup robot soccer environment. The developed system is demonstrated to yield an 8% improvement over slower traditional methods of manual colour mapping.

Self-localisation, or the process of an autonomous agent determining its own position and orientation within some local environment, is a critical task in modern robotics. Although this task may be formally defined as a simple transformation between local and global coordinate systems, the process of accurately and efficiently determining this transformation is a complex task. This is particularly the case in an environment where localisation must be inferred entirely from noisy visual data, such as the RoboCup robot soccer competitions. Although many effective probabilistic filters exist for solving this task in its general form, pseudo-uniform belief distributions (such as those arising from course-grain observations) exhibit properties allowing for further performance improvement. This paper explores the RoboCup 2D Simulation League as one such scenario, approximating the artificially constrained noise models as uniform to derive an improved particle filter for self-localisation. The developed system is demonstrated to yield from 38.2 to 201.3% reduction in localisation error, which is further shown as corresponding with a 6.4% improvement in goal difference across approximately 750 games.

The ability of an autonomous agent to self-localise within its environment is critically dependent on its ability to make accurate observations of static, salient features. This notion has driven considerable research into the development and improvement of feature extraction and object recognition algorithms, both within RoboCup and the robotics community at large. Instead, this paper focuses on the rarely-considered issue imposed by the limited field of view of humanoid robots; namely, determining an optimal policy for actuating a robot’s head, to ensure it observes regions of the environment that will maximise the positional information provided. The complexity of this task is magnified by a number of common computational issues; specifically high dimensional state spaces and noisy environmental observations. This paper details the application of motivated reinforcement learning to partially overcome these issues, leading to an 11% improvement (relative to the null case of uniformly distributed actuation policies) in self-localisation and ball-localisation for an agent trained online for less than one hour. The method is demonstrated as a viable method for improving self-localisation in robotics, without the need for further optimisation of object recognition or tuning of probabilistic filters.

As modern autonomous robots have improved in their ability to demonstrate human-like motor skills and reasoning, the size and complexity of software systems have increased proportionally, with developers actively working to leverage the full processing performance of next-generation computational hardware. This software complexity corresponds with increased difficulty in debugging low-level coding issues, with the traditional methodology of inferring such issues from emergent high-level behaviour rapidly approaching intractability. This paper details the development and functionality of NUbugger: a visual, real-time and open source robot debugging utility that provides the user with comprehensive information regarding low-level functionality. This represents a paradigm shift from corrective to preventative debugging, and concrete examples of the application of NUbugger to the identification of fundamental implementation errors are described. The system implementation facilitates simple and rapid extension or modification, making it a useful utility for debugging any similar complex robotic framework.

Background Within the structural and grammatical bounds of a common language, all authors develop their own distinctive writing styles. Whether the relative occurrence of common words can be measured to produce accurate models of authorship is of particular interest. This work introduces a new score that helps to highlight such variations in word occurrence, and is applied to produce models of authorship of a large group of plays from the Shakespearean era. Methodology A text corpus containing 55,055 unique words was generated from 168 plays from the Shakespearean era (16th and 17th centuries) of undisputed authorship. A new score, CM1, is introduced to measure variation patterns based on the frequency of occurrence of each word for the authors John Fletcher, Ben Jonson, Thomas Middleton and William Shakespeare, compared to the rest of the authors in the study (which provides a reference of relative word usage at that time). A total of 50 WEKA methods were applied for Fletcher, Jonson and Middleton, to identify those which were able to produce models yielding over 90% classification accuracy. This ensemble of WEKA methods was then applied to model Shakespearean authorship across all 168 plays, yielding a Matthews' correlation coefficient (MCC) performance of over 90%. Furthermore, the best model yielded an MCC of 99%. Conclusions Our results suggest that different authors, while adhering to the structural and grammatical bounds of a common language, develop measurably distinct styles by the tendency to over-utilise or avoid particular common words and phrasings. Considering language and the potential of words as an abstract chaotic system with a high entropy, similarities can be drawn to the Maxwell's Demon thought experiment; authors subconsciously favour or filter certain words, modifying the probability profile in ways that could reflect their individuality and style.

The ability to accurately track a ball is a critical issue in humanoid robot soccer, made difficult by processor limitations and resultant inability to process all available data from a high-definition image. This paper proposes a computationally efficient method of determining position and size of balls in a RoboCup environment, and compares the performance to two common methods: one utilising Levenberg-Marquardt least squares circle fitting, and the other utilising a circular Hough transform. The proposed method is able to determine the position of a non-occluded tennis ball with less than 10% error at a distance of 5 meters, and a half-occluded ball with less than 20% error, overall outperforming both compared methods whilst executing 300 times faster than the circular Hough transform method. The proposed method is described fully in the context of a colour based vision system, with an explanation of how it may be implemented independent of system paradigm. An extension to allow tracking of multiple balls utilising unsupervised learning and internal cluster validation is described.

Summary : The wide variety of published approaches for the problem of regulatory network inference makes using multiple inference algorithms complex and time-consuming. Network Analysis and Inference Library (NAIL) is a set of software tools to simplify the range of computational activities involved in regulatory network inference. It uses a modular approach to connect different network inference algorithms to the same visualization and network-based analyses. NAIL is technology-independent and includes an interface layer to allow easy integration of components into other applications. Availability and implementation : NAIL is implemented in MATLAB, runs on Windows, Linux and OSX, and is available from SourceForge at https://sourceforge.net/projects/nailsystemsbiology/ for all researchers to use. Contact : daniel.hurley@unimelb.edu.au Supplementary information : Supplementary data are available at Bioinformatics online.

Background Predictive modelling of gene expression is a powerful framework for the in silico exploration of transcriptional regulatory interactions through the integration of high-throughput -omics data. A major limitation of previous approaches is their inability to handle conditional interactions that emerge when genes are subject to different regulatory mechanisms. Although chromatin immunoprecipitation-based histone modification data are often used as proxies for chromatin accessibility, the association between these variables and expression often depends upon the presence of other epigenetic markers (e.g. DNA methylation or histone variants). These conditional interactions are poorly handled by previous predictive models and reduce the reliability of downstream biological inference. Results We have previously demonstrated that integrating both transcription factor and histone modification data within a single predictive model is rendered ineffective by their statistical redundancy. In this study, we evaluate four proposed methods for quantifying gene-level DNA methylation levels and demonstrate that inclusion of these data in predictive modelling frameworks is also subject to this critical limitation in data integration. Based on the hypothesis that statistical redundancy in epigenetic data is caused by conditional regulatory interactions within a dynamic chromatin context, we construct a new gene expression model which is the first to improve prediction accuracy by unsupervised identification of latent regulatory classes. We show that DNA methylation and H2A.Z histone variant data can be interpreted in this way to identify and explore the signatures of silenced and bivalent promoters, substantially improving genome-wide predictions of mRNA transcript abundance and downstream biological inference across multiple cell lines. Conclusions Previous models of gene expression have been applied successfully to several important problems in molecular biology, including the discovery of transcription factor roles, identification of regulatory elements responsible for differential expression patterns and comparative analysis of the transcriptome across distant species. Our analysis supports our hypothesis that statistical redundancy in epigenetic data is partially due to conditional relationships between these regulators and gene expression levels. This analysis provides insight into the heterogeneous roles of H3K4me3 and H3K27me3 in the presence of the H2A.Z histone variant (implicated in cancer progression) and how these signatures change during lineage commitment and carcinogenesis. Electronic supplementary material The online version of this article (doi:10.1186/s13072-015-0013-9) contains supplementary material, which is available to authorized users.

Background Predictive gene expression modelling is an important tool in computational biology due to the volume of high-throughput sequencing data generated by recent consortia. However, the scope of previous studies has been restricted to a small set of cell-lines or experimental conditions due an inability to leverage distributed processing architectures for large, sharded data-sets. Results We present a distributed implementation of gene expression modelling using the MapReduce paradigm and prove that performance improves as a linear function of available processor cores. We then leverage the computational efficiency of this framework to explore the variability of epigenetic function across fifty histone modification data-sets from variety of cancerous and non-cancerous cell-lines. Conclusions We demonstrate that the genome-wide relationships between histone modifications and mRNA transcription are lineage, tissue and karyotype-invariant, and that models trained on matched -omics data from non-cancerous cell-lines are able to predict cancerous expression with equivalent genome-wide fidelity. Electronic supplementary material The online version of this article (doi:10.1186/s12859-016-1313-1) contains supplementary material, which is available to authorized users.

Background Transcription factors (TFs) and histone modifications (HMs) play critical roles in gene expression by regulating mRNA transcription. Modelling frameworks have been developed to integrate high-throughput omics data, with the aim of elucidating the regulatory logic that results from the interactions of DNA, TFs and HMs. These models have yielded an unexpected and poorly understood result: that TFs and HMs are statistically redundant in explaining mRNA transcript abundance at a genome-wide level. Results We constructed predictive models of gene expression by integrating RNA-sequencing, TF and HM chromatin immunoprecipitation sequencing and DNase I hypersensitivity data for two mammalian cell types. All models identified genome-wide statistical redundancy both within and between TFs and HMs, as previously reported. To investigate potential explanations, groups of genes were constructed for ontology-classified biological processes. Predictive models were constructed for each process to explore the distribution of statistical redundancy. We found significant variation in the predictive capacity of TFs and HMs across these processes and demonstrated the predictive power of HMs to be inversely proportional to process enrichment for housekeeping genes. Conclusions It is well established that the roles played by TFs and HMs are not functionally redundant. Instead, we attribute the statistical redundancy reported in this and previous genome-wide modelling studies to the heterogeneous distribution of HMs across chromatin domains. Furthermore, we conclude that statistical redundancy between individual TFs can be readily explained by nucleosome-mediated cooperative binding. This could possibly help the cell confer regulatory robustness by rejecting signalling noise and allowing control via multiple pathways. Electronic supplementary material The online version of this article (doi:10.1186/1756-8935-7-36) contains supplementary material, which is available to authorized users.