Search

To generate actions in the face of physiological delays, the brain must predict the future. Here we explore how prediction may lie at the core of brain function by considering a neuron predicting the future of a scalar time series input. Assuming that the dynamics of the lag vector (a vector composed of several consecutive elements of the time series) are locally linear, Normal Mode Decomposition decomposes the dynamics into independently evolving (eigen-)modes allowing for straightforward prediction. We propose that a neuron learns the top mode and projects its input onto the associated subspace. Under this interpretation, the temporal filter of a neuron corresponds to the left eigenvector of a generalized eigenvalue problem. We mathematically analyze the operation of such an algorithm on noisy observations of synthetic data generated by a linear system. Interestingly, the shape of the temporal filter varies with the signal-to-noise ratio (SNR): a noisy input yields a monophasic filter and a growing SNR leads to multiphasic filters with progressively greater number of phases. Such variation in the temporal filter with input SNR resembles that observed experimentally in biological neurons.

Transformers have revolutionized machine learning across diverse domains, yet understanding their behavior remains crucial, particularly in high-stakes applications. This paper introduces the contextual counting task, a novel toy problem aimed at enhancing our understanding of Transformers in quantitative and scientific contexts. This task requires precise localization and computation within datasets, akin to object detection or region-based scientific analysis. We present theoretical and empirical analysis using both causal and non-causal Transformer architectures, investigating the influence of various positional encodings on performance and interpretability. In particular, we find that causal attention is much better suited for the task, and that no positional embeddings lead to the best accuracy, though rotary embeddings are competitive and easier to train. We also show that out of distribution performance is tightly linked to which tokens it uses as a bias term.

We present AstroCLIP, a strategy to facilitate the construction of astronomical foundation models that bridge the gap between diverse observational modalities. We demonstrate that a cross-modal contrastive learning approach between images and optical spectra of galaxies yields highly informative embeddings of both modalities. In particular, we apply our method on multi-band images and optical spectra from the Dark Energy Spectroscopic Instrument (DESI), and show that: (1) these embeddings are well-aligned between modalities and can be used for accurate cross-modal searches, and (2) these embeddings encode valuable physical information about the galaxies -- in particular redshift and stellar mass -- that can be used to achieve competitive zero- and few- shot predictions without further finetuning. Additionally, in the process of developing our approach, we also construct a novel, transformer-based model and pretraining approach for processing galaxy spectra., Comment: Submitted to the NeurIPS 2023 AI4Science Workshop

An established normative approach for understanding the algorithmic basis of neural computation is to derive online algorithms from principled computational objectives and evaluate their compatibility with anatomical and physiological observations. Similarity matching objectives have served as successful starting points for deriving online algorithms that map onto neural networks (NNs) with point neurons and Hebbian/anti-Hebbian plasticity. These NN models account for many anatomical and physiological observations; however, the objectives have limited computational power and the derived NNs do not explain multi-compartmental neuronal structures and non-Hebbian forms of plasticity that are prevalent throughout the brain. In this article, we unify and generalize recent extensions of the similarity matching approach to address more complex objectives, including a large class of unsupervised and self-supervised learning tasks that can be formulated as symmetric generalized eigenvalue problems or nonnegative matrix factorization problems. Interestingly, the online algorithms derived from these objectives naturally map onto NNs with multi-compartmental neurons and local, non-Hebbian learning rules. Therefore, this unified extension of the similarity matching approach provides a normative framework that facilitates understanding multi-compartmental neuronal structures and non-Hebbian plasticity found throughout the brain., Comment: Added: Figure 1, sections 2, 3

We introduce multiple physics pretraining (MPP), an autoregressive task-agnostic pretraining approach for physical surrogate modeling. MPP involves training large surrogate models to predict the dynamics of multiple heterogeneous physical systems simultaneously by learning features that are broadly useful across diverse physical tasks. In order to learn effectively in this setting, we introduce a shared embedding and normalization strategy that projects the fields of multiple systems into a single shared embedding space. We validate the efficacy of our approach on both pretraining and downstream tasks over a broad fluid mechanics-oriented benchmark. We show that a single MPP-pretrained transformer is able to match or outperform task-specific baselines on all pretraining sub-tasks without the need for finetuning. For downstream tasks, we demonstrate that finetuning MPP-trained models results in more accurate predictions across multiple time-steps on new physics compared to training from scratch or finetuning pretrained video foundation models. We open-source our code and model weights trained at multiple scales for reproducibility and community experimentation.

Large Language Models have not yet been broadly adapted for the analysis of scientific datasets due in part to the unique difficulties of tokenizing numbers. We propose xVal, a numerical encoding scheme that represents any real number using just a single token. xVal represents a given real number by scaling a dedicated embedding vector by the number value. Combined with a modified number-inference approach, this strategy renders the model end-to-end continuous when considered as a map from the numbers of the input string to those of the output string. This leads to an inductive bias that is generally more suitable for applications in scientific domains. We empirically evaluate our proposal on a number of synthetic and real-world datasets. Compared with existing number encoding schemes, we find that xVal is more token-efficient and demonstrates improved generalization., Comment: 10 pages 7 figures. Supplementary: 5 pages 2 figures

In Elmarakeby et al., "Biologically informed deep neural network for prostate cancer discovery", a feedforward neural network with biologically informed, sparse connections (P-NET) was presented to model the state of prostate cancer. We verified the reproducibility of the study conducted by Elmarakeby et al., using both their original codebase, and our own re-implementation using more up-to-date libraries. We quantified the contribution of network sparsification by Reactome biological pathways, and confirmed its importance to P-NET's superior performance. Furthermore, we explored alternative neural architectures and approaches to incorporating biological information into the networks. We experimented with three types of graph neural networks on the same training data, and investigated the clinical prediction agreement between different models. Our analyses demonstrated that deep neural networks with distinct architectures make incorrect predictions for individual patient that are persistent across different initializations of a specific neural architecture. This suggests that different neural architectures are sensitive to different aspects of the data, an important yet under-explored challenge for clinical prediction tasks., Comment: 9 pages, 3 figures. Submitted to Nature Machine Intelligence

Lithium (Li) is a metal with critical therapeutic properties ranging from the treatment of bipolar depression to antibacterial, anticancer, antiviral and pro-regenerative effects. This element can be incorporated into the structure of various biomaterials through the inclusion of Li chloride/carbonate into polymeric matrices or being doped in bioceramics. The biocompatibility and multifunctionality of Li-doped bioceramics present many opportunities for biomedical researchers and clinicians. Li-doped bioceramics (capable of immunomodulation) have been used extensively for bone and tooth regeneration, and they have great potential for cartilage/nerve regeneration, osteochondral repair, and wound healing. The synergistic effect of Li in combination with other anticancer drugs as well as the anticancer properties of Li underline the rationale that bioceramics doped with Li may be impactful in cancer treatments. The role of Li in autophagy may explain its impact in regenerative, antiviral, and anticancer research. The combination of Li-doped bioceramics with polymers can provide new biomaterials with suitable flexibility, especially as bio-ink used in 3D printing for clinical applications of tissue engineering. Such Li-doped biomaterials have significant clinical potential in the foreseeable future.

Air pollution, as one of the most significant environmental challenges, has adversely affected the global economy, human health, and ecosystems. Consequently, comprehensive research is being conducted to provide solutions to air quality management. Recently, it has been demonstrated that environmental parameters, including temperature, relative humidity, wind speed, air pressure, and vegetation, interact with air pollutants, such as particulate matter (PM), NO2, SO2, O3, and CO, contributing to frameworks for forecasting air quality. The objective of the present study is to explore these interactions in three Iranian metropolises of Tehran, Tabriz, and Shiraz from 2015 to 2019 and develop a machine learning-based model to predict daily air pollution. Three distinct assessment criteria were used to assess the proposed XGBoost model, including R squared (R2), Root Mean Squared Error (RMSE), and Mean Absolute Error (MAE). Preliminary results showed that although air pollutants were significantly associated with meteorological factors and vegetation, the formulated model had low accuracy in predicting (R2PM2.5 = 0.36, R2PM10 = 0.27, R2NO2 = 0.46, R2SO2 = 0.41, R2O3 = 0.52, and R2CO = 0.38). Accordingly, future studies should consider more variables, including emission data from manufactories and traffic, as well as sunlight and wind direction. It is also suggested that strategies be applied to minimize the lack of observational data by considering second-and third-order interactions between parameters, increasing the number of simultaneous air pollution and meteorological monitoring stations, as well as hybrid machine learning models based on proximal and satellite data.

Lithium (Li) is a metal with critical therapeutic properties ranging from the treatment of bipolar depression to antibacterial, anticancer, antiviral and pro-regenerative effects. This element can be incorporated into the structure of various biomaterials through the inclusion of Li chloride/carbonate into polymeric matrices or being doped in bioceramics. The biocompatibility and multifunctionality of Li-doped bioceramics present many opportunities for biomedical researchers and clinicians. Li-doped bioceramics (capable of immunomodulation) have been used extensively for bone and tooth regeneration, and they have great potential for cartilage/nerve regeneration, osteochondral repair, and wound healing. The synergistic effect of Li in combination with other anticancer drugs as well as the anticancer properties of Li underline the rationale that bioceramics doped with Li may be impactful in cancer treatments. The role of Li in autophagy may explain its impact in regenerative, antiviral, and anticancer research. The combination of Li-doped bioceramics with polymers can provide new biomaterials with suitable flexibility, especially as bio-ink used in 3D printing for clinical applications of tissue engineering. Such Li-doped biomaterials have significant clinical potential in the foreseeable future.

Finding informative low-dimensional representations that can be computed efficiently in large datasets is an important problem in data analysis. Recently, contrastive Principal Component Analysis (cPCA) was proposed as a more informative generalization of PCA that takes advantage of contrastive learning. However, the performance of cPCA is sensitive to hyper-parameter choice and there is currently no online algorithm for implementing cPCA. Here, we introduce a modified cPCA method, which we denote cPCA*, that is more interpretable and less sensitive to the choice of hyper-parameter. We derive an online algorithm for cPCA* and show that it maps onto a neural network with local learning rules, so it can potentially be implemented in energy efficient neuromorphic hardware. We evaluate the performance of our online algorithm on real datasets and highlight the differences and similarities with the original formulation., Comment: 5 pages, 4 figures

The Federated Satellite Systems/ 3 Cat-5 (FSSCat) mission was the winner of the European Space Agency (ESA) Sentinel Small Satellite (S 3 ) Challenge and overall winner of the 2017 Copernicus Masters competition. It consisted of two six-unit CubeSats. The Earth observation payloads were 1) the Flexible Microwave Payload 2 (FMPL-2) onboard 3 Cat-5/A, an L-band microwave radiometer and GNSS reflectometer (GNSS-R) implemented using a software-defined radio (SDR), and 2) the HyperScout-2 onboard 3 Cat-5/B, a hyperspectral camera, with the first experiment using artificial intelligence to discard cloudy images. FSSCat was launched on 3 September 2020 and injected into a 535-km synchronous orbit. 3 Cat-5/A was operated for three months until the payload was probably damaged by a solar flare and coronal mass ejection. During this time, all scientific requirements were met, including the generation of coarse-resolution and downscaled soil moisture (SM) maps, sea ice extent (SIE) maps, concentration and thickness maps, and even wind speed (WS) and sea surface salinity (SSS) maps, which were not originally foreseen. 3 Cat-5/B was operated a few more months until the number of images acquired met the requirements. This article briefly describes the FSSCat mission and the FMPL-2 payload and summarizes the main scientific results., This work was supported by the 2017 ESA S3 challenge and Copernicus Masters overall winner award (FSSCat project) and ESA project FSSCat Validation Experiment in MOSAIC; Spanish Ministry of Science, Innovation, and Universities Sensing With Pioneering Opportunistic Techniques grant RTI2018-099008-BC21/AEI/10.13039/501100011033; GNSS Environmental and Societal Missions grant PID2021- 126436OB-C21; and Unidad de Excelencia Maria de Maeztu grant MDM-2016-0600., Peer Reviewed, Article signat per 30 autors/es: Adriano Camps, Universitat Politècnica de Catalunya, Barcelona, Spain, Institut d’Estudis Espacials de Catalunya, Barcelona, Spain, United Arab Emirates University, Abu Dhabi, United Arab Emirates; Joan Francesc Munoz-Martin, Universitat Politècnica de Catalunya, Barcelona, Spain, Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA, USA; Joan Adrià Ruiz-de-Azua, Universitat Politècnica de Catalunya, Barcelona, Spain, i2Cat, The Internet Research Center, Barcelona, Spain; Lara Fernandez, Universitat Politècnica de Catalunya, Barcelona, Spain; Adrian Perez-Portero, Universitat Politècnica de Catalunya, Barcelona, Spain; David Llavería, Universitat Politècnica de Catalunya, Barcelona, Spain; Christoph Herbert, Universitat Politècnica de Catalunya, Barcelona, Spain; Miriam Pablos, Institut de Ciències del Mar, Barcelona, Spain, Barcelona Expert Center on Remote Sensing, Barcelona, Spain; Alessandro Golkar, Universitat Politècnica de Catalunya, Barcelona, Spain, Department of Aerospace and Geodesy, Technical University of Munich, Munich, Germany; Antonio Gutiérrez, Deimos Engenharia, Lisbon, Portugal; Carlos António, Deimos Engenharia, Lisbon, Portugal; Jorge Bandeiras, Deimos Engenharia, Lisbon, Portugal; Jorge Andrade, Deimos Engenharia, Lisbon, Portugal; David Cordeiro, Deimos Engenharia, Lisbon, Portugal; Simone Briatore, Golbriak Space, Tallin, Estonia; Nicola Garzaniti, Golbriak Space, Tallin, Estonia, School of Aerospace, Transport, and Manufacturing, Cranfield University, Cranfield, Bedfordshire, U.K.; Fabio Nichele, Tyvak International, Torino, Italy; Raffaele Mozzillo, Tyvak International, Torino, Italy; Alessio Piumatti, Tyvak International, Torino, Italy; Margherita Cardi, Tyvak International, Torino, Italy; Marco Esposito, Cosine, Sassenheim, The Netherlands; Chris van Dijk, Cosine, Sassenheim, The Netherlands; Nathan Vercruyssen, Cosine, Sassenheim, The Netherlands; Joao Barbosa, Cosine, Sassenheim, T, Postprint (author's final draft)

The Federated Satellite Systems/ 3 Cat-5 (FSSCat) mission was the winner of the European Space Agency (ESA) Sentinel Small Satellite (S 3 ) Challenge and overall winner of the 2017 Copernicus Masters competition. It consisted of two six-unit CubeSats. The Earth observation payloads were 1) the Flexible Microwave Payload 2 (FMPL-2) onboard 3 Cat-5/A, an L-band microwave radiometer and GNSS reflectometer (GNSS-R) implemented using a software-defined radio (SDR), and 2) the HyperScout-2 onboard 3 Cat-5/B, a hyperspectral camera, with the first experiment using artificial intelligence to discard cloudy images. FSSCat was launched on 3 September 2020 and injected into a 535-km synchronous orbit. 3 Cat-5/A was operated for three months until the payload was probably damaged by a solar flare and coronal mass ejection. During this time, all scientific requirements were met, including the generation of coarse-resolution and downscaled soil moisture (SM) maps, sea ice extent (SIE) maps, concentration and thickness maps, and even wind speed (WS) and sea surface salinity (SSS) maps, which were not originally foreseen. 3 Cat-5/B was operated a few more months until the number of images acquired met the requirements. This article briefly describes the FSSCat mission and the FMPL-2 payload and summarizes the main scientific results

Zusammenfassung 1.Ziel der Arbeit Für Hornhautendotheldystrophien gibt es verschiedene Behandlungsmöglichkeiten. Die DMEK Operation ist seit einigen Jahren zum Standard der Behandlung von Endotheldystrophien geworden. Diese Arbeit beschäftigt sich mit drei unterschiedlichen Formen der DMEK Operation, der Phaken DMEK, der Pseudophaken DMEK der Triple DMEK. Bei der Phaken DMEK wird die Descemet-Membran und das Hornhautendothel des Patienten entfernt und an diese Stelle ein entsprechendes Transplantat eines Hornhautspenders eingesetzt. Bei dieser Operation bleibt die natürliche Linse erhalten. Diese Operation wird vor allem bei Patienten eingesetzt, bei denen die natürliche Linse noch keine starke Trübung aufweist. Bei der Triple DMEK, die häufig bei Patienten angewendet wird, die zusätzlich zur Endotheldystrophie schon eine starke Linsentrübung aufweisen, wird diese Operation mit einer Phakoemulsifikation und einer Linsenersatzoperation kombiniert. Zusätzlich besteht die Operationsmethode der Pseudophaken DMEK, bei der Patienten eine DMEK Operation erhalten, bei denen bereits eine Kataraktoperation durchgeführt wurde. Hier stellt sich die Frage einer zeitgleichen Kataraktoperation also nicht. Ziel dieser Arbeit ist das Ergebnis von phaken und pseudophaken Augen zu evaluieren, die mit einer Descemet-Membrane endothelial keratoplasty (DMEK) oder mit einer DMEK kombiniert mit einer Kataraktoperation (Triple-DMEK) therapiert wurden. 2. Material und Methoden Bei der vorliegenden Studie handelt es sich um eine monozentrische retrospektive Datenanalyse von 557 Augen, die mit einer Triple-DMEK behandelt wurden, 518 Augen, bei denen eine Pseudophake DMEK durchgeführt wurde, sowie 62 Augen, die mit einer Phaken DMEK behandelt wurden, einbezogen. Es wurden Patienten einbezogen, die im Zeitraum 07/2011 bis 07/2017 im Zentrum für Augenheilkunde der Uniklinik Köln mit einer Phaken DMEK, einer Pseudophaken DMEK oder einer Triple-DMEK behandelt wurden. Die Patientendaten wurden mit d

Diabetic Retinopathy (DR) caused by diabetes occurs as a result of changes in the retinal vessels and causes visual impairment. Microaneurysms (MAs) are the early clinical signs of DR, whose timely diagnosis can help detecting DR in the early stages of its development. It has been observed that MAs are more common in the inner retinal layers compared to the outer retinal layers in eyes suffering from DR. Optical Coherence Tomography (OCT) is a noninvasive imaging technique that provides a cross-sectional view of the retina and it has been used in recent years to diagnose many eye diseases. As a result, in this paper has attempted to identify areas with MA from normal areas of the retina using OCT images. This work is done using the dataset collected from FA and OCT images of 20 patients with DR. In this regard, firstly Fluorescein Angiography (FA) and OCT images were registered. Then the MA and normal areas were separated and the features of each of these areas were extracted using the Bag of Features (BOF) approach with Speeded-Up Robust Feature (SURF) descriptor. Finally, the classification process was performed using a multilayer perceptron network. For each of the criteria of accuracy, sensitivity, specificity, and precision, the obtained results were 96.33%, 97.33%, 95.4%, and 95.28%, respectively. Utilizing OCT images to detect MAsautomatically is a new idea and the results obtained as preliminary research in this field are promising .

Purpose: To evaluate the outcome of phakic and pseudophakic eyes treated by Descemet membrane endothelial keratoplasty (DMEK) versus DMEK combined with cataract surgery (triple DMEK). Methods: Retrospective cohort study based on the prospective Cologne DMEK data base. This study is a single-center analysis of 62 eyes undergoing phakic (age mean +/- SD 52 +/- 6.43), 518 eyes undergoing pseudophakic (73.5 +/- 8.57), and 557 eyes undergoing triple DMEK (67.93 +/- 8.57). Outcome measures were changes in best spectacle-corrected visual acuity (BSCVA), central corneal thickness, endothelial cell density (ECD), rebubbling rates within the first year after surgery, and the need for cataract surgery within the first 2 years after phakic DMEK. Results: The preoperative BSCVA (mean +/- SD; logarithm of the minimum angle of resolution) was 0.30 +/- 0.22 in the phakic, 0.63 +/- 0.45 in the pseudophakic, and 0.44 +/- 0.30 in the triple DMEK group (P < 0.001), which changed to 0.09 +/- 0.12, 0.14 +/- 0.1, and 0.1 +/- 0.1 (P < 0.001) 1 year after surgery, respectively. There was no difference in central corneal thickness (P = 0.929) and endothelial cell density (P = 0.606) 1 year postoperatively. Rebubbling rates in DMEK using SF6 20% for anterior chamber tamponade were not significantly different (P = 0.839). After phakic DMEK, 40% of eyes underwent cataract surgery within the second year. However, there was a high loss to follow-up in this group. Conclusions: Phakic and triple DMEK procedures tend to have a better 1-year BSCVA than pseudophakic DMEK, with no differences in all other parameters analyzed. However, patients from the pseudophakic DMEK group were older and already had worse BSCVA before surgery.

Past research has shown that attributions of intentions to other's actions determine how we experience these actions and their consequences. Yet, it is unknown how such attributions affect our learning and memory. Addressing this question, we combined neuroimaging with an interactive threat learning paradigm in which two interaction partners (confederates) made choices that had either threatening (shock) or safe (no shock) consequences for the participants. Importantly, participants were led to believe that one partner intentionally caused the delivery of shock, whereas the other did not (i.e. unintentional partner). Following intentional versus unintentional shocks, participants reported an inflated number of shocks and a greater increase in anger and vengeance. We applied a model-based representational similarity analysis to blood-oxygen-level-dependent (BOLD)-MRI patterns during learning. Surprisingly, we did not find any effects of intentionality. The threat value of actions, however, was represented as a trial-by-trial increase in representational similarity in the insula and the inferior frontal gyrus. Our findings illustrate how neural pattern formation can be used to study a complex interaction.

Background: Cognitive reappraisal is a strategy for emotional regulation, important in the context of anxiety disorders. It is not known whether anxiolytic effects of benzodiazepines affect cognitive reappraisal. Aims: We aimed to investigate the effect of 25 mg oxazepam on cognitive reappraisal. Methods: In a preliminary investigation, 33 healthy male volunteers were randomised to oxazepam or placebo, and then underwent an experiment where they were asked to use cognitive reappraisal to upregulate or downregulate their emotional response to images with negative or neutral emotional valence. We recorded unpleasantness ratings, skin conductance, superciliary corrugator muscle activity, and heart rate. Participants completed rating scales measuring empathy (Interpersonal Reactivity Index, IRI), anxiety (State-Trait Anxiety Inventory, STAI), alexithymia (Toronto Alexithymia Scale-20, TAS-20), and psychopathy (Psychopathy Personality Inventory-Revised, PPI-R). Results: Upregulation to negative-valence images in the cognitive reappraisal task caused increased unpleasantness ratings, corrugator activity, and heart rate compared to downregulation. Upregulation to both negative- and neutral-valence images caused increased skin conductance responses. Oxazepam caused lower unpleasantness ratings to negative-valence stimuli, but did not interact with reappraisal instruction on any outcome. Self-rated trait empathy was associated with stronger responses to negative-valence stimuli, whereas self-rated psychopathic traits were associated with weaker responses to negative-valence stimuli. Conclusions: While 25 mg oxazepam caused lower unpleasantness ratings in response to negative-valence images, we did not observe an effect of 25 mg oxazepam on cognitive reappraisal., This work was supported by the Swedish Society for Medicine and the Osher Center for Integrative Medicine.

FSSCat, the “Federated Satellite Systems/ 3 Cat-5” mission was the winner of the 2017 ESA S^3 (Sentinel Small Satellite) Challenge and overall winner of the Copernicus Masters competition. FSSCat consists of two 6 unit cubesats carrying on board UPC's Flexible Microwave Payload - 2 (FMPL-2), an L-band microwave radiometer and GNSS-Reflectometer implemented in a software defined radio, and Cosine's HyperScout-2 visible and near infrared + thermal infrared hyperspectral imager, enhanced with PhiSat-1, a on board Artificial intelligence experiment for cloud detection. Both spacecrafts include optical and UHF inter-satellite links technology demonstrators, provided by Golbriak Space and UPC, respectively. This paper describes the mission, and the main scientific results of the FMPL-2 obtained during the first three months of the mission, notably the sea ice concentration and thickness, and the downscaled soil moisture products over the Northern hemisphere., This work was supported by 2017 ESA S 3 challenge and Copernicus Masters overall winner award (“FSSCat” project) and ESA project “FSSCat Validation Experiment in MOSAIC”, by the Spanish Ministry of Science, Innovation and Universities, "Sensing with Pioneering Opportunistic Techniques" SPOT, grant RTI2018-099008- BC21/AEI/10.13039/501100011033, and by the Unidad de Excelencia Maria de Maeztu MDM-2016-0600., Peer Reviewed, Article signat per 25 autors/es: A. Camps 1,2; J.F. Munoz‐Martin 1; J.A. Ruiz‐de‐Azua 1,2; L. Fernandez 1; A. Perez-Portero 1; D. Llavería 1; C. Herbert 1; M. Pablos 3; A. Golkar 4,1; A. Gutiérrrez 5; C. António 5; J. Bandeiras 5; J. Andrade 5; D. Cordeiro 5; S. Briatore 4,6; N. Garzaniti 4,6; F. Nichele 7; R. Mozzillo 7; A. Piumatti 7; M. Cardi 7; M. Esposito 8; B. Carnicero Dominguez 9; M. Pastena 9; G. Filippazzo 10; A. Reagan 10 // 1. Universitat Politècnica de Catalunya, Barcelona, Spain; 2. Institut d’Estudis Espacials de Catalunya, Barcelona, Spain; 3. Institut de Ciències del Mar (ICM-CSIC) & Barcelona Expert Center (BEC) on Remote Sensing, Barcelona, Spain; 4. Skolkovo Institute of Science and Technology, Moscow, Russia; 5. Deimos Eng., Lisbon, Portugal; 6. Golbriak Space, Tallin, Estonia; 7. Tyvak International, Torino, Italy; 8. Cosine, Oosteinde, The Netherlands; 9. ESA ESTEC, Noordwijk, The Netherlands; 10. ESA ESRIN, Frascati, Italy, Postprint (author's final draft)

Presently, the Earth Observation community is demanding applications that provide low latency and high downlink capabilities. An increase in downlink contacts becomes essential to meet these new requirements. The Federated Satellite Systems concept addresses this demand by promoting satellite collaborations to share unused downlink opportunities. These collaborations are established opportunistically and temporarily, posing multiple technology challenges to be implemented in-orbit. This work contributes to the definition of the Federation Deployment Control Protocol which formalizes a mechanism to fairly establish and manage these collaborations by employing a negotiation process between the satellites. Moreover, this manuscript presents the results of a validation campaign of this protocol with three stratospheric balloons. In summary, more than 27 federations with 63.0% of throughput were established during the field campaign. Some of these federations were used to download data to the ground, and others were established to balance data storage between balloons. These federations allowed also the extension of the coverage of a ground station with a federation that relayed data through a balloon, and the achievement of a hybrid scenario with one balloon forwarding data from a ground device. The results demonstrate that the proposed protocol is functional and ready to be embedded in a CubeSat mission., This work has been (partially) funded by “CommSensLab” Excellence Research Unit Maria de Maeztu (MINECO grant MDM-2016-0600), the Spanish Ministerio MICINN and EU ERDF project “SPOT: Sensing with pioneering opportunistic techniques” (grant RTI2018-099008-BC21/AEI/10.13039/501100011033), by the grant PID2019-106808RA-I00/AEI/FEDER/UE from the EDRF and the Spanish Government, and by the Secretaria d’Universitats i Recerca del Departament d’Empresa i Coneixement de la Generalitat de Catalunya (2017 SGR 376, and 2017 SGR 219)., Peer Reviewed, Postprint (published version)

A major problem in motor control is understanding how the brain plans and executes proper movements in the face of delayed and noisy stimuli. A prominent framework for addressing such control problems is Optimal Feedback Control (OFC). OFC generates control actions that optimize behaviorally relevant criteria by integrating noisy sensory stimuli and the predictions of an internal model using the Kalman filter or its extensions. However, a satisfactory neural model of Kalman filtering and control is lacking because existing proposals have the following limitations: not considering the delay of sensory feedback, training in alternating phases, and requiring knowledge of the noise covariance matrices, as well as that of systems dynamics. Moreover, the majority of these studies considered Kalman filtering in isolation, and not jointly with control. To address these shortcomings, we introduce a novel online algorithm which combines adaptive Kalman filtering with a model free control approach (i.e., policy gradient algorithm). We implement this algorithm in a biologically plausible neural network with local synaptic plasticity rules. This network performs system identification and Kalman filtering, without the need for multiple phases with distinct update rules or the knowledge of the noise covariances. It can perform state estimation with delayed sensory feedback, with the help of an internal model. It learns the control policy without requiring any knowledge of the dynamics, thus avoiding the need for weight transport. In this way, our implementation of OFC solves the credit assignment problem needed to produce the appropriate sensory-motor control in the presence of stimulus delay., Comment: Manuscript and supplementary material of NeurIPS 2021 paper

A renewed interest in lunar exploration with the focus on establishing a constant human presence on the Moon calls for developing new lunar human landing systems (HLS) which would deliver the crew from the prospective Lunar Gateway station to the surface of the Moon and back. Over the years, different human lunar lander architectures were proposed and multiple architecture studies were performed. However, those studies are relevant to the specific assumptions and lunar architectures proposed at the time of conducting the study. Since the current vision for lunar exploration includes new features, such as having the Lunar Gateway and switching to reusable systems, there is a need for a new HLS architecture study. Such studies are being performed by private companies; however, those are rarely publicly available. The goal of this paper is to address this gap and provide a publicly available architectural analysis within the current views on the future human lunar exploration. We assume the Lunar Gateway in an L2 near rectilinear halo orbit and a landing site at the lunar South Pole; the number of HLS crew of 4; the surface stay time of ~7 days, the payload mass delivered to the surface of 500 kg, and the payload mass returned from the surface of 250 kg. A set of parametric models including an HLS model and an HLS program cost model is developed for the analysis. 39 architectures with varying number of stages (1, 2, and 3 stages) and propellant combinations (LOX/LH2, LOX/CH4, and MMH/NTO) are explored. The Pareto analysis shows that there is a difference between typical performance trends for expendable and reusable architectures. For expendable architectures, the 2-stage option seems to be the most advantageous while, for reusable architectures, the 1- and 3-stage options are either comparable or win over the 2-stage option even for the number of system uses as low as 3. In terms of the propellant combinations, pure LOX/LH2 or combined LOX/LH2/LOX/CH4 architectures do

FSSCat, the “Federated Satellite Systems/ 3 Cat-5” mission was the winner of the 2017 ESA S^3 (Sentinel Small Satellite) Challenge and overall winner of the Copernicus Masters competition. FSSCat consists of two 6 unit cubesats carrying on board UPC's Flexible Microwave Payload - 2 (FMPL-2), an L-band microwave radiometer and GNSS-Reflectometer implemented in a software defined radio, and Cosine's HyperScout-2 visible and near infrared + thermal infrared hyperspectral imager, enhanced with PhiSat-1, a on board Artificial intelligence experiment for cloud detection. Both spacecrafts include optical and UHF inter-satellite links technology demonstrators, provided by Golbriak Space and UPC, respectively. This paper describes the mission, and the main scientific results of the FMPL-2 obtained during the first three months of the mission, notably the sea ice concentration and thickness, and the downscaled soil moisture products over the Northern hemisphere

In this paper, a two-stage stochastic day-ahead (DA) scheduling model is proposed incorporating wind power units and compressed air energy storage (CAES) to clear a co-optimized energy and reserve market. The two-stage stochastic programming method is employed to deal with the wind power generation uncertain nature. A linearized AC optimal power flow (LAC-OPF) approach with consideration of network losses, reactive power, and voltage magnitude constraints is utilized in the proposed two-stage stochastic DA scheduling model. Using an engineering insight, a two-level LAC-OPF (TL-LAC-OPF) approach is proposed to (i) reduce the number of binary variables of the LAC-OPF approach which decreases the computational burden, and (ii) obtain LAC-OPF pre-defined parameters adaptively so that the accuracy of LAC-OPF approach is increased as a result of reducing artificial losses. Furthermore, as the CAES efficiency depends on its thermodynamic and operational conditions, the proposed two-stage stochastic DA scheduling model is developed by considering its thermodynamic characteristics to obtain a more realistic market decision at the first place. The proposed model is applied to IEEE 30-bus and 57-bus test systems using GAMS software, and is compared with three traditional approaches, i.e., AC-OPF, DC-OPF, and LAC-OPF. Simulation results demonstrate effectiveness of the proposed methodology.

A significant amount of work has been done to improve design and fabrication of DNA constructs with photonic and electronic transfer properties. In this thesis, we evaluated the prior work for both electronic and photonic transfer. While DNA constructs with first-order fluorescent resonant energy transfer (FRET) properties have proven useful, incorporation of higher-order FRET and electronic properties into DNA has not yet led to any viable applications. In this thesis, we generated 35 base pairs long double-stranded (ds) DNA structures with different arrangements of five TAMRA donor dyes and a single TexasRed acceptor. In these constructs the distance of the distal donor dyes to the acceptor is greater than 1.7 nm or five base pairs (which is beyond the optimal FRET distance). The average FRET efficiency of these double stranded systems based on the donor intensity change and the acceptor-to-donor ratio of intensity change was 66% and 26%, respectively. Addition of surfactants and metal cations reduced quenching and enhanced the FRET efficiency of these DNA structures. After adding the surfactant and metal cations, the average FRET efficiency of these ds-systems based on the donor intensity change and the acceptor-to-donor ratio of intensity change was 89% and 75%, respectively.We also reviewed the conductivity properties of DNA and how it is influenced by temperature, UV illumination and GC content. Results from literature indicate that temperature significantly changes DNA conductivity. Moreover, the UV exposure experiments indicate a decrease in DNA conductivity due to damage of GC base pairs and the phosphate group. We also investigated the effect of nucleotide content on DNA conductivity and we showed that the higher GC content results in higher conductivity.

Humans, like other social animals, learn about threats and safety in the environment through social cues. Yet, the processes that contribute to the efficacy of social safety learning during threat transmission remain unknown. Here, we developed a novel dyadic model of associative threat and extinction learning. In three separate social groups, we manipulated whether safety information during extinction was acquired via direct exposure to the conditioned stimulus (CS) in the presence of another individual (Direct exposure), via observation of other's safety behavior (Vicarious exposure), or via the combination of both (Shared exposure).These groups were contrasted against a fourth group receiving direct CS exposure alone (Asocial exposure). Based on skin conductance responses, we observed that all social groups outperformed asocial learning in inhibiting the recovery of threat, but only Shared exposure abolished threat recovery. These results suggest that social safety learning is optimized by a combination of direct exposure and vicariously transmitted safety signals. This work might help develop exposure therapies used to treat symptoms of threat and anxiety-related disorders to counteract maladaptive fears in humans.

© <2020>. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0, Nowadays, constellations and distributed networks of satellites are emerging as clear development trends in the space system market to enable augmentation, enhancement, and possibilities of new applications for future Earth Observation (EO) missions. While the adoption of these satellite architectures is gaining momentum for the attaining of ever more stringent application requirements and stakeholder needs, the efforts to analyze their benefits and suitability, and to assess their impact for future programmes remains as an open challenge to the EO community. In this context, this paper presents the mission and system architecture conceived during the Horizon 2020 ONION project, a European Union research activity that proposes a systematic approach to the optimization of EO space infrastructures. In particular, ONION addressed the design of complementary assets that progressively supplement current programs and took part in the exploration of needs and implementation of architectures for the Copernicus Space Component for EO. Among several use cases considered, the ONION project focused on proposing system architectures to provide improved revisit time, data latency and image resolution for a demanding application scenario of interest: Marine Weather Forecast (MWF). A set of promising system architectures has been subject of a comprehensive assessment, based on mission analysis expertise and detailed simulation for evaluating several key parameters such as revisit time and data latency of each measurement of interest, on-board memory evolution and power budget of each satellite of the constellation, ground station contacts and inter-satellite links. The architectures are built with several heterogeneous satellite nodes distributed in different orbital planes. Each platform can embark different instrument sets, which provide the required measurements for each use case. A detailed mission analysis has then been performed to the selected architecture for the MWF use ca, This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 687490., Peer Reviewed, Postprint (author's final draft)

© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., An accurate electricity price forecasting (EPF) plays a vital role in the deregulated energy markets and has a specific effect on optimal management of the power system. Considering all the potent factors in determining the electricity prices—some of which have stochastic nature—makes this a cumbersome task. In this article, first, Grey correlation analysis is applied to select the effective parameters in EPF problem and eliminate redundant factors based on low correlation grades. Then, a deep neural network with stacked denoising auto-encoders has been utilized to denoise data sets from different sources individually. After that, to detect the main features of the input data and putting aside the unnecessary features, dimension reduction process is implemented. Finally, the rough structure artificial neural network (ANN) has been executed to forecast the day-ahead electricity price. The proposed method is implemented on the data of Ontario, Canada, and the forecasted results are compared with different structures of ANN, support vector machine, long shortterm memory—benchmarking methods in this field—and forecasting data reported by independent electricity system operator (IESO) to evaluate the efficiency of the proposed approach. Furthermore, the results of this article indicate that the proposed method is efficient in terms of reducing error criterion and improves the forecasting error about 5–10 percent in comparison with IESO. This is a remarkable achievement in EPF field., Peer Reviewed, Postprint (author's final draft)

Recently, various relations and criteria have been presented to establish a proper relationship between control systems and control the Global Positioning System (GPS)-intelligent buoy system. Given the importance of controlling the position of buoys and the construction of intelligent systems, in this paper, dynamic system modeling is applied to position marine buoys through the improved neural network with a backstepping technique. This study aims at developing a novel controller based on an adaptive fuzzy neural network to optimally track the dynamically positioned vehicle on the water with unavailable velocities and unidentified control parameters. In order to model the network with the proposed technique, uncertainties and the unwanted disturbances are studied in the neural network. The presented study aims at developing a neural controlling which applies the vectorial back-stepping technique to the surface ships, which have been dynamically positioned with undetermined disturbances and ambivalences. Moreover, the objective function is to minimize the output error for the neural network (NN) based on the closed-loop system. The most important feature of the proposed model for the positioning buoys is its independence from comparative knowledge or information on the dynamics and the unwanted disturbances of ships. The numerical and obtained consequences demonstrate that the control system can adjust the routes and the position of the buoys to the desired objective with relatively few position errors., Comment: 32 pages, 10 figures

The backpropagation algorithm is an invaluable tool for training artificial neural networks; however, because of a weight sharing requirement, it does not provide a plausible model of brain function. Here, in the context of a two-layer network, we derive an algorithm for training a neural network which avoids this problem by not requiring explicit error computation and backpropagation. Furthermore, our algorithm maps onto a neural network that bears a remarkable resemblance to the connectivity structure and learning rules of the cortex. We find that our algorithm empirically performs comparably to backprop on a number of datasets., Comment: Abstract presented at the NeurIPS 2020 workshop "Beyond Backpropagation". arXiv admin note: text overlap with arXiv:2010.12660

To guide behavior, the brain extracts relevant features from high-dimensional data streamed by sensory organs. Neuroscience experiments demonstrate that the processing of sensory inputs by cortical neurons is modulated by instructive signals which provide context and task-relevant information. Here, adopting a normative approach, we model these instructive signals as supervisory inputs guiding the projection of the feedforward data. Mathematically, we start with a family of Reduced-Rank Regression (RRR) objective functions which include Reduced Rank (minimum) Mean Square Error (RRMSE) and Canonical Correlation Analysis (CCA), and derive novel offline and online optimization algorithms, which we call Bio-RRR. The online algorithms can be implemented by neural networks whose synaptic learning rules resemble calcium plateau potential dependent plasticity observed in the cortex. We detail how, in our model, the calcium plateau potential can be interpreted as a backpropagating error signal. We demonstrate that, despite relying exclusively on biologically plausible local learning rules, our algorithms perform competitively with existing implementations of RRMSE and CCA., Comment: Body and supplementary materials of NeurIPS 2020 paper. 19 pages, 7 figures

Learning latent features from time series data is an important problem in both machine learning and brain function. One approach, called Slow Feature Analysis (SFA), leverages the slowness of many salient features relative to the rapidly varying input signals. Furthermore, when trained on naturalistic stimuli, SFA reproduces interesting properties of cells in the primary visual cortex and hippocampus, suggesting that the brain uses temporal slowness as a computational principle for learning latent features. However, despite the potential relevance of SFA for modeling brain function, there is currently no SFA algorithm with a biologically plausible neural network implementation, by which we mean an algorithm operates in the online setting and can be mapped onto a neural network with local synaptic updates. In this work, starting from an SFA objective, we derive an SFA algorithm, called Bio-SFA, with a biologically plausible neural network implementation. We validate Bio-SFA on naturalistic stimuli., Comment: 17 pages, 7 figures

Cortical pyramidal neurons receive inputs from multiple distinct neural populations and integrate these inputs in separate dendritic compartments. We explore the possibility that cortical microcircuits implement Canonical Correlation Analysis (CCA), an unsupervised learning method that projects the inputs onto a common subspace so as to maximize the correlations between the projections. To this end, we seek a multi-channel CCA algorithm that can be implemented in a biologically plausible neural network. For biological plausibility, we require that the network operates in the online setting and its synaptic update rules are local. Starting from a novel CCA objective function, we derive an online optimization algorithm whose optimization steps can be implemented in a single-layer neural network with multi-compartmental neurons and local non-Hebbian learning rules. We also derive an extension of our online CCA algorithm with adaptive output rank and output whitening. Interestingly, the extension maps onto a neural network whose neural architecture and synaptic updates resemble neural circuitry and synaptic plasticity observed experimentally in cortical pyramidal neurons., Comment: 46 pages, 14 figures

Nowadays, constellations and distributed networks of satellites are emerging as clear development trends in the space system market to enable augmentation, enhancement, and possibilities of new applications for future Earth Observation (EO) missions. While the adoption of these satellite architectures is gaining momentum for the attaining of ever more stringent application requirements and stakeholder needs, the efforts to analyze their benefits and suitability, and to assess their impact for future programmes remains as an open challenge to the EO community. In this context, this paper presents the mission and system architecture conceived during the Horizon 2020 ONION project, a European Union research activity that proposes a systematic approach to the optimization of EO space infrastructures. In particular, ONION addressed the design of complementary assets that progressively supplement current programs and took part in the exploration of needs and implementation of architectures for the Copernicus Space Component for EO. Among several use cases considered, the ONION project focused on proposing system architectures to provide improved revisit time, data latency and image resolution for a demanding application scenario of interest: Marine Weather Forecast (MWF). A set of promising system architectures has been subject of a comprehensive assessment, based on mission analysis expertise and detailed simulation for evaluating several key parameters such as revisit time and data latency of each measurement of interest, on-board memory evolution and power budget of each satellite of the constellation, ground station contacts and inter-satellite links. The architectures are built with several heterogeneous satellite nodes distributed in different orbital planes. Each platform can embark different instrument sets, which provide the required measurements for each use case. A detailed mission analysis has then been performed to the selected architecture for the MWF use ca

In the last years, the need to access Earth Observation high spatial resolution data with very low latency, ideally in near-real-time, has increased. Distributed Satellite Systems have emerged as an effective and efficient architecture to deal with these tight requirements. One of these systems is the Federated Satellite Systems which explore the benefits of sharing unused and available resources between satellites, such as memory storage or downlink opportunities. This interaction is known as satellite federations. Additional downlink contacts may enable low-latency communications and increase the downlink capacity. Therefore, the deployment of federations would allow satisfying current Earth Observation community demands. This work contributes to establish this promising paradigm by presenting a proof-of-concept of a federated system, called FSS Experiment payload. This payload is boarded into three stratospheric balloons, and it provides them communications means to create federations. The design of the payload, and the flight campaign results are presented in this work. These results demonstrate the feasibility of deploying federations, as well as the benefits of deploying them in future EO missions., This work has been funded by “CommSensLab” Excellence Research Unit Maria de Maeztu (MINECO grant MDM-2016-0600), and the Spanish Ministerio MICINN and EU ERDF project (RTI2018-099008-B-C21) “Sensing with pioneering opportunistic techniques” by the grant FI-DGR 2015 of AGAUR - Generalitat de Catalunya (FEDER). The authors would like to acknowledge Skoltech and all the students that with their passion have made possible this balloon campaign., Peer Reviewed, Postprint (published version)

This thesis developed a methodology in order to compare the performance of different types of coalescing filters (fibrous, knitted, and foam media) which has not been reported in literature. Moreover, the local optimum in pressure drop curve which identified in this work is a novel contribution to the field of interfacial science and gas-liquid separation which can significantly affect the coalescence filters design in future.

Sleep restriction has been proposed to cause impaired emotional processing and emotional regulation by inhibiting top-down control from prefrontal cortex to amygdala. Intentional emotional regulation after sleep restriction has, however, never been studied using brain imaging. We aimed here to investigate the effect of partial sleep restriction on emotional regulation through cognitive reappraisal. Forty-seven young (age 20–30) and 33 older (age 65–75) participants (38/23 with complete data and successful sleep intervention) performed a cognitive reappraisal task during fMRI after a night of normal sleep and after restricted sleep (3 h). Emotional downregulation was associated with significantly increased activity in the dorsolateral prefrontal cortex (pFWE < 0.05) and lateral orbital cortex (pFWE < 0.05) in young, but not in older subjects. Sleep restriction was associated with a decrease in self-reported regulation success to negative stimuli (p< 0.01) and a trend towards perceiving all stimuli as less negative (p = 0.07) in young participants. No effects of sleep restriction on brain activity nor connectivity were found in either age group. In conclusion, our data do not support the idea of a prefrontal-amygdala disconnect after sleep restriction, and neural mechanisms underlying behavioural effects on emotional regulation after insufficient sleep require further investigation.

We introduce two methods for estimating the density matrix for a quantum system: Quantum Maximum Likelihood and Quantum Variational Inference. In these methods, we construct a variational family to model the density matrix of a mixed quantum state. We also introduce quantum flows, the quantum analog of normalizing flows, which can be used to increase the expressivity of this variational family. The eigenstates and eigenvalues of interest are then derived by optimizing an appropriate loss function. The approach is qualitatively different than traditional lattice techniques that rely on the time dependence of correlation functions that summarize the lattice configurations. The resulting estimate of the density matrix can then be used to evaluate the expectation of an arbitrary operator, which opens the door to new possibilities., Comment: 12 pages, 3 figures

The market penetration of Plug-in Electric Vehicles (PEVs) is escalating due to their energy saving and environmental benefits. In order to address PEVs impact on the electric networks, the aggregators need to accurately predict the PEV Travel Behavior (PEV-TB) since the addition of a great number of PEVs to the current distribution network poses serious challenges to the power system. Forecasting PEV-TB is critical because of the high degree of uncertainties in drivers’ behavior. Existing studies mostly simplified the PEV-TB by mapping travel behavior from conventional vehicles. This could cause bias in power estimation considering the differences in PEV-TB because of charging pattern which consequently could bungle economic analysis of aggregators. In this study, to forecast PEV-TB an artificial intelligence-based method -feedforward and recurrent Artificial Neural Networks (ANN) with Levenberg Marquardt (LM) training method based on Rough structure - is developed. The method is based on historical data including arrival time, departure time and trip length. In this study, the correlation among arrival time, departure time and trip length is also considered. The forecasted PEV-TB is then compared with Monte Carlo Simulation (MCS) which is the main benchmarking method in this field. The results comparison depicted the robustness of the proposed methodology. The proposed method reduces the aggregators’ financial loss approximately by 16 $/PEV per year compared to the conventional methods. The findings underline the importance of applying more accurate methods to forecast PEV-TB to gain the most benefit of vehicle electrification in the years to come., Peer Reviewed, Postprint (author's final draft)

In today’s world, mass-media and online social networks present us with unprecedented exposure to second-hand, vicarious experiences and thereby the chance of forming associations between previously innocuous events (e.g., being in a subway station) and aversive outcomes (e.g., footage or verbal reports from a violent terrorist attack) without direct experience. Such social threat, or fear, learning can have dramatic consequences, as manifested in acute stress symptoms and maladaptive fears. However, most research has so far focused on socially acquired threat responses that are expressed as increased arousal rather than active behavior. In three experiments (n = 120), we examined the effect of indirect experiences on behaviors by establishing a link between social threat learning and instrumental decision making. We contrasted learning from direct experience (i.e., Pavlovian conditioning) (experiment 1) against two common forms of social threat learning - social observation (experiment 2) and verbal instruction (experiment 3) - and how this learning transferred to subsequent instrumental decision making using behavioral experiments and computational modeling. We found that both types of social threat learning transfer to decision making in a strong and surprisingly inflexible manner. Notably, computational modeling indicated that the transfer of observational and instructed threat learning involved different computational mechanisms. Our results demonstrate the strong influence of others’ expressions of fear on one’s own decisions and have important implications for understanding both healthy and pathological human behaviors resulting from the indirect exposure to threatening events.

Motivated by the flexibility of biological neural networks whose connectivity structure changes significantly during their lifetime, we introduce the Unstructured Recursive Network (URN) and demonstrate that it can exhibit similar flexibility during training via gradient descent. We show empirically that many of the different neural network structures commonly used in practice today (including fully connected, locally connected and residual networks of different depths and widths) can emerge dynamically from the same URN. These different structures can be derived using gradient descent on a single general loss function where the structure of the data and the relative strengths of various regulator terms determine the structure of the emergent network. We show that this loss function and the regulators arise naturally when considering the symmetries of the network as well as the geometric properties of the input data., Comment: Proceedings of NeurIPS workshop on Real Neurons & Hidden Units. 5 Pages, 6 figures

We introduce a novel algorithm for the detection of possible sample corruption such as mislabeled samples in a training dataset given a small clean validation set. We use a set of inclusion variables which determine whether or not any element of the noisy training set should be included in the training of a network. We compute these inclusion variables by optimizing the performance of the network on the clean validation set via "gradient descent on gradient descent" based learning. The inclusion variables as well as the network trained in such a way form the basis of our methods, which we call Corruption Detection via Gradient Descent (CDGD). This algorithm can be applied to any supervised machine learning task and is not limited to classification problems. We provide a quantitative comparison of these methods on synthetic and real world datasets., Comment: 10 pages, 6 figures

We introduce Continual Learning via Neural Pruning (CLNP), a new method aimed at lifelong learning in fixed capacity models based on neuronal model sparsification. In this method, subsequent tasks are trained using the inactive neurons and filters of the sparsified network and cause zero deterioration to the performance of previous tasks. In order to deal with the possible compromise between model sparsity and performance, we formalize and incorporate the concept of graceful forgetting: the idea that it is preferable to suffer a small amount of forgetting in a controlled manner if it helps regain network capacity and prevents uncontrolled loss of performance during the training of future tasks. CLNP also provides simple continual learning diagnostic tools in terms of the number of free neurons left for the training of future tasks as well as the number of neurons that are being reused. In particular, we see in experiments that CLNP verifies and automatically takes advantage of the fact that the features of earlier layers are more transferable. We show empirically that CLNP leads to significantly improved results over current weight elasticity based methods., Comment: 12 pages, 5 figures, 3 tables

Earth observation satellite programs are currently facing, for some applications, the need to deliver hourly revisit times, subkilometric spatial resolutions, and near-real-time data access times. These stringent requirements, combined with the consolidation of small-satellite platforms and novel distributed architecture approaches, are stressing the need to study the design of new, heterogeneous, and heavily networked satellite systems that can potentially replace or complement traditional space assets. In this context, this paper presents partial results from ONION, a research project devoted to studying distributed satellite systems and their architecting characteristics. A design-oriented framework that allows selecting optimal architectures for the given user needs is presented in this paper. The framework has been used in the study of a strategic use-case and its results are hereby presented. From an initial design space of 5586 potential architectures, the framework has been able to preselect 28 candidate designs by an exhaustive analysis of their performance and by quantifying their quality attributes. This very exploration of architectures and the characteristics of the solution space are presented in this paper along with the selected solution and the results of a detailed performance analysis., Postprint (author's final draft)

Droop control is the conventional way to share the demand power among the generators in a microgrid. Analyzing this control method shows that it has poor performance in reactive power sharing. Generated reactive power of each microsource depends on the active load demand. In this paper, a control method is proposed to improve the reactive power sharing performance. It is indicated that higher reactive power droop gains make the generated reactive power to be less influenced from the changes in active load. However, high droop gains can lead the system to instability. Hence considering eigenvalue analysis of the microgrid, a new auxiliary controller is proposed to damp the oscillations caused by high droop gains. In the new method, there is not the need for monitoring the grid impedance to tune the controllers. To verify the proposed method, load step is applied to a test microgrid and the results are shown. In the new method, total generated reactive power is reduced in comparison to the conventional droop controller and the system stability is maintained.

We introduce backdrop, a flexible and simple-to-implement method, intuitively described as dropout acting only along the backpropagation pipeline. Backdrop is implemented via one or more masking layers which are inserted at specific points along the network. Each backdrop masking layer acts as the identity in the forward pass, but randomly masks parts of the backward gradient propagation. Intuitively, inserting a backdrop layer after any convolutional layer leads to stochastic gradients corresponding to features of that scale. Therefore, backdrop is well suited for problems in which the data have a multi-scale, hierarchical structure. Backdrop can also be applied to problems with non-decomposable loss functions where standard SGD methods are not well suited. We perform a number of experiments and demonstrate that backdrop leads to significant improvements in generalization., Comment: 11 pages, 9 figures, 2 tables. Source code available at https://github.com/dexgen/backdrop

Satellite architectures where networked, heterogeneous observation nodes capture data in a distributed manner are seen as feasible solutions to address the needs of next-generation Earth observation services (i.e. higher spatial, spectral and temporal resolutions at viable costs). Nevertheless, the problems that designers face when approaching these systems-of-systems are still eclipsed by the heterogeneity, dimensionality and multi-level complexity of those. In spite of the many underlying technological challenges, how to optimally architect distributed satellite systems, remains an open source of debate. In this context, this paper presents a design-oriented methodology that is aimed at providing high-level design solutions for this type of architectures in generic EO use-cases. In order to find optimal solutions, the methodology detailed in this paper is grounded on an aggregated architectural figure-of-merit that compresses: (a) system-level performance metrics; (b) use-case requirements; (c) development and launch costs; and (d) a set of architectural quality attributes. The latter contributing term models, assesses and weights several of the so-called 'ilities' of an architecture and allows to select designs that exhibit some desired qualities. With a dimensionality of more than five thousand architectural alternatives, the study has been illustrated with a marine weather forecast use-case. Both the exploration of design alternatives and the analysis of the results have shown the benefits of medium and small satellite platforms and have stressed their potential in the design of distributed satellite systems. Finally, this paper concludes by suggesting that this very optimization framework and methodology could also be used for a quantitative gap analysis aiming at deriving the technological road map for future engineering teams., Peer Reviewed, Postprint (published version)