Search

A novel mixed quantum-classical approach to simulating nonadiabatic dynamics of molecules at metal surfaces is presented. The method combines the numerically exact hierarchical equations of motion approach for the quantum electronic degrees of freedom with Langevin dynamics for the classical degrees of freedom, namely, low-frequency vibrational modes within the molecule. The approach extends previous mixed quantum-classical methods based on Langevin equations to models containing strong electron-electron or quantum electronic-vibrational interactions, while maintaining a nonperturbative and non-Markovian treatment of the molecule-metal coupling. To demonstrate the approach, nonequilibrium transport observables are calculated for a molecular nanojunction containing strong interactions.

Understanding the dynamics of biomolecular complexes, e.g., of protein-ligand (un)binding, requires the understanding of paths such systems take between metastable states. In MD simulation data, paths are usually not observable per se, but need to be inferred from simulation trajectories. Here we present a novel approach to cluster trajectories based on a community detection algorithm that requires the definition of only a single free parameter. Using the streptavidin-biotin complex as benchmark system and the A\textsubscript{2a} adenosine receptor in complex with the inhibitor ZM241385 as an elaborate application, we demonstrate how such clusters of trajectories correspond to pathways, and how the approach helps in the identification of reaction coordinates for a considered (un)binding process., Comment: This preprint is the unedited version of a manuscript that has been published as peer-reviewed article by J. Chem. Theory Comput. and can be downloaded for private use only. Copyright with the journal, ACS and the authors

Segmentation of objects in microscopy images is required for many biomedical applications. We introduce object-centric embeddings (OCEs), which embed image patches such that the spatial offsets between patches cropped from the same object are preserved. Those learnt embeddings can be used to delineate individual objects and thus obtain instance segmentations. Here, we show theoretically that, under assumptions commonly found in microscopy images, OCEs can be learnt through a self-supervised task that predicts the spatial offset between image patches. Together, this forms an unsupervised cell instance segmentation method which we evaluate on nine diverse large-scale microscopy datasets. Segmentations obtained with our method lead to substantially improved results, compared to state-of-the-art baselines on six out of nine datasets, and perform on par on the remaining three datasets. If ground-truth annotations are available, our method serves as an excellent starting point for supervised training, reducing the required amount of ground-truth needed by one order of magnitude, thus substantially increasing the practical applicability of our method. Source code is available at https://github.com/funkelab/cellulus.

To sample rare events, dissipation-corrected targeted molecular dynamics (dcTMD) applies a constant velocity constraint along a one-dimensional reaction coordinate $s$, which drives an atomistic system from an initial state into a target state. Employing a cumulant approximation of Jarzynski's identity, the free energy $\Delta G (s)$ is calculated from the mean external work and dissipated work of the process. By calculating the friction coefficient $\Gamma (s)$ from the dissipated work, in a second step the equilibrium dynamics of the process can be studied by propagating a Langevin equation. While so far dcTMD has been mostly applied to study the unbinding of protein-ligand complexes, here its applicability to rare conformational transitions within a protein and the prediction of their kinetics is investigated. As this typically requires the introduction of multiple collective variables $\{x_j\}= \vec{x}$, a theoretical framework is outlined to calculate the associated free energy $\Delta G (\vec{x})$ and friction $\matrix{\Gamma}(\vec{x})$ from dcTMD simulations along coordinate $s$. Adopting the $\alpha$-$\beta$ transition of alanine dipeptide as well as the open-closed transition of T4 lysozyme as representative examples, the virtues and shortcomings of dcTMD to predict protein conformational transitions and the related kinetics are studied., Comment: This preprint is the unedited version of a manuscript that has been sent to a scientific journal for consideration as a publication and can be downloaded for private use only. Copyright with the journal and its publisher after publication

Supervised machine learning and deep learning require a large amount of labeled data, which data scientists obtain in a manual, and time-consuming annotation process. To mitigate this challenge, Active Learning (AL) proposes promising data points to annotators they annotate next instead of a subsequent or random sample. This method is supposed to save annotation effort while maintaining model performance. However, practitioners face many AL strategies for different tasks and need an empirical basis to choose between them. Surveys categorize AL strategies into taxonomies without performance indications. Presentations of novel AL strategies compare the performance to a small subset of strategies. Our contribution addresses the empirical basis by introducing a reproducible active learning evaluation (ALE) framework for the comparative evaluation of AL strategies in NLP. The framework allows the implementation of AL strategies with low effort and a fair data-driven comparison through defining and tracking experiment parameters (e.g., initial dataset size, number of data points per query step, and the budget). ALE helps practitioners to make more informed decisions, and researchers can focus on developing new, effective AL strategies and deriving best practices for specific use cases. With best practices, practitioners can lower their annotation costs. We present a case study to illustrate how to use the framework., Comment: The Version of Record of this contribution is published in Deep Learning Theory and Applications 4th International Conference, DeLTA 2023 Proceedings, and is available online at https://doi.org/10.1007/978-3-031-39059-3_16

Protein-ligand (un)binding simulations are a recent focus of biased molecular dynamics simulations. Such binding and unbinding can occur via different pathways in and out of a binding site. We here present a theoretical framework how to compute kinetics along separate paths and to combine the path-specific rates into global binding and unbinding rates for comparison with experiment. Using dissipation-corrected targeted molecular dynamics in combination with temperature-boosted Langevin equation simulations [Nat. Commun. \textbf{11}, 2918 (2020)] applied to a two-dimensional model and the trypsin-benzamidine complex as test systems, we assess the robustness of the procedure and discuss aspects of its practical applicability to predict multisecond kinetics of complex biomolecular systems., Comment: This preprint is the unedited version of a manuscript that has been published in J. Chen. Phys. and can be downloaded for private use only. Copyright with AIP and and the authors

While allostery is of paramount importance for protein signaling and regulation, the underlying dynamical process of allosteric communication is not well understood. PDZ3 domain represents a prime example of an allosteric single-domain protein, as it features a well-established long-range coupling between the C-terminal $\alpha_3$-helix and ligand binding. In an intriguing experiment, Hamm and coworkers employed photoswitching of the $\alpha_3$-helix to initiate a conformational change of PDZ3 that propagates from the C-terminus to the bound ligand within 200 ns. Performing extensive nonequilibrium molecular dynamics simulations, the modeling of the experiment reproduces the measured timescales and reveals a detailed picture of the allosteric communication in PDZ3. In particular, a correlation analysis identifies a network of contacts connecting the $\alpha_3$-helix and the core of the protein, which move in a concerted manner. Representing a one-step process and involving direct $\alpha_3$-ligand contacts, this cooperative transition is considered as elementary step in the propagation of conformational change.

Photoproteins such as bacteriorhodopsin (bR) and rhodopsin (Rho) need to effectively dissipate photoinduced excess energy to prevent their damage. Another well-studied G protein-coupled receptor (GPCR) containing 7 transmembrane (TM) helices is the B2 adrenergic receptor (B2AR), for which energy dissipation paths have been linked with allosteric communication. To study the vibrational energy transport in the active and inactive states of these GPCRs, a master equation approach [J. Chem. Phys. 152, 045103 (2020)] is employed, which uses scaling rules that allow to calculate energy transport rates solely based on the protein structure. Despite their structural similarities, the three GPCRs reveal quite different strategies to redistribute excess energy. While bR quickly removes the energy using the TM7 helix as a "lightning rod", Rho exhibits a rather poor energy dissipation, which might eventually require the hydrolysis of the Schiff base between the protein and the retinal chromophore to prevent overheating. Heating the ligand adrenaline of B2AR, the resulting energy transport network of the protein is found to change significantly upon switching from the active to the inactive state. While the energy flow may highlight aspects of the interresidue couplings of B2AR, it seems not particularly suited to explain allosteric phenomena.

We present two methods to reveal protein-ligand unbinding mechanisms in biased unbinding simulations by clustering trajectories into ensembles representing unbinding paths. The first approach is based on a contact principal component analysis for reducing the dimensionality of the input data, followed by identification of unbinding paths and training a machine learning model for trajectory clustering. The second approach clusters trajectories according to their pairwise mean Euclidean distance employing the neighbor-net algorithm, which takes into account input data bias in the distances set and is superior to dendrogram construction. Finally, we describe a more complex case where the reaction coordinate relevant for path identification is a single intra-ligand hydrogen bond, highlighting the challenges involved in unbinding path reaction coordinate detection., Comment: This preprint is the unformatted version of a manuscript that has been published as article in the Journal of Chemical Information and Modeling and can be downloaded for private use only. Copyright with ACS, the journal and the authors

The friction coefficient of fluids may become a function of the velocity at increased external driving. This non-Newtonian behavior is of general theoretical interest as well as of great practical importance, e.g., for the design of lubricants. While the effect has been observed in large-scale atomistic simulations of bulk liquids, its theoretical formulation and microscopic origin is not well understood. Here we use dissipation-corrected targeted molecular dynamics, which pulls apart two tagged liquid molecules in the presence of surrounding molecules and analyzes this nonequilibrium process via a generalized Langevin equation. The approach is based on a second-order cumulant expansion of Jarzynski's identity, which is shown to be valid for fluids and therefore allows for an exact computation of the friction profile as well of the underlying memory kernel. We show that velocity-dependent friction in fluids results from an intricate interplay of near-order structural effects and the non-Markovian behavior of the friction memory kernel. For complex fluids such as the model lubricant \alkane, the memory kernel exhibits a stretched-exponential long-time decay, which reflects the multitude of timescales of the system., Comment: This preprint is the unedited version of a manuscript that has been submitted to a scientific journal for consideration as publication and may be downloaded for personal use only. Copyright will be with the publisher after publication

Multi-band images of galaxies reveal a huge amount of information about their morphology and structure. However, inferring properties of the underlying stellar populations such as age, metallicity or kinematics from those images is notoriously difficult. Traditionally such information is best extracted from expensive spectroscopic observations. Here we present the $Painting\, IntrinsiC\, Attributes\, onto\, SDSS\, Objects$ (PICASSSO) project and test the information content of photometric multi-band images of galaxies. We train a convolutional neural network on 27,558 galaxy image pairs to establish a connection between broad-band images and the underlying physical stellar and gaseous galaxy property maps. We test our machine learning (ML) algorithm with SDSS $ugriz$ mock images for which uncertainties and systematics are exactly known. We show that multi-band galaxy images contain enough information to reconstruct 2d maps of stellar mass, metallicity, age and gas mass, metallicity as well as star formation rate. We recover the true stellar properties on a pixel by pixel basis with only little scatter, $\lesssim20\%$ compared to $\sim50\%$ statistical uncertainty from traditional mass-to-light-ratio based methods. We further test for any systematics of our algorithm with image resolution, training sample size or wavelength coverage. We find that galaxy morphology alone constrains stellar properties to better than $\sim20\%$ thus highlighting the benefits of including morphology into the parameter estimation. The machine learning approach can predict maps of high resolution, only limited by the resolution of the input bands, thus achieving higher resolution than IFU observations. The network architecture and all code is publicly available on GitHub., Comment: 14 pages main text, 9 figures, 3 pages appendix with 5 figures, submitted to MNRAS

Protein dynamics has been investigated on a wide range of time scales. Nano- and picosecond dynamics have been assigned to local fluctuations, while slower dynamics have been attributed to larger conformational changes. However, it is largely unknown how local fluctuations can lead to global allosteric changes. Here we show that molecule-spanning dynamics on the 100 ns time scale precede larger allosteric changes. We assign global real-space movements to dynamic modes on the 100 ns time scales, which became possible by a combination of single-molecule fluorescence, quasi-elastic neutron scattering and all-atom MD simulations. Additionally, we demonstrate the effect of Sba1, a co-chaperone of Hsp90, on these molecule-spanning dynamics, which implies functional importance of such dynamics. Our integrative approach provides comprehensive insights into molecule-spanning dynamics on the nanosecond time scale for a multi-domain protein and indicates that such dynamics are the molecular basis for allostery and large conformational changes in proteins., Comment: 29 main text pages, 7 main text figures; 27 SI pages, 15 SI figures

Given nonstationary data from molecular dynamics simulations, a Markovian Langevin model is constructed that aims to reproduce the time evolution of the underlying process. While at equilibrium the free energy landscape is sampled, nonequilibrium processes can be associated with a biased energy landscape, which accounts for finite sampling effects and external driving. Extending the data-driven Langevin equation (dLE) approach [Phys.\ Rev.\ Lett.\ {\bf 115}, 050602 (2015)] to the modeling of nonequilibrium processes, an efficient way to calculate multidimensional Langevin fields is outlined. The dLE is shown to correctly account for various nonequilibrium processes, including the enforced dissociation of sodium chloride in water, the pressure-jump induced nucleation of a liquid of hard spheres, and the conformational dynamics of a helical peptide sampled from nonstationary short trajectories., Comment: J. Phys. Chem. B 2021. This preprint is the unformatted and early version of a manuscript that has been published in an updated version in the Journal of Physical Chemistry B and may be downloaded for private use only. Copyright will with the journal / ACS

Ion channels are important proteins for physiological information transfer and functional control. To predict the microscopic origins of their voltage-conductance characteristics, we here applied dissipation-corrected targeted Molecular Dynamics in combination with Langevin equation simulations to potassium diffusion through the Gramicidin A channel as a test system. Performing a non-equilibrium principal component analysis on backbone dihedral angles, we find coupled protein-ion dynamics to occur during ion transfer. The dissipation-corrected free energy profiles correspond well to predictions from other biased simulation methods. The incorporation of an external electric field in Langevin simulations enables the prediction of macroscopic observables in the form of I-V characteristics., Comment: This preprint is the unedited version of a manuscript that has been published in J. Chem. Theory Comput. (2021) 10.1021/acs.jctc.1c00426 and can be downloaded for private use only. Copyright is with J. Chem. Theory Comput. and ACS

This work introduces a new proposal-free instance segmentation method that builds on single-instance segmentation masks predicted across the entire image in a sliding window style. In contrast to related approaches, our method concurrently predicts all masks, one for each pixel, and thus resolves any conflict jointly across the entire image. Specifically, predictions from overlapping masks are combined into edge weights of a signed graph that is subsequently partitioned to obtain all final instances concurrently. The result is a parameter-free method that is strongly robust to noise and prioritizes predictions with the highest consensus across overlapping masks. All masks are decoded from a low dimensional latent representation, which results in great memory savings strictly required for applications to large volumetric images. We test our method on the challenging CREMI 2016 neuron segmentation benchmark where it achieves competitive scores., Comment: Presented at GCPR 2020

While allostery is of paramount importance for protein regulation, the underlying dynamical process of ligand (un)binding at one site, resulting time evolution of the protein structure, and change of the binding affinity at a remote site is not well understood. Here the ligand-induced conformational transition in a widely studied model system of allostery, the PDZ2 domain, is investigated by transient infrared spectroscopy accompanied by molecular dynamics simulations. To this end, an azobenzene derived photoswitch is linked to a peptide ligand in a way that its binding affinity to the PDZ2 domain changes upon switching, thus initiating an allosteric transition in the PDZ2 domain protein. The subsequent response of the protein, covering four decades of time ranging from $\sim$1~ns to $\sim$10~$\mu$s, can be rationalize by a remodelling of its rugged free energy landscape, with ver subtle shifts in the populations of a small number of structurally well defined states. It is proposed that structurally and dynamically driven allostery, often discussed as limiting scenarios of allosteric communication, actually go hand-in-hand, allowing the protein to adapt its free energy landscape to incoming signals., Comment: This unedited earlier version of the manuscript may be downloaded for personal use only. Any other use requires prior permission of the author and the National Academy of Sciences USA. The final manuscript was published in Proceedings of the National Academy of Sciences USA 117, 26031-26039 (2020) and can be found under https://www.pnas.org/content/117/42/26031.short

Protein chains of the (FG)$_n$ ($n \simeq$ 300) type cap the cytoplasmatic side of the nucleopore complex, which connects the nucleus to the remainder of an eukaryotic cell. We study the properties of three fundamental polymer models that represent these filaments using Monte Carlo computer simulations. Random walks and the worm like chain model cannot account for the unusual size selectivity of the pore, while a two-dimensional arrangement of intrinsically disordered block copolymers with a high content of $\alpha$-helices is in agreement with the biochemical findings. We predict a linear increase of the free energy barrier of protein transport through the pore with increasing protein diameter, which can be probed experimentally using atomic force microscopy or optical tweezers., Comment: This document is the unedited version of a manuscript that has been submitted to a scientific journal for consideration as publication and may be downloaded for personal use only

Deep neural networks trained to inpaint partially occluded images show a deep understanding of image composition and have even been shown to remove objects from images convincingly. In this work, we investigate how this implicit knowledge of image composition can be leveraged for fully self-supervised instance separation. We propose a measure for the independence of two image regions given a fully self-supervised inpainting network and separate objects by maximizing this independence. We evaluate our method on two microscopy image datasets and show that it reaches similar segmentation performance to fully supervised methods.

Coarse-graining of fully atomistic molecular dynamics simulations is a long-standing goal in order to allow the description of processes occurring on biologically relevant timescales. For example, the prediction of pathways, rates and rate-limiting steps in protein-ligand unbinding is crucial for modern drug discovery. To achieve the enhanced sampling, we first perform dissipation-corrected targeted molecular dynamics simulations, which yield free energy and friction profiles of the molecular process under consideration. In a second step, we use these fields to perform temperature-boosted Langevin simulations which account for the desired molecular kinetics occurring on multisecond timescales and beyond. Adopting the dissociation of solvated sodium chloride as well as trypsin-benzamidine and Hsp90-inhibitor protein-ligand complexes as test problems, we are able to reproduce rates from molecular dynamics simulation and experiments within a factor of 2-20, and dissociation constants within a factor of 1-4. Analysis of the friction profiles reveals that binding and unbinding dynamics are mediated by changes of the surrounding hydration shells in all investigated systems., Comment: This unedited earlier version of the manuscript may be downloaded for personal use only. The final manuscript was published in Nature Communications 11, 2918 (2020) as open access publication and is available at https://www.nature.com/articles/s41467-020-16655-1

Semantic instance segmentation is the task of simultaneously partitioning an image into distinct segments while associating each pixel with a class label. In commonly used pipelines, segmentation and label assignment are solved separately since joint optimization is computationally expensive. We propose a greedy algorithm for joint graph partitioning and labeling derived from the efficient Mutex Watershed partitioning algorithm. It optimizes an objective function closely related to the Symmetric Multiway Cut objective and empirically shows efficient scaling behavior. Due to the algorithm's efficiency it can operate directly on pixels without prior over-segmentation of the image into superpixels. We evaluate the performance on the Cityscapes dataset (2D urban scenes) and on a 3D microscopy volume. In urban scenes, the proposed algorithm combined with current deep neural networks outperforms the strong baseline of `Panoptic Feature Pyramid Networks' by Kirillov et al. (2019). In the 3D electron microscopy images, we show explicitly that our joint formulation outperforms a separate optimization of the partitioning and labeling problems.

We here report on non-equilibrium targeted Molecular Dynamics simulations as tool for the estimation of protein-ligand unbinding kinetics. Correlating simulations with experimental data from SPR kinetics measurements and X-ray crystallography on two small molecule compound libraries bound to the N-terminal domain of the chaperone Hsp90, we show that the mean non-equilibrium work computed in an ensemble of trajectories of enforced ligand unbinding is a promising predictor for ligand unbinding rates. We furthermore investigate the molecular basis determining unbinding rates within the compound libraries. We propose ligand conformational changes and protein-ligand nonbonded interactions to impact on unbinding rates. Ligands may remain longer at the protein if they exhibit strong electrostatic and/or van der Waals interactions with the target. In the case of ligands with rigid chemical scaffold that exhibit longer residence times however, transient electrostatic interactions with the protein appear to facilitate unbinding. Our results imply that understanding the unbinding pathway and the protein-ligand interactions along this path is crucial for the prediction of small molecule ligands with defined unbinding, Comment: This unedited version of the article may be downloaded for personal use only. Any other use requires prior permission of the author and the American Chemical Society. This article appeared in J. Chem. Inf. Model. (2019), 10.1021/acs.jcim.9b00592 and may be found at https://pubs.acs.org/doi/10.1021/acs.jcim.9b00592

We humans seem to have an innate understanding of the asymmetric progression of time, which we use to efficiently and safely perceive and manipulate our environment. Drawing inspiration from that, we address the problem of learning an arrow of time in a Markov (Decision) Process. We illustrate how a learned arrow of time can capture meaningful information about the environment, which in turn can be used to measure reachability, detect side-effects and to obtain an intrinsic reward signal. We show empirical results on a selection of discrete and continuous environments, and demonstrate for a class of stochastic processes that the learned arrow of time agrees reasonably well with a known notion of an arrow of time given by the celebrated Jordan-Kinderlehrer-Otto result., Comment: A shorter version of this work was presented at the Theoretical Phyiscs for Deep Learning Workshop, ICML 2019

We propose a theoretical framework that generalizes simple and fast algorithms for hierarchical agglomerative clustering to weighted graphs with both attractive and repulsive interactions between the nodes. This framework defines GASP, a Generalized Algorithm for Signed graph Partitioning, and allows us to explore many combinations of different linkage criteria and cannot-link constraints. We prove the equivalence of existing clustering methods to some of those combinations and introduce new algorithms for combinations that have not been studied before. We study both theoretical and empirical properties of these combinations and prove that some of these define an ultrametric on the graph. We conduct a systematic comparison of various instantiations of GASP on a large variety of both synthetic and existing signed clustering problems, in terms of accuracy but also efficiency and robustness to noise. Lastly, we show that some of the algorithms included in our framework, when combined with the predictions from a CNN model, result in a simple bottom-up instance segmentation pipeline. Going all the way from pixels to final segments with a simple procedure, we achieve state-of-the-art accuracy on the CREMI 2016 EM segmentation benchmark without requiring domain-specific superpixels., Comment: Published in CVPR 2022

Image partitioning, or segmentation without semantics, is the task of decomposing an image into distinct segments, or equivalently to detect closed contours. Most prior work either requires seeds, one per segment; or a threshold; or formulates the task as multicut / correlation clustering, an NP-hard problem. Here, we propose an efficient algorithm for graph partitioning, the "Mutex Watershed''. Unlike seeded watershed, the algorithm can accommodate not only attractive but also repulsive cues, allowing it to find a previously unspecified number of segments without the need for explicit seeds or a tunable threshold. We also prove that this simple algorithm solves to global optimality an objective function that is intimately related to the multicut / correlation clustering integer linear programming formulation. The algorithm is deterministic, very simple to implement, and has empirically linearithmic complexity. When presented with short-range attractive and long-range repulsive cues from a deep neural network, the Mutex Watershed gives the best results currently known for the competitive ISBI 2012 EM segmentation benchmark.

Principal component analysis (PCA) represents a standard approach to identify collective variables $\{x_i\}\!=\!\boldsymbol{x}$, which can be used to construct the free energy landscape $\Delta G(\boldsymbol{x})$ of a molecular system. While PCA is routinely applied to equilibrium molecular dynamics (MD) simulations, it is less obvious how to extend the approach to nonequilibrium simulation techniques. This includes, e.g., the definition of the statistical averages employed in PCA, as well as the relation between the equilibrium free energy landscape $\Delta G(\boldsymbol{x})$ and energy landscapes $\Delta{\cal G} (\boldsymbol{x})$ obtained from nonequilibrium MD. As an example for a nonequilibrium method, `targeted MD' is considered which employs a moving distance constraint to enforce rare transitions along some biasing coordinate $s$. The introduced bias can be described by a weighting function $P(s)$, which provides a direct relation between equilibrium and nonequilibrium data, and thus establishes a well-defined way to perform PCA on nonequilibrium data. While the resulting distribution ${\cal P}(\boldsymbol{x})$ and energy $\Delta{\cal G} \propto \ln {\cal P}$ will not reflect the equilibrium state of the system, the nonequilibrium energy landscape $\Delta{\cal G} (\boldsymbol{x})$ may directly reveal the molecular reaction mechanism. Applied to targeted MD simulations of the unfolding of decaalanine, for example, a PCA performed on backbone dihedral angles is shown to discriminate several unfolding pathways. Although the formulation is in principle exact, its practical use depends critically on the choice of the biasing coordinate $s$, which should account for a naturally occurring motion between two well-defined end-states of the system., Comment: This article may be downloaded for personal use only. Any other use requires prior permission of the author and AIP Publishing. This article appeared in J. Chem. Phys., 150(20), 204110 and may be found at https://aip.scitation.org/doi/10.1063/1.5089636

Neuronal assemblies, loosely defined as subsets of neurons with reoccurring spatio-temporally coordinated activation patterns, or "motifs", are thought to be building blocks of neural representations and information processing. We here propose LeMoNADe, a new exploratory data analysis method that facilitates hunting for motifs in calcium imaging videos, the dominant microscopic functional imaging modality in neurophysiology. Our nonparametric method extracts motifs directly from videos, bypassing the difficult intermediate step of spike extraction. Our technique augments variational autoencoders with a discrete stochastic node, and we show in detail how a differentiable reparametrization and relaxation can be used. An evaluation on simulated data, with available ground truth, reveals excellent quantitative performance. In real video data acquired from brain slices, with no ground truth available, LeMoNADe uncovers nontrivial candidate motifs that can help generate hypotheses for more focused biological investigations.

Learned boundary maps are known to outperform hand- crafted ones as a basis for the watershed algorithm. We show, for the first time, how to train watershed computation jointly with boundary map prediction. The estimator for the merging priorities is cast as a neural network that is con- volutional (over space) and recurrent (over iterations). The latter allows learning of complex shape priors. The method gives the best known seeded segmentation results on the CREMI segmentation challenge., Comment: The first two authors contributed equally

We study the problem of multi-target tracking and data association in video. We formulate this in terms of selecting a subset of high-quality tracks subject to the constraint that no pair of selected tracks is associated with a common detection (of an object). This objective is equivalent to the classic NP-hard problem of finding a maximum-weight set packing (MWSP) where tracks correspond to sets and is made further difficult since the number of candidate tracks grows exponentially in the number of detections. We present a relaxation of this combinatorial problem that uses a column generation formulation where the pricing problem is solved via dynamic programming to efficiently explore the space of tracks. We employ row generation to tighten the bound in such a way as to preserve efficient inference in the pricing problem. We show the practical utility of this algorithm for tracking problems in natural and biological video datasets.