Search

Protein language models trained on the masked language modeling objective learn to predict the identity of hidden amino acid residues within a sequence using the remaining observable sequence as context. They do so by embedding the residues into a high dimensional space that encapsulates the relevant contextual cues. These embedding vectors serve as an informative context-sensitive representation that not only aids with the defined training objective, but can also be used for other tasks by downstream models. We propose a scheme to use the embeddings of an unmasked sequence to estimate the corresponding masked probability vectors for all the positions in a single forward pass through the language model. This One Fell Swoop (OFS) approach allows us to efficiently estimate the pseudo-perplexity of the sequence, a measure of the model's uncertainty in its predictions, that can also serve as a fitness estimate. We find that ESM2 OFS pseudo-perplexity performs nearly as well as the true pseudo-perplexity at fitness estimation, and more notably it defines a new state of the art on the ProteinGym Indels benchmark. The strong performance of the fitness measure prompted us to investigate if it could be used to detect the elevated stability reported in reconstructed ancestral sequences. We find that this measure ranks ancestral reconstructions as more fit than extant sequences. Finally, we show that the computational efficiency of the technique allows for the use of Monte Carlo methods that can rapidly explore functional sequence space.

What genotypic features explain the evolvability of organisms that have to accomplish many different tasks? The genotype of behaviorally complex organisms may be more likely to encode modular neural architectures because neural modules dedicated to distinct tasks avoid neural interference, i.e., the arrival of conflicting messages for changing the value of connection weights during learning. However, if the connection weights for the various modules are genetically inherited, this raises the problem of genetic linkage: favorable mutations may fall on one portion of the genotype encoding one neural module and unfavorable mutations on another portion encoding another module. We show that this can prevent the genotype from reaching an adaptive optimum. This effect is different from other linkage effects described in the literature and we argue that it represents a new class of genetic constraints. Using simulations we show that sexual reproduction can alleviate the problem of genetic linkage by recombining separate modules all of which incorporate either favorable or unfavorable mutations. We speculate that this effect may contribute to the taxonomic prevalence of sexual reproduction among higher organisms. In addition to sexual recombination, the problem of genetic linkage for behaviorally complex organisms may be mitigated by entrusting evolution with the task of finding appropriate modular architectures and learning with the task of finding the appropriate connection weights for these architectures.

The temperature dependence of the Seebeck coefficient of homoepitaxial metal organic vapor phase (MOVPE) grown, silicon doped $\beta$-Ga$_2$O$_3$ thin films was measured relative to aluminum. For room temperature we found the relative Seebeck coefficient of $S_{\beta\text{-Ga}_2\text{O}_3\text{-Al}}=(-300\pm20)\;\mu$V/K. At high bath temperatures $T>240$ K, the scattering is determined by electron-phonon-interaction. At lower bath temperatures between $T=100$ K and $T=300$ K, an increase in the magnitude of the Seebeck coefficient is explained in the frame of Strattons formula. The influence of the different scattering mechanisms on the magnitude of the Seebeck coefficient is discussed and compared with Hall measurement results., Comment: 6 pages, 4 figures

Thin films of the wide band gap semiconductor $\beta$-Ga$_2$O$_3$ have a high potential for applications in transparent electronics and high power devices. However, the role of interfaces remains to be explored. Here, we report on fundamental limits of transport properties in thin films. The conductivities, Hall densities and mobilities in thin homoepitaxially MOVPE grown \mbox{(100)-orientated $\beta$-Ga$_2$O$_3$} films were measured as a function of temperature and film thickness. At room temperature, the electron mobilities ((115$\pm$10) $\mathrm{\frac{cm^2}{Vs}}$) in thicker films (> 150 nm) are comparable to the best of bulk. However, the mobility is strongly reduced by more than two orders of magnitude with decreasing film thickness ((5.5$\pm$0.5) $\mathrm{\frac{cm^2}{Vs}}$ for a 28 nm thin film). We find that the commonly applied classical Fuchs-Sondheimer model does not explain the contribution of electron scattering at the film surfaces sufficiently. Instead, by applying an electron wave model by Bergmann, a contribution to the mobility suppression due to the large de Broglie wavelength in $\beta$-Ga$_2$O$_3$ is proposed as a limiting quantum mechanical size effect., Comment: 12 pages, 6 figures

It has been hypothesized that one of the main reasons evolution has been able to produce such impressive adaptations is because it has improved its own ability to evolve -- "the evolution of evolvability". Rupert Riedl, for example, an early pioneer of evolutionary developmental biology, suggested that the evolution of complex adaptations is facilitated by a developmental organization that is itself shaped by past selection to facilitate evolutionary innovation. However, selection for characteristics that enable future innovation seems paradoxical: natural selection cannot favor structures for benefits they have not yet produced, and favoring characteristics for benefits that have already been produced does not constitute future innovation. Here we resolve this paradox by exploiting a formal equivalence between the evolution of evolvability and learning systems. We use the conditions that enable simple learning systems to generalize, i.e., to use past experience to improve performance on previously unseen, future test cases, to demonstrate conditions where natural selection can systematically favor developmental organizations that benefit future evolvability. Using numerical simulations of evolution on highly epistatic fitness landscapes, we illustrate how the structure of evolved gene regulation networks can result in increased evolvability capable of avoiding local fitness peaks and discovering higher fitness phenotypes. Our findings support Riedl's intuition: Developmental organizations that "mimic" the organization of constraints on phenotypes can be favored by short-term selection and also facilitate future innovation. Importantly, the conditions that enable the evolution of such surprising evolvability follow from the same non-mysterious conditions that permit generalization in learning systems.

A major challenge in biology is explaining how novel characters originate, however, the molecular mechanisms that underlie the emergence of evolutionary innovations are unclear. Here we show that while gene expression in the uterus evolves at a slow and relatively constant rate, it has been punctuated by periods of rapid change associated with the recruitment of thousands of genes into uterine expression during the evolution of pregnancy in mammals. We found that numerous genes and signaling pathways essential for the establishment of pregnancy and maternal-fetal communication evolved uterine expression in mammals. Remarkably the majority of genes recruited into endometrial expression have cis-regulatory elements derived from lineage-specific transposons, suggesting that that bursts of transposition facilitate adaptation and speciation through genomic and regulatory reorganization.