In search of the Goldilocks zone for hybrid speciation

Hybridization has recently gained considerable interest both as a unique window for observing speciation mechanisms and as a potential engine of speciation. The latter remains a controversial topic. It was recently hypothesized that the reciprocal sorting of genetic incompatibilities from parental species could result in hybrid speciation, when the hybrid population maintains a mixed combination of the parental incompatibilities that prevents further gene exchange with both parental populations. However, the specifics of the purging/sorting process of multiple incompatibilities have not been examined theoretically.We here investigate the allele-frequency dynamics of an isolated hybrid population that results from a single hybridization event. Using models of 2 or 4 loci, we investigate the fate of one or two genetic incompatibilities of the Dobzhansky-Muller type (DMIs). We study how various parameters affect both the sorting/purging of the DMIs and the probability of observing hybrid speciation by reciprocal sorting. We find that the probability of hybrid speciation is strongly dependent on the linkage architecture (i.e. the order and recombination distance between loci along chromosomes), the population size of the hybrid population, and the initial relative contribution of the parental populations to the hybrid population. We identify a Goldilocks zone for specific linkage architectures and intermediate recombination rates, in which hybrid speciation becomes highly probable. Whereas an equal contribution of parental populations to the hybrid populations maximizes the hybrid speciation probability in the Goldilocks zone, other linkage architectures yield unintuitive asymmetric maxima. We provide an explanation for this pattern, and discuss our results both with respect to the best conditions for observing hybrid speciation in nature and their implications regarding patterns of introgression in hybrid zones.SummaryHybridization is observed ubiquitously in nature. Its outcome can range from extinction to the creation of new species. With respect to the latter, the probability of homoploid hybrid speciation, i.e. the formation of a new species as a result of an hybridization event without changes in the ploidy of the organism, is a hotly debated topic. Here, we analyze a minimal model for homoploid hybrid speciation, in which reproductive isolation is achieved by means of (postzygotic) Dobzhansky-Muller incompatibilities. When these postzygotic genetic incompatibilities are resolved in the hybrid population, their reciprocal sorting can result in reproductive isolation from both parental populations, thus creating a hybrid species. We show that, in accordance with the current literature, hybrid speciation tends to be rare. However, specific arrangements of the genes responsible for reproductive isolation can make reciprocal sorting almost unavoidable and thus create barriers to the parental population in an almost deterministic matter. We discuss the implications of these results for hybrid speciation and patterns of introgression in nature.