Search

A recurrent neural network (RNN) is a widely used deep-learning network for dealing with sequential data. Imitating a dynamical system, an infinite-width RNN can approximate any open dynamical system in a compact domain. In general, deep networks with bounded widths are more effective than wide networks in practice; however, the universal approximation theorem for deep narrow structures has yet to be extensively studied. In this study, we prove the universality of deep narrow RNNs and show that the upper bound of the minimum width for universality can be independent of the length of the data. Specifically, we show that a deep RNN with ReLU activation can approximate any continuous function or $L^p$ function with the widths $d_x+d_y+2$ and $\max\{d_x+1,d_y\}$, respectively, where the target function maps a finite sequence of vectors in $\mathbb{R}^{d_x}$ to a finite sequence of vectors in $\mathbb{R}^{d_y}$. We also compute the additional width required if the activation function is $\tanh$ or more. In addition, we prove the universality of other recurrent networks, such as bidirectional RNNs. Bridging a multi-layer perceptron and an RNN, our theory and proof technique can be an initial step toward further research on deep RNNs.

There are inevitably many mislabeled data in real-world datasets. Because deep neural networks (DNNs) have an enormous capacity to memorize noisy labels, a robust training scheme is required to prevent labeling errors from degrading the generalization performance of DNNs. Current state-of-the-art methods present a co-training scheme that trains dual networks using samples associated with small losses. In practice, however, training two networks simultaneously can burden computing resources. In this study, we propose a simple yet effective robust training scheme that operates by training only a single network. During training, the proposed method generates temporal self-ensemble by sampling intermediate network parameters from the weight trajectory formed by stochastic gradient descent optimization. The loss sum evaluated with these self-ensembles is used to identify incorrectly labeled samples. In parallel, our method generates multi-view predictions by transforming an input data into various forms and considers their agreement to identify incorrectly labeled samples. By combining the aforementioned metrics, we present the proposed {\it self-ensemble-based robust training} (SRT) method, which can filter the samples with noisy labels to reduce their influence on training. Experiments on widely-used public datasets demonstrate that the proposed method achieves a state-of-the-art performance in some categories without training the dual networks.

We demonstrate the creation of a spin-1/2 state via the atomically controlled generation of magnetic carbon radical ions (CRIs) in synthetic two-dimensional transition metal dichalcogenides (TMDs). Hydrogenated carbon impurities located at chalcogen sites introduced by chemical doping can be activated with atomic precision by hydrogen depassivation using a scanning probe tip. In its anionic state, the carbon impurity exhibits a magnetic moment of 1 $\mu_\text{B}$ resulting from an unpaired electron populating a spin-polarized in-gap orbital of C$^{\bullet -}_\text{S}$. Fermi level control by the underlying graphene substrate can charge and decharge the defect, thereby activating or quenching the defect magnetic moment. By inelastic tunneling spectroscopy and density functional theory calculations we show that the CRI defect states couple to a small number of vibrational modes, including a local, breathing-type mode. Interestingly, the electron-phonon coupling strength critically depends on the spin state and differs for monolayer and bilayer WS$_2$. These carbon radical ions in TMDs comprise a new class of surface-bound, single-atom spin-qubits that can be selectively introduced, are spatially precise, feature a well-understood vibronic spectrum, and are charge state controlled.

Control of impurity concentrations in semiconducting materials is essential to device technology. Because of their intrinsic confinement, the properties of two-dimensional semiconductors such as transition metal dichalcogenides (TMDs) are more sensitive to defects than traditional bulk materials. The technological adoption of TMDs is dependent on the mitigation of deleterious defects and guided incorporation of functional foreign atoms. The first step towards impurity control is the identification of defects and assessment of their electronic properties. Here, we present a comprehensive study of point defects in monolayer tungsten disulfide (WS$_2$) grown by chemical vapor deposition (CVD) using scanning tunneling microscopy/spectroscopy, CO-tip noncontact atomic force microscopy, Kelvin probe force spectroscopy, density functional theory, and tight-binding calculations. We observe four different substitutional defects: chromium (Cr$_{\text{W}}$) and molybdenum (Mo$_{\text{W}}$) at a tungsten site, oxygen at sulfur sites in both bottom and top layers (O$_{\text{S}}$ top/bottom), as well as two negatively charged defects (CDs). Their electronic fingerprints unambiguously corroborate the defect assignment and reveal the presence or absence of in-gap defect states. The important role of charge localization, spin-orbit coupling, and strain for the formation of deep defect states observed at substitutional defects in WS$_2$ as reported here will guide future efforts of targeted defect engineering and doping of TMDs.

A detailed understanding of charged defects in two-dimensional semiconductors is needed for the development of ultrathin electronic devices. Here, we study negatively charged acceptor impurities in monolayer WS$_2$ using a combination of scanning tunnelling spectroscopy and large-scale atomistic electronic structure calculations. We observe several localized defect states of hydrogenic wave function character in the vicinity of the valence band edge. Some of these defect states are bound, while others are resonant. The resonant states result from the multi-valley valence band structure of WS$_2$, whereby localized states originating from the secondary valence band maximum at $\Gamma$ hybridize with continuum states from the primary valence band maximum at K/K$^{\prime}$. Resonant states have important consequences for electron transport as they can trap mobile carriers for several tens of picoseconds., Comment: 6 pages, 3 figures