Search

Topological superconductors are essential elements of the periodic table of topological quantum matter. However, the relevant odd-parity spin-triplet superconductors are rare. We report high-resolution measurements of the complex electrical impedance of YbRh$_2$Si$_2$ down to 0.4 mK, that reveal the presence of several superconducting states, suppressed differently by magnetic field, both Pauli-limited and beyond the Pauli limit. Superconductivity is abruptly switched off at the critical field of the primary antiferromagnetic order. The onset of electro-nuclear spin density wave order enhances the superconductivity, which we account for by the simultaneous formation of a spin-triplet pair density wave. Together these observations provide compelling evidence for odd-parity superconductivity, and its underpinning by antiferromagnetism, and allow us to identify the topological helical state., Comment: 24 pages, 4 main and 15 supplementary figures, 1 supplementary table

Concept-based learning enhances prediction accuracy and interpretability by leveraging high-level, human-understandable concepts. However, existing CBL frameworks do not address survival analysis tasks, which involve predicting event times in the presence of censored data -- a common scenario in fields like medicine and reliability analysis. To bridge this gap, we propose two novel models: SurvCBM (Survival Concept-based Bottleneck Model) and SurvRCM (Survival Regularized Concept-based Model), which integrate concept-based learning with survival analysis to handle censored event time data. The models employ the Cox proportional hazards model and the Beran estimator. SurvCBM is based on the architecture of the well-known concept bottleneck model, offering interpretable predictions through concept-based explanations. SurvRCM uses concepts as regularization to enhance accuracy. Both models are trained end-to-end and provide interpretable predictions in terms of concepts. Two interpretability approaches are proposed: one leveraging the linear relationship in the Cox model and another using an instance-based explanation framework with the Beran estimator. Numerical experiments demonstrate that SurvCBM outperforms SurvRCM and traditional survival models, underscoring the importance and advantages of incorporating concept information. The code for the proposed algorithms is publicly available.

We report the determination of the magnetic phase diagram of the heavy fermion metal YbRh\textsubscript{2}Si\textsubscript{2} in magnetic fields up to 70\,mT applied perpendicular to the crystallographic c-axis. By a combination of heat capacity, magneto-caloric, and magneto-resistance measurements we map two antiferromagnetic phases: the electronic AFM1 below 70\,mK and electro-nuclear AFM2 below 1.5\,mK. The measurements extend into the microkelvin regime to explore the quantum phase transitions in this system. We demonstrate how the hyperfine interaction significantly modifies the phase diagram and the putative field-tuned quantum critical point. The determination of the rich magnetic properties of YbRh\textsubscript{2}Si\textsubscript{2} is essential to understanding the interplay of the two magnetic orders and superconductivity in this compound.

Many ensemble-based models have been proposed to solve machine learning problems in the survival analysis framework, including random survival forests, the gradient boosting machine with weak survival models, ensembles of the Cox models. To extend the set of models, a new ensemble-based model called SurvBETA (the Survival Beran estimator Ensemble using Three Attention mechanisms) is proposed where the Beran estimator is used as a weak learner in the ensemble. The Beran estimator can be regarded as a kernel regression model taking into account the relationship between instances. Outputs of weak learners in the form of conditional survival functions are aggregated with attention weights taking into account the distance between the analyzed instance and prototypes of all bootstrap samples. The attention mechanism is used three times: for implementation of the Beran estimators, for determining specific prototypes of bootstrap samples and for aggregating the weak model predictions. The proposed model is presented in two forms: in a general form requiring to solve a complex optimization problem for its training; in a simplified form by considering a special representation of the attention weights by means of the imprecise Huber's contamination model which leads to solving a simple optimization problem. Numerical experiments illustrate properties of the model on synthetic data and compare the model with other survival models on real data. A code implementing the proposed model is publicly available.

Anisotropic pair breaking close to surfaces favors chiral superfluid $^3$He-A over time-reversal invariant $^3$He-B. Confining superfluid $^3$He into a cavity of height $D$ of the order of the Cooper pair size characterized by the coherence length $\xi_0$ -- ranging between 16 nm (34 bar) and 77 nm (0 bar) -- extends the surface effects over the whole sample volume, thus allowing stabilization of the A phase at pressures $P$ and temperatures $T$ where otherwise the B phase would be stable. In this work the surfaces of such a confined sample are covered with a superfluid $^4$He film to create specular quasiparticle scattering boundary conditions, preventing the suppression of the superfluid order parameter. We show that the chiral A phase is the stable superfluid phase under strong confinement over the full $P-T$ phase diagram down to a quasi-two-dimensional limit $D/\xi_0 = 1$. The planar phase, which is degenerate with the chiral A phase in the weak-coupling limit, is not observed. The gap inferred from measurements over the wide pressure range from 0.2 to 21.0 bar leads to an empirical ansatz for temperature-dependent strong-coupling effects. We discuss how these results pave the way for the realization of the fully-gapped two-dimensional $p_x + ip_y$ superfluid under more extreme confinement., Comment: 11 pages, 8 figures, including the Supplemental Material

A method for solving concept-based learning (CBL) problem is proposed. The main idea behind the method is to divide each concept-annotated image into patches, to transform the patches into embeddings by using an autoencoder, and to cluster the embeddings assuming that each cluster will mainly contain embeddings of patches with certain concepts. To find concepts of a new image, the method implements the frequentist inference by computing prior and posterior probabilities of concepts based on rates of patches from images with certain values of the concepts. Therefore, the proposed method is called the Frequentist Inference CBL (FI-CBL). FI-CBL allows us to incorporate the expert rules in the form of logic functions into the inference procedure. An idea behind the incorporation is to update prior and conditional probabilities of concepts to satisfy the rules. The method is transparent because it has an explicit sequence of probabilistic calculations and a clear frequency interpretation. Numerical experiments show that FI-CBL outperforms the concept bottleneck model in cases when the number of training data is small. The code of proposed algorithms is publicly available.

We describe a compact calorimeter that opens ultra-low temperature heat capacity studies of small metal crystals in moderate magnetic fields. The performance is demonstrated on the canonical heavy Fermion metal YbRh2Si2. Thermometry is provided by a fast current sensing noise thermometer. This single thermometer enables us to cover a wide temperature range of interest from 175 $\mu$K to 90 mK with temperature independent relative precision. Temperatures are tied to the international temperature scale with a single point calibration. A superconducting solenoid surrounding the cell provides the sample field for tuning its properties and operates a superconducting heat switch. Both adiabatic and relaxation calorimetry techniques, as well as magnetic field sweeps, are employed. The design of the calorimeter results in an addendum heat capacity which is negligible for the study reported. The keys to sample and thermometer thermalisation are the lack of dissipation in the temperature measurement and the steps taken to reduce the parasitic heat leak into the cell to the tens of fW level., Comment: 19 pages, 10 figures

To increase the transparency of modern computer-aided diagnosis (CAD) systems for assessing the malignancy of lung nodules, an interpretable model based on applying the generalized additive models and the concept-based learning is proposed. The model detects a set of clinically significant attributes in addition to the final malignancy regression score and learns the association between the lung nodule attributes and a final diagnosis decision as well as their contributions into the decision. The proposed concept-based learning framework provides human-readable explanations in terms of different concepts (numerical and categorical), their values, and their contribution to the final prediction. Numerical experiments with the LIDC-IDRI dataset demonstrate that the diagnosis results obtained using the proposed model, which explicitly explores internal relationships, are in line with similar patterns observed in clinical practice. Additionally, the proposed model shows the competitive classification and the nodule attribute scoring performance, highlighting its potential for effective decision-making in the lung nodule diagnosis.

A problem of incorporating the expert rules into machine learning models for extending the concept-based learning is formulated in the paper. It is proposed how to combine logical rules and neural networks predicting the concept probabilities. The first idea behind the combination is to form constraints for a joint probability distribution over all combinations of concept values to satisfy the expert rules. The second idea is to represent a feasible set of probability distributions in the form of a convex polytope and to use its vertices or faces. We provide several approaches for solving the stated problem and for training neural networks which guarantee that the output probabilities of concepts would not violate the expert rules. The solution of the problem can be viewed as a way for combining the inductive and deductive learning. Expert rules are used in a broader sense when any logical function that connects concepts and class labels or just concepts with each other can be regarded as a rule. This feature significantly expands the class of the proposed results. Numerical examples illustrate the approaches. The code of proposed algorithms is publicly available.

A new model for generating survival trajectories and data based on applying an autoencoder of a specific structure is proposed. It solves three tasks. First, it provides predictions in the form of the expected event time and the survival function for a new generated feature vector on the basis of the Beran estimator. Second, the model generates additional data based on a given training set that would supplement the original dataset. Third, the most important, it generates a prototype time-dependent trajectory for an object, which characterizes how features of the object could be changed to achieve a different time to an event. The trajectory can be viewed as a type of the counterfactual explanation. The proposed model is robust during training and inference due to a specific weighting scheme incorporating into the variational autoencoder. The model also determines the censored indicators of new generated data by solving a classification task. The paper demonstrates the efficiency and properties of the proposed model using numerical experiments on synthetic and real datasets. The code of the algorithm implementing the proposed model is publicly available.

A new approach to the local and global explanation is proposed. It is based on selecting a convex hull constructed for the finite number of points around an explained instance. The convex hull allows us to consider a dual representation of instances in the form of convex combinations of extreme points of a produced polytope. Instead of perturbing new instances in the Euclidean feature space, vectors of convex combination coefficients are uniformly generated from the unit simplex, and they form a new dual dataset. A dual linear surrogate model is trained on the dual dataset. The explanation feature importance values are computed by means of simple matrix calculations. The approach can be regarded as a modification of the well-known model LIME. The dual representation inherently allows us to get the example-based explanation. The neural additive model is also considered as a tool for implementing the example-based explanation approach. Many numerical experiments with real datasets are performed for studying the approach. The code of proposed algorithms is available.

First order phase transitions in the very early universe are a prediction of many extensions of the Standard Model of particle physics and could provide the departure from equilibrium needed for a dynamical explanation of the baryon asymmetry of the Universe. They could also produce gravitational waves of a frequency observable by future space-based detectors such as the Laser Interferometer Space Antenna (LISA). All calculations of the gravitational wave power spectrum rely on a relativistic version of the classical nucleation theory of Cahn-Hilliard and Langer, due to Coleman and Linde. The high purity and precise control of pressure and temperature achievable in the laboratory made the first-order A to B transition of superfluid $^3$He an ideal for test of classical nucleation theory. As Leggett and others have noted the theory fails dramatically. The lifetime of the metastable A phase is measurable, typically of order minutes to hours, far faster than classical nucleation theory predicts. If the nucleation of B phase from the supercooled A phase is due to a new, rapid intrinsic mechanism that would have implications for first-order cosmological phase transitions as well as predictions for gravitational wave (GW) production in the early universe. Here we discuss studies of the AB phase transition dynamics in $^3$He, both experimental and theoretical, and show how the computational technology for cosmological phase transition can be used to simulate the dynamics of the A-B transition, support the experimental investigations of the A-B transition in the QUEST-DMC collaboration with the goal of identifying and quantifying the mechanism(s) responsible for nucleation of stable phases in ultra-pure metastable quantum phases., Comment: 34 pp, 4 figures. Based on a talk given at Quantum Fluids and Solids 2023, Manchester, U.K, on behalf of the QUEST-DMC collaboration

The symmetry-breaking first-order phase transition between superfluid phases $^3$He-A and $^3$He-B can be triggered extrinsically by ionising radiation or heterogeneous nucleation arising from the details of the sample cell construction. However, the role of potential homogeneous intrinsic nucleation mechanisms remains elusive. Discovering and resolving the intrinsic processes may have cosmological consequences, since an analogous first-order phase transition, and the production of gravitational waves, has been predicted for the very early stages of the expanding Universe in many extensions of the Standard Model of particle physics. Here we introduce a new approach for probing the phase transition in superfluid $^3$He. The setup consists of a novel stepped-height nanofluidic sample container with close to atomically smooth walls. The $^3$He is confined in five tiny nanofabricated volumes and assayed non-invasively by NMR. Tuning of the state of $^3$He by confinement is used to isolate each of these five volumes so that the phase transitions in them can occur independently and free from any obvious sources of heterogeneous nucleation. The small volumes also ensure that the transitions triggered by ionising radiation are strongly suppressed. Here we present the preliminary measurements using this setup, showing both strong supercooling of $^3$He-A and superheating of $^3$He-B, with stochastic processes dominating the phase transitions between the two. The objective is to study the nucleation as a function of temperature and pressure over the full phase diagram, to both better test the proposed extrinsic mechanisms and seek potential parallel intrinsic mechanisms., Comment: 18 pages, 6 figures, part of the Proceedings of the International Conference on Quantum Fluids and Solids 2023. This is a copy of the version of record of this article, first published in Journal of Low Temperature Physics, available online at Publisher's website: https://doi.org/10.1007/s10909-024-03146-6

A new method called the Survival Beran-based Neural Importance Model (SurvBeNIM) is proposed. It aims to explain predictions of machine learning survival models, which are in the form of survival or cumulative hazard functions. The main idea behind SurvBeNIM is to extend the Beran estimator by incorporating the importance functions into its kernels and by implementing these importance functions as a set of neural networks which are jointly trained in an end-to-end manner. Two strategies of using and training the whole neural network implementing SurvBeNIM are proposed. The first one explains a single instance, and the neural network is trained for each explained instance. According to the second strategy, the neural network only learns once on all instances from the dataset and on all generated instances. Then the neural network is used to explain any instance in a dataset domain. Various numerical experiments compare the method with different existing explanation methods. A code implementing the proposed method is publicly available.

Electrical and thermal transport were studied in a vacuum-annealed polycrystalline silver wire with residual resistivity ratio 200-400, in the temperature range 0.1-1.2K and in magnetic fields up to 5T. Both at zero field and at 5T the wire exhibits the Wiedemann-Franz law with the fundamental Lorenz number, contrary to an earlier report [Gloos, K. et al, Cryogenics 30, 14 (1990)]. Our result demonstrates that silver is an excellent material for thermal links in ultra-low-temperature experiments operating at high magnetic fields., Comment: 8 pages, 3 figures

We measure electron transport through point contacts in an electron gas in AlGaAs/GaAs heterostructures and graphene for a range of temperatures, magnetic fields and electron densities. We find a magnetoconductance peak around B = 0. With increasing temperature, the width of the peak increases monotonically, while its amplitude first increases and then decreases. For GaAs point contacts the peak is particularly sharp at relatively low temperatures $T\approx$1.5 K: the curve rounds on a scale of few tens of $\mu$T hinting at length scales of several millimeters for the corresponding scattering processes. We propose a model based on the transition between different transport regimes with increasing temperature: from ballistic transport to few electron-electron scatterings to hydrodynamic superballistic flow to hydrodynamic Poiseuille-like flow. The model is in qualitative and, in many cases, quantitative agreement with the experimental observations., Comment: 14 pages, 11 figures

An explanation method called SurvBeX is proposed to interpret predictions of the machine learning survival black-box models. The main idea behind the method is to use the modified Beran estimator as the surrogate explanation model. Coefficients, incorporated into Beran estimator, can be regarded as values of the feature impacts on the black-box model prediction. Following the well-known LIME method, many points are generated in a local area around an example of interest. For every generated example, the survival function of the black-box model is computed, and the survival function of the surrogate model (the Beran estimator) is constructed as a function of the explanation coefficients. In order to find the explanation coefficients, it is proposed to minimize the mean distance between the survival functions of the black-box model and the Beran estimator produced by the generated examples. Many numerical experiments with synthetic and real survival data demonstrate the SurvBeX efficiency and compare the method with the well-known method SurvLIME. The method is also compared with the method SurvSHAP. The code implementing SurvBeX is available at: https://github.com/DanilaEremenko/SurvBeX

A new computationally simple method of imposing hard convex constraints on the neural network output values is proposed. The key idea behind the method is to map a vector of hidden parameters of the network to a point that is guaranteed to be inside the feasible set defined by a set of constraints. The mapping is implemented by the additional neural network layer with constraints for output. The proposed method is simply extended to the case when constraints are imposed not only on the output vectors, but also on joint constraints depending on inputs. The projection approach to imposing constraints on outputs can simply be implemented in the framework of the proposed method. It is shown how to incorporate different types of constraints into the proposed method, including linear and quadratic constraints, equality constraints, and dynamic constraints, constraints in the form of boundaries. An important feature of the method is its computational simplicity. Complexities of the forward pass of the proposed neural network layer by linear and quadratic constraints are O(n*m) and O(n^2*m), respectively, where n is the number of variables, m is the number of constraints. Numerical experiments illustrate the method by solving optimization and classification problems. The code implementing the method is publicly available.

Breaking of inversion symmetry leads to nonlinear and nonreciprocal electron transport, in which the voltage response does not invert with the reversal of the current direction. Many systems have incorporated inversion symmetry breaking into their band or crystal structures. In this work, we demonstrate that a conventional two-dimensional electron gas (2DEG) system with a back gate shows non-reciprocal behavior (with voltage proportional to current squared) in the quantum Hall regime, which depends on the out-of-plane magnetic field and contact configuration. The inversion symmetry is broken due to the presence of the back gate and magnetic field, and our phenomenological model provides a qualitative explanation of the experimental data. Our results suggest a universal mechanism that gives rise to non-reciprocal behavior in gated samples., Comment: 8 pages, 6 figures

A new approach called NAF (the Neural Attention Forest) for solving regression and classification tasks under tabular training data is proposed. The main idea behind the proposed NAF model is to introduce the attention mechanism into the random forest by assigning attention weights calculated by neural networks of a specific form to data in leaves of decision trees and to the random forest itself in the framework of the Nadaraya-Watson kernel regression. In contrast to the available models like the attention-based random forest, the attention weights and the Nadaraya-Watson regression are represented in the form of neural networks whose weights can be regarded as trainable parameters. The first part of neural networks with shared weights is trained for all trees and computes attention weights of data in leaves. The second part aggregates outputs of the tree networks and aims to minimize the difference between the random forest prediction and the truth target value from a training set. The neural network is trained in an end-to-end manner. The combination of the random forest and neural networks implementing the attention mechanism forms a transformer for enhancing the forest predictions. Numerical experiments with real datasets illustrate the proposed method. The code implementing the approach is publicly available., Comment: Submitted for the 7th International Scientific Conference "Intelligent Information Technologies for Industry" in St. Petersburg

A new extremely simple ensemble-based model with the uniformly generated axis-parallel hyper-rectangles as base models (HRBM) is proposed. Two types of HRBMs are studied: closed rectangles and corners. The main idea behind HRBM is to consider and count training examples inside and outside each rectangle. It is proposed to incorporate HRBMs into the gradient boosting machine (GBM). Despite simplicity of HRBMs, it turns out that these simple base models allow us to construct effective ensemble-based models and avoid overfitting. A simple method for calculating optimal regularization parameters of the ensemble-based model, which can be modified in the explicit way at each iteration of GBM, is considered. Moreover, a new regularization called the "step height penalty" is studied in addition to the standard L1 and L2 regularizations. An extremely simple approach to the proposed ensemble-based model prediction interpretation by using the well-known method SHAP is proposed. It is shown that GBM with HRBM can be regarded as a model extending a set of interpretable models for explaining black-box models. Numerical experiments with real datasets illustrate the proposed GBM with HRBMs for regression and classification problems. Experiments also illustrate computational efficiency of the proposed SHAP modifications. The code of proposed algorithms implementing GBM with HRBM is publicly available.

A new random forest based model for solving the Multiple Instance Learning (MIL) problem under small tabular data, called Soft Tree Ensemble MIL (STE-MIL), is proposed. A new type of soft decision trees is considered, which is similar to the well-known soft oblique trees, but with a smaller number of trainable parameters. In order to train the trees, it is proposed to convert them into neural networks of a specific form, which approximate the tree functions. It is also proposed to aggregate the instance and bag embeddings (output vectors) by using the attention mechanism. The whole STE-MIL model, including soft decision trees, neural networks, the attention mechanism and a classifier, is trained in an end-to-end manner. Numerical experiments with tabular datasets illustrate STE-MIL. The corresponding code implementing the model is publicly available.

The discovery of superconductivity in the quantum critical Kondo-lattice system YbRh2Si2 at an extremely low temperature of 2 mK has inspired efforts to perform high-resolution electrical resistivity measurements down to this temperature range in highly conductive materials. Here we show that control over the sample geometry by microstructuring using focused-ion-beam (FIB) techniques allows to reach ultra-low temperatures and increase signal-to-noise ratios (SNR) tenfold, without adverse effects to sample quality. In five experiments we show four-terminal sensing resistance and magnetoresistance measurements which exhibit sharp phase transitions at the N\'eel temperature, and Shubnikov-de-Haas (SdH) oscillations between 13 T and 18 T where we identified a new SdH frequency of 0.39 kT. The increased SNR allowed resistance fluctuation (noise) spectroscopy that would not be possible for bulk crystals, and confirmed intrinsic 1/f-type fluctuations. Under controlled strain, two thin microstructured samples exhibited a large increase of T_N from 67 mK up to 188 mK while still showing clear signatures of the phase transition and SdH oscillations. SQUID-based thermal noise spectroscopy measurements in a nuclear demagnetisation refrigerator down to 0.95 mK, show a sharp superconducting transition at T_c = 1.2 mK. These experiments demonstrate microstructuring as a powerful tool to investigate the resistance and the noise spectrum of highly conductive correlated metals over wide temperature ranges., Comment: 23 pages, 7 figures

A method for estimating the conditional average treatment effect under condition of censored time-to-event data called BENK (the Beran Estimator with Neural Kernels) is proposed. The main idea behind the method is to apply the Beran estimator for estimating the survival functions of controls and treatments. Instead of typical kernel functions in the Beran estimator, it is proposed to implement kernels in the form of neural networks of a specific form called the neural kernels. The conditional average treatment effect is estimated by using the survival functions as outcomes of the control and treatment neural networks which consists of a set of neural kernels with shared parameters. The neural kernels are more flexible and can accurately model a complex location structure of feature vectors. Various numerical simulation experiments illustrate BENK and compare it with the well-known T-learner, S-learner and X-learner for several types of the control and treatment outcome functions based on the Cox models, the random survival forest and the Nadaraya-Watson regression with Gaussian kernels. The code of proposed algorithms implementing BENK is available in https://github.com/Stasychbr/BENK.

New models of the attention-based random forests called LARF (Leaf Attention-based Random Forest) are proposed. The first idea behind the models is to introduce a two-level attention, where one of the levels is the "leaf" attention and the attention mechanism is applied to every leaf of trees. The second level is the tree attention depending on the "leaf" attention. The second idea is to replace the softmax operation in the attention with the weighted sum of the softmax operations with different parameters. It is implemented by applying a mixture of the Huber's contamination models and can be regarded as an analog of the multi-head attention with "heads" defined by selecting a value of the softmax parameter. Attention parameters are simply trained by solving the quadratic optimization problem. To simplify the tuning process of the models, it is proposed to make the tuning contamination parameters to be training and to compute them by solving the quadratic optimization problem. Many numerical experiments with real datasets are performed for studying LARFs. The code of proposed algorithms can be found in https://github.com/andruekonst/leaf-attention-forest.

A new modification of Isolation Forest called Attention-Based Isolation Forest (ABIForest) for solving the anomaly detection problem is proposed. It incorporates the attention mechanism in the form of the Nadaraya-Watson regression into the Isolation Forest for improving solution of the anomaly detection problem. The main idea underlying the modification is to assign attention weights to each path of trees with learnable parameters depending on instances and trees themselves. The Huber's contamination model is proposed to be used for defining the attention weights and their parameters. As a result, the attention weights are linearly depend on the learnable attention parameters which are trained by solving the standard linear or quadratic optimization problem. ABIForest can be viewed as the first modification of Isolation Forest, which incorporates the attention mechanism in a simple way without applying gradient-based algorithms. Numerical experiments with synthetic and real datasets illustrate outperforming results of ABIForest. The code of proposed algorithms is available.

Improved accessibility to the microkelvin temperature regime is important for future research in quantum materials; for quantum information science; and for applications of quantum sensors. Here we report the design and performance of a microkelvin platform based on a nuclear demagnetization stage, engineered and well optimized for operation on a standard cryogen-free dilution refrigerator. PrNi5 is used as the dominant refrigerant. The platform provides a large area for mounting experiments in an ultralow temperature, low electromagnetic noise environment. The performance is characterized using current sensing noise thermometry. Temperatures as low as 395 $\mu$K have been reached, and a protocol has been established in which it is possible to operate experiments below 1 mK for 95% of the time, providing an efficient cryogen-free microkelvin environment for a wide range of science applications, Comment: 8 pages, 5 figures

A new method for estimating the conditional average treatment effect is proposed in the paper. It is called TNW-CATE (the Trainable Nadaraya-Watson regression for CATE) and based on the assumption that the number of controls is rather large whereas the number of treatments is small. TNW-CATE uses the Nadaraya-Watson regression for predicting outcomes of patients from the control and treatment groups. The main idea behind TNW-CATE is to train kernels of the Nadaraya-Watson regression by using a weight sharing neural network of a specific form. The network is trained on controls, and it replaces standard kernels with a set of neural subnetworks with shared parameters such that every subnetwork implements the trainable kernel, but the whole network implements the Nadaraya-Watson estimator. The network memorizes how the feature vectors are located in the feature space. The proposed approach is similar to the transfer learning when domains of source and target data are similar, but tasks are different. Various numerical simulation experiments illustrate TNW-CATE and compare it with the well-known T-learner, S-learner and X-learner for several types of the control and treatment outcome functions. The code of proposed algorithms implementing TNW-CATE is available in https://github.com/Stasychbr/TNW-CATE.

New models of random forests jointly using the attention and self-attention mechanisms are proposed for solving the regression problem. The models can be regarded as extensions of the attention-based random forest whose idea stems from applying a combination of the Nadaraya-Watson kernel regression and the Huber's contamination model to random forests. The self-attention aims to capture dependencies of the tree predictions and to remove noise or anomalous predictions in the random forest. The self-attention module is trained jointly with the attention module for computing weights. It is shown that the training process of attention weights is reduced to solving a single quadratic or linear optimization problem. Three modifications of the general approach are proposed and compared. A specific multi-head self-attention for the random forest is also considered. Heads of the self-attention are obtained by changing its tuning parameters including the kernel parameters and the contamination parameter of models. Numerical experiments with various datasets illustrate the proposed models and show that the supplement of the self-attention improves the model performance for many datasets., Comment: arXiv admin note: text overlap with arXiv:2201.02880

A new approach called ABRF (the attention-based random forest) and its modifications for applying the attention mechanism to the random forest (RF) for regression and classification are proposed. The main idea behind the proposed ABRF models is to assign attention weights with trainable parameters to decision trees in a specific way. The weights depend on the distance between an instance, which falls into a corresponding leaf of a tree, and instances, which fall in the same leaf. This idea stems from representation of the Nadaraya-Watson kernel regression in the form of a RF. Three modifications of the general approach are proposed. The first one is based on applying the Huber's contamination model and on computing the attention weights by solving quadratic or linear optimization problems. The second and the third modifications use the gradient-based algorithms for computing trainable parameters. Numerical experiments with various regression and classification datasets illustrate the proposed method.

A new multi-attention based method for solving the MIL problem (MAMIL), which takes into account the neighboring patches or instances of each analyzed patch in a bag, is proposed. In the method, one of the attention modules takes into account adjacent patches or instances, several attention modules are used to get a diverse feature representation of patches, and one attention module is used to unite different feature representations to provide an accurate classification of each patch (instance) and the whole bag. Due to MAMIL, a combined representation of patches and their neighbors in the form of embeddings of a small dimensionality for simple classification is realized. Moreover, different types of patches are efficiently processed, and a diverse feature representation of patches in a bag by using several attention modules is implemented. A simple approach for explaining the classification predictions of patches is proposed. Numerical experiments with various datasets illustrate the proposed method.

Two-dimensional electron gases (2DEGs) with high mobility, engineered in semiconductor heterostructures host a variety of ordered phases arising from strong correlations, which emerge at sufficiently low temperatures. The 2DEG can be further controlled by surface gates to create quasi-one dimensional systems, with potential spintronic applications. Here we address the long-standing challenge of cooling such electrons to below 1$\,$mK, potentially important for identification of topological phases and spin correlated states. The 2DEG device was immersed in liquid $^3$He, cooled by the nuclear adiabatic demagnetization of copper. The temperature of the 2D electrons was inferred from the electronic noise in a gold wire, connected to the 2DEG by a metallic ohmic contact. With effective screening and filtering, we demonstrate a temperature of 0.9$\,\pm\,$0.1$\,$mK, with scope for significant further improvement. This platform is a key technological step, paving the way to observing new quantum phenomena, and developing new generations of nanoelectronic devices exploiting correlated electron states., Comment: 9 pages, 5 figures plus 5 pages of supplementary information

A new method for local and global explanation of the machine learning black-box model predictions by tabular data is proposed. It is implemented as a system called AFEX (Attention-like Feature EXplanation) and consisting of two main parts. The first part is a set of the one-feature neural subnetworks which aim to get a specific representation for every feature in the form of a basis of shape functions. The subnetworks use shortcut connections with trainable parameters to improve the network performance. The second part of AFEX produces shape functions of features as the weighted sum of the basis shape functions where weights are computed by using an attention-like mechanism. AFEX identifies pairwise interactions between features based on pairwise multiplications of shape functions corresponding to different features. A modification of AFEX with incorporating an additional surrogate model which approximates the black-box model is proposed. AFEX is trained end-to-end on a whole dataset only once such that it does not require to train neural networks again in the explanation stage. Numerical experiments with synthetic and real data illustrate AFEX.

One of the most popular methods of the machine learning prediction explanation is the SHapley Additive exPlanations method (SHAP). An imprecise SHAP as a modification of the original SHAP is proposed for cases when the class probability distributions are imprecise and represented by sets of distributions. The first idea behind the imprecise SHAP is a new approach for computing the marginal contribution of a feature, which fulfils the important efficiency property of Shapley values. The second idea is an attempt to consider a general approach to calculating and reducing interval-valued Shapley values, which is similar to the idea of reachable probability intervals in the imprecise probability theory. A simple special implementation of the general approach in the form of linear optimization problems is proposed, which is based on using the Kolmogorov-Smirnov distance and imprecise contamination models. Numerical examples with synthetic and real data illustrate the imprecise SHAP.

A new modification of the Neural Additive Model (NAM) called SurvNAM and its modifications are proposed to explain predictions of the black-box machine learning survival model. The method is based on applying the original NAM to solving the explanation problem in the framework of survival analysis. The basic idea behind SurvNAM is to train the network by means of a specific expected loss function which takes into account peculiarities of the survival model predictions and is based on approximating the black-box model by the extension of the Cox proportional hazards model which uses the well-known Generalized Additive Model (GAM) in place of the simple linear relationship of covariates. The proposed method SurvNAM allows performing the local and global explanation. A set of examples around the explained example is randomly generated for the local explanation. The global explanation uses the whole training dataset. The proposed modifications of SurvNAM are based on using the Lasso-based regularization for functions from GAM and for a special representation of the GAM functions using their weighted linear and non-linear parts, which is implemented as a shortcut connection. A lot of numerical experiments illustrate the SurvNAM efficiency.