Search

We develop an in-situ index of refraction profile using the transit time of radio signals broadcast from an englacial transmitter to 2-5 km distant radio-frequency receivers, deployed at depths up to 200 m. Maxwell's equations generally admit two ray propagation solutions from a given transmitter, corresponding to a direct path (D) and a refracted path (R); the measured D vs. R (dt(D,R)) timing differences provide constraints on the index of refraction profile near South Pole, where the Askaryan Radio Array (ARA) neutrino observatory is located. We constrain the refractive index profile by simulating D and R ray paths via ray tracing and comparing those to measured dt(D,R) signals. Using previous ice density data as a proxy for n(z), we demonstrate that our data strongly favors a glaciologically-motivated three-phase densification model rather than a single exponential scale height model. Simulations show that the single exponential model overestimates ARA neutrino sensitivity compared to the three-phase model.

Digital experimentation and measurement (DEM) capabilities -- the knowledge and tools necessary to run experiments with digital products, services, or experiences and measure their impact -- are fast becoming part of the standard toolkit of digital/data-driven organisations in guiding business decisions. Many large technology companies report having mature DEM capabilities, and several businesses have been established purely to manage experiments for others. Given the growing evidence that data-driven organisations tend to outperform their non-data-driven counterparts, there has never been a greater need for organisations to build/acquire DEM capabilities to thrive in the current digital era. This thesis presents several novel approaches to statistical and data challenges for organisations building DEM capabilities. We focus on the fundamentals associated with building DEM capabilities, which lead to a richer understanding of the underlying assumptions and thus enable us to develop more appropriate capabilities. We address why one should engage in DEM by quantifying the benefits and risks of acquiring DEM capabilities. This is done using a ranking under lower uncertainty model, enabling one to construct a business case. We also examine what ingredients are necessary to run digital experiments. In addition to clarifying the existing literature around statistical tests, datasets, and methods in experimental design and causal inference, we construct an additional dataset and detailed case studies on applying state-of-the-art methods. Finally, we investigate when a digital experiment design would outperform another, leading to an evaluation framework that compares competing designs' data efficiency., Comment: PhD thesis. Imperial College London. Official library version available on: https://spiral.imperial.ac.uk/handle/10044/1/110307

The Radio Neutrino Observatory - Greenland (RNO-G) seeks discovery of ultra-high energy neutrinos from the cosmos through their interactions in ice. The science program extends beyond particle astrophysics to include radioglaciology and, as we show herein, solar observations, as well. Currently seven of 35 planned radio-receiver stations (24 antennas/station) are operational. These stations are sensitive to impulsive radio signals with frequencies between 80 and 700 MHz and feature a neutrino trigger threshold for recording data close to the thermal floor. RNO-G can also trigger on elevated signals from the Sun, resulting in nanosecond resolution time-domain flare data; such temporal resolution is significantly shorter than from most dedicated solar observatories. In addition to possible RNO-G solar flare polarization measurements, the Sun also represents an extremely useful above-surface calibration source. Using RNO-G data recorded during the summers of 2022 and 2023, we find signal excesses during solar flares reported by the solar-observing Callisto network and also in coincidence with $\sim$2/3 of the brightest excesses recorded by the SWAVES satellite. These observed flares are characterized by significant time-domain impulsivity. Using the known position of the Sun, the flare sample is used to calibrate the RNO-G absolute pointing on the radio signal arrival direction to sub-degree resolution. We thus establish the Sun as a regularly observed astronomical calibration source to provide the accurate absolute pointing required for neutrino astronomy.

The Askaryan Radio Array Station 1 (A1), the first among five autonomous stations deployed for the ARA experiment at the South Pole, is a unique ultra-high energy neutrino (UHEN) detector based on the Askaryan effect that uses Antarctic ice as the detector medium. Its 16 radio antennas (distributed across 4 strings, each with 2 Vertically Polarized (VPol), 2 Horizontally Polarized (HPol) receivers), and 2 strings of transmitting antennas (calibration pulsers, CPs), each with 1 VPol and 1 HPol channel, are deployed at depths less than 100 m within the shallow firn zone of the 2.8 km thick South Pole (SP) ice. We apply different methods to calibrate its Ice Ray Sampler second generation (IRS2) chip for timing offset and ADC-to-Voltage conversion factors using a known continuous wave input signal to the digitizer, and achieve a precision of sub-nanoseconds. We achieve better calibration for odd, compared to even samples, and also find that the HPols under-perform relative to the VPol channels. Our timing calibrated data is subsequently used to calibrate the ADC-to-Voltage conversion as well as precise antenna locations, as a precursor to vertex reconstruction. The calibrated data will then be analyzed for UHEN signals in the final step of data compression. The ability of A1 to scan the firn region of SP ice sheet will contribute greatly towards a 5-station analysis and will inform the design of the planned IceCube Gen-2 radio array., Comment: 10 pages

Glacial ice is used as a target material for the detection of ultra-high energy neutrinos, by measuring the radio signals that are emitted when those neutrinos interact in the ice. Thanks to the large attenuation length at radio frequencies, these signals can be detected over distances of several kilometers. One experiment taking advantage of this is the Radio Neutrino Observatory Greenland (RNO-G), currently under construction at Summit Station, near the apex of the Greenland ice sheet. These experiments require a thorough understanding of the dielectric properties of ice at radio frequencies. Towards this goal, calibration campaigns have been undertaken at Summit, during which we recorded radio reflections off internal layers in the ice sheet. Using data from the nearby GISP2 and GRIP ice cores, we show that these reflectors can be associated with features in the ice conductivity profiles; we use this connection to determine the index of refraction of the bulk ice as n=1.778 +/- 0.006.

Digital technology organizations routinely use online experiments (e.g. A/B tests) to guide their product and business decisions. In e-commerce, we often measure changes to transaction- or item-based business metrics such as Average Basket Value (ABV), Average Basket Size (ABS), and Average Selling Price (ASP); yet it remains a common pitfall to ignore the dependency between the value/size of transactions/items during experiment design and analysis. We present empirical evidence on such dependency, its impact on measurement uncertainty, and practical implications on A/B test outcomes if left unmitigated. By making the evidence available, we hope to drive awareness of the pitfall among experimenters in e-commerce and hence encourage the adoption of established mitigation approaches. We also share lessons learned when incorporating selected mitigation approaches into our experimentation analysis platform currently in production., Comment: To appear in WWW '23 Companion. 5 pages, 4 figures, 2 tables. The experiment code and results on the two publicly available datasets are available on GitHub/Zenodo: https://doi.org/10.5281/zenodo.7659092. This version supersedes a previous working paper with a different title

Quantum error correction can preserve quantum information in the presence of local errors, but correlated errors are fatal. For superconducting qubits, high-energy particle impacts from background radioactivity produce energetic phonons that travel throughout the substrate and create excitations above the superconducting ground state, known as quasiparticles, which can poison all qubits on the chip. We use normal metal reservoirs on the chip back side to downconvert phonons to low energies where they can no longer poison qubits. We introduce a pump-probe scheme involving controlled injection of pair-breaking phonons into the qubit chips. We examine quasiparticle poisoning on chips with and without back-side metallization and demonstrate a reduction in the flux of pair-breaking phonons by over a factor of 20. We use a Ramsey interferometer scheme to simultaneously monitor quasiparticle parity on three qubits for each chip and observe a two-order of magnitude reduction in correlated poisoning due to background radiation., Comment: 24 pages, 17 figures, 5 tables

Online Controlled Experiments (OCE) are the gold standard to measure impact and guide decisions for digital products and services. Despite many methodological advances in this area, the scarcity of public datasets and the lack of a systematic review and categorization hinder its development. We present the first survey and taxonomy for OCE datasets, which highlight the lack of a public dataset to support the design and running of experiments with adaptive stopping, an increasingly popular approach to enable quickly deploying improvements or rolling back degrading changes. We release the first such dataset, containing daily checkpoints of decision metrics from multiple, real experiments run on a global e-commerce platform. The dataset design is guided by a broader discussion on data requirements for common statistical tests used in digital experimentation. We demonstrate how to use the dataset in the adaptive stopping scenario using sequential and Bayesian hypothesis tests and learn the relevant parameters for each approach., Comment: 35th Conference on Neural Information Processing Systems (NeurIPS 2021) Track on Datasets and Benchmarks. 17 pages, 2 figures, 2 tables. Dataset available on Open Science Framework: https://osf.io/64jsb/

Superconducting qubits are resonant absorbers of pair-breaking radiation. The metal pads that form the qubit capacitance support standing wave modes at frequencies of order 100~GHz; these modes are strongly coupled to free-space impedance through their electric dipole moment. While the antenna mode of the 3D transmon qubit is easily seen to be a resonant dipole, other 2D qubit types can be understood as the aperture duals of wire loop antennas or folded dipoles. For typical Josephson junction parameters, the junction provides a reasonable conjugate match to the fundamental antenna mode. We calculate the contribution to quasiparticle poisoning from resonant absorption of blackbody radiation. We extend our analysis to dissipation at the qubit frequency, where radiative losses provide an ultimate limit to qubit energy relaxation time. A clear understanding of the spurious antenna modes of qubits will allow designs that are insensitive to pair-breaking radiation and that display reduced radiative losses at the qubit frequency., Comment: 9 pages, 6 figures, LaTex; figure 1 updated, section 5 clarification added, reference added, bibliography updated; submitted to Phys. Rev. X

The central challenge in building a quantum computer is error correction. Unlike classical bits, which are susceptible to only one type of error, quantum bits ("qubits") are susceptible to two types of error, corresponding to flips of the qubit state about the $X$- and $Z$-directions. While the Heisenberg Uncertainty Principle precludes simultaneous monitoring of $X$- and $Z$-flips on a single qubit, it is possible to encode quantum information in large arrays of entangled qubits that enable accurate monitoring of all errors in the system, provided the error rate is low. Another crucial requirement is that errors cannot be correlated. Here, we characterize a superconducting multiqubit circuit and find that charge fluctuations are highly correlated on a length scale over 600~$\mu$m; moreover, discrete charge jumps are accompanied by a strong transient suppression of qubit energy relaxation time across the millimeter-scale chip. The resulting correlated errors are explained in terms of the charging event and phonon-mediated quasiparticle poisoning associated with absorption of gamma rays and cosmic-ray muons in the qubit substrate. Robust quantum error correction will require the development of mitigation strategies to protect multiqubit arrays from correlated errors due to particle impacts.

We describe an approach to the high-fidelity measurement of a superconducting qubit using an on-chip microwave photon counter. The protocol relies on the transient response of a dispersively coupled measurement resonator to map the state of the qubit to "bright" and "dark" cavity pointer states that are characterized by a large differential photon occupation. Following this mapping, we photodetect the resonator using the Josephson Photomultipler (JPM), which transitions between classically distinguishable flux states when cavity photon occupation exceeds a certain threshold. Our technique provides access to the binary outcome of projective quantum measurement at the millikelvin stage without the need for quantum-limited preamplification and thresholding at room temperature. We achieve raw single-shot measurement fidelity in excess of 98% across multiple samples using this approach in total measurement times under 500 ns. In addition, we show that the backaction and crosstalk associated with our measurement protocol can be mitigated by exploiting the intrinsic damping of the JPM itself., Comment: 15 pages, 13 Figures, 3 Tables

Online Controlled Experiments (OCEs) are the gold standard in evaluating the effectiveness of changes to websites. An important type of OCE evaluates different personalization strategies, which present challenges in low test power and lack of full control in group assignment. We argue that getting the right experiment setup -- the allocation of users to treatment/analysis groups -- should take precedence of post-hoc variance reduction techniques in order to enable the scaling of the number of experiments. We present an evaluation framework that, along with a few simple rule of thumbs, allow experimenters to quickly compare which experiment setup will lead to the highest probability of detecting a treatment effect under their particular circumstance., Comment: Presented in the AdKDD 2020 workshop, in conjunction with The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2020. Main paper: 7 pages, 2 figures, 2 tables, Supplementary document: 6 pages. Fixed minor typos in Eqs. (17) and (18), and Expr. (27a)

Experimentation and Measurement (E&M) capabilities allow organizations to accurately assess the impact of new propositions and to experiment with many variants of existing products. However, until now, the question of measuring the measurer, or valuing the contribution of an E&M capability to organizational success has not been addressed. We tackle this problem by analyzing how, by decreasing estimation uncertainty, E&M platforms allow for better prioritization. We quantify this benefit in terms of expected relative improvement in the performance of all new propositions and provide guidance for how much an E&M capability is worth and when organizations should invest in one., Comment: Accepted into IEEE International Conference on Data Mining (ICDM) 2019. Main paper: 6 pages, 3 figures; Supplementary document: 7 pages, 2 figures. Code available on: https://github.com/liuchbryan/value_of_experimentation

The size of a website's active user base directly affects its value. Thus, it is important to monitor and influence a user's likelihood to return to a site. Essential to this is predicting when a user will return. Current state of the art approaches to solve this problem come in two flavors: (1) Recurrent Neural Network (RNN) based solutions and (2) survival analysis methods. We observe that both techniques are severely limited when applied to this problem. Survival models can only incorporate aggregate representations of users instead of automatically learning a representation directly from a raw time series of user actions. RNNs can automatically learn features, but can not be directly trained with examples of non-returning users who have no target value for their return time. We develop a novel RNN survival model that removes the limitations of the state of the art methods. We demonstrate that this model can successfully be applied to return time prediction on a large e-commerce dataset with a superior ability to discriminate between returning and non-returning users than either method applied in isolation., Comment: Accepted into ECML PKDD 2018; 8 figures and 1 table

The importance of Facebook advertising has risen dramatically in recent years, with the platform accounting for almost 20% of the global online ad spend in 2017. An important consideration in advertising is incrementality: how much of the change in an experimental metric is an advertising campaign responsible for. To measure incrementality, Facebook provide lift studies. As Facebook lift studies differ from standard A/B tests, the online experimentation literature does not describe how to calculate parameters such as power and minimum sample size. Facebook also offer multi-cell lift tests, which can be used to compare campaigns that don't have statistically identical audiences. In this case, there is no literature describing how to measure the significance of the difference in incrementality between cells, or how to estimate the power or minimum sample size. We fill these gaps in the literature by providing the statistical power and required sample size calculation for Facebook lift studies. We then generalise the statistical significance, power, and required sample size calculation to multi-cell lift studies. We represent our results theoretically in terms of the distributions of test metrics and in practical terms relating to the metrics used by practitioners, making all of our code publicly available., Comment: Accepted into 2018 AdKDD & TargetAd Workshop in conjunction with KDD 2018; 6 pages, 4 figures, and 2 tables

Online controlled experiments are the primary tool for measuring the causal impact of product changes in digital businesses. It is increasingly common for digital products and services to interact with customers in a personalised way. Using online controlled experiments to optimise personalised interaction strategies is challenging because the usual assumption of statistically equivalent user groups is violated. Additionally, challenges are introduced by users qualifying for strategies based on dynamic, stochastic attributes. Traditional A/B tests can salvage statistical equivalence by pre-allocating users to control and exposed groups, but this dilutes the experimental metrics and reduces the test power. We present a stacked incrementality test framework that addresses problems with running online experiments for personalised user strategies. We derive bounds that show that our framework is superior to the best simple A/B test given enough users and that this condition is easily met for large scale online experiments. In addition, we provide a test power calculator and describe a selection of pitfalls and lessons learnt from our experience using it., Comment: Not peer-reviewed but retained for historic interest. Removed an erroneous statement on Welch's t-test assumptions in Section 3.2. 9 pages, 7 figures

We perform a detailed analysis of the C++ implementation of the Cluster Affiliation Model for Big Networks (BigClam) on the Stanford Network Analysis Project (SNAP). BigClam is a popular graph mining algorithm that is capable of finding overlapping communities in networks containing millions of nodes. Our analysis shows a key stage of the algorithm - determining if a node belongs to a community - dominates the runtime of the implementation, yet the computation is not parallelized. We show that by parallelizing computations across multiple threads using OpenMP we can speed up the algorithm by 5.3 times when solving large networks for communities, while preserving the integrity of the program and the result., Comment: To appear in 2018 Imperial College Computing Student Workshop (ICCSW'18); 12 pages, 4 figures, and 3 tables

Random forests are among the most popular classification and regression methods used in industrial applications. To be effective, the parameters of random forests must be carefully tuned. This is usually done by choosing values that minimize the prediction error on a held out dataset. We argue that error reduction is only one of several metrics that must be considered when optimizing random forest parameters for commercial applications. We propose a novel metric that captures the stability of random forests predictions, which we argue is key for scenarios that require successive predictions. We motivate the need for multi-criteria optimization by showing that in practical applications, simply choosing the parameters that lead to the lowest error can introduce unnecessary costs and produce predictions that are not stable across independent runs. To optimize this multi-criteria trade-off, we present a new framework that efficiently finds a principled balance between these three considerations using Bayesian optimisation. The pitfalls of optimising forest parameters purely for error reduction are demonstrated using two publicly available real world datasets. We show that our framework leads to parameter settings that are markedly different from the values discovered by error reduction metrics., Comment: To appear in ECML-PKDD 2017

We describe the Customer LifeTime Value (CLTV) prediction system deployed at ASOS.com, a global online fashion retailer. CLTV prediction is an important problem in e-commerce where an accurate estimate of future value allows retailers to effectively allocate marketing spend, identify and nurture high value customers and mitigate exposure to losses. The system at ASOS provides daily estimates of the future value of every customer and is one of the cornerstones of the personalised shopping experience. The state of the art in this domain uses large numbers of handcrafted features and ensemble regressors to forecast value, predict churn and evaluate customer loyalty. Recently, domains including language, vision and speech have shown dramatic advances by replacing handcrafted features with features that are learned automatically from data. We detail the system deployed at ASOS and show that learning feature representations is a promising extension to the state of the art in CLTV modelling. We propose a novel way to generate embeddings of customers, which addresses the issue of the ever changing product catalogue and obtain a significant improvement over an exhaustive set of handcrafted features., Comment: 10 pages, 11 figures

We study charge transport in a monolayer molybdenum disulfide nanoflake over a wide range of carrier density, temperature, and electric bias. We find that the transport is best described by a percolating picture in which the disorder breaks translational invariance, breaking the system up into a series of puddles, rather than previous pictures in which the disorder is treated as homogeneous and uniform. Our work provides insight to a unified picture of charge transport in monolayer molybdenum disulfide nanoflakes and contributes to the development of next-generation molybdenum disulfide based devices., Comment: Evidence for inhomogeneous charge transport in Molybdenum disulfide

The behaviors of a rapid rotating Bose-Einstein condensate under extreme elongation in a 2D anisotropic harmonic plus quartic trap are investigated. Due to the quartic trap, the system remains stable at high rotating velocity, $\Omega\geq \omega_\perp$ ($\omega_\perp$ is the radial harmonic trap frequency), and vortex lattices form even in the absence of the repulsive s-wave interaction ($g$). When $g$ is present, the interplay between $g$ and the quartic trap potential can lead to rich vortex lattice transition states as a function of $\Omega$, to which vortex lattices vanish eventually at some higher $\Omega$., Comment: 5 pages, 5 figures

Traditionally, agent and web service are two separate research areas. We figure that, through agent communication, agent is suitable to coordinate web services. However, there exist agent communication problems due to the lack of uniform, cross-platform vocabulary. Fortunately, ontology defines a vocabulary. We thus propose a new agent communication layer and present the web ontology language (OWL)-based operational ontologies that provides a declarative description. It can be accessed by various engines to facilitate agent communication. Further, in our operational ontologies, we define the mental attitudes of agents that can be shared among other agents. Our architecture enhanced the 3APL agent platform, and it is implemented as an agent communication framework. Finally, we extended the framework to be compatible with the web ontology language for service (OWL-S), and then develop a movie recommendation system with four OWL-S semantic web services on the framework. The benefits of this work are: 1) dynamic web service coordination, 2) ontological reasoning through uniform representation, namely, the declarative description, and 3) easy reuse and extension of both ontology and engine through extending ontology., Comment: 7 pages, International Journal of Computer Science and Information Security

We study theoretically the complex network of forces that is responsible for the static structure and properties of granular materials. We present detailed calculations for a model in which the fluctuations in the force distribution arise because of variations in the contact angles and the constraints imposed by the force balance on each bead of the pile. We compare our results for force distribution function for this model, including exact results for certain contact angle probability distributions, with numerical simulations of force distributions in random sphere packings. This model reproduces many aspects of the force distribution observed both in experiment and in numerical simulations of sphere packings.