Search

This paper releases and analyzes a month-long trace of 85 billion user requests and 11.9 million cold starts from Huawei's serverless cloud platform. Our analysis spans workloads from five data centers. We focus on cold starts and provide a comprehensive examination of the underlying factors influencing the number and duration of cold starts. These factors include trigger types, request synchronicity, runtime languages, and function resource allocations. We investigate components of cold starts, including pod allocation time, code and dependency deployment time, and scheduling delays, and examine their relationships with runtime languages, trigger types, and resource allocation. We introduce pod utility ratio to measure the pod's useful lifetime relative to its cold start time, giving a more complete picture of cold starts, and see that some pods with long cold start times have longer useful lifetimes. Our findings reveal the complexity and multifaceted origins of the number, duration, and characteristics of cold starts, driven by differences in trigger types, runtime languages, and function resource allocations. For example, cold starts in Region 1 take up to 7 seconds, dominated by dependency deployment time and scheduling. In Region 2, cold starts take up to 3 seconds and are dominated by pod allocation time. Based on this, we identify opportunities to reduce the number and duration of cold starts using strategies for multi-region scheduling. Finally, we suggest directions for future research to address these challenges and enhance the performance of serverless cloud platforms. Our datasets and code are available here https://github.com/sir-lab/data-release

It is challenging to scale time series forecasting models such that they forecast accurately for multiple distinct domains and datasets, all with potentially different underlying collection procedures (e.g., sample resolution), patterns (e.g., periodicity), and prediction requirements (e.g., reconstruction vs. forecasting). We call this general task universal forecasting. Existing methods usually assume that input data is regularly sampled, and they forecast to pre-determined horizons, resulting in failure to generalise outside of the scope of their training. We propose the DAM - a neural model that takes randomly sampled histories and outputs an adjustable basis composition as a continuous function of time for forecasting to non-fixed horizons. It involves three key components: (1) a flexible approach for using randomly sampled histories from a long-tail distribution, that enables an efficient global perspective of the underlying temporal dynamics while retaining focus on the recent history; (2) a transformer backbone that is trained on these actively sampled histories to produce, as representational output, (3) the basis coefficients of a continuous function of time. We show that a single univariate DAM, trained on 25 time series datasets, either outperformed or closely matched existing SoTA models at multivariate long-term forecasting across 18 datasets, including 8 held-out for zero-shot transfer, even though these models were trained to specialise for each dataset-horizon combination. This single DAM excels at zero-shot transfer and very-long-term forecasting, performs well at imputation, is interpretable via basis function composition and attention, can be tuned for different inference-cost requirements, is robust to missing and irregularly sampled data {by design}.

Accurately predicting task performance at runtime in a cluster is advantageous for a resource management system to determine whether a task should be migrated due to performance degradation caused by interference. This is beneficial for both cluster operators and service owners. However, deploying performance prediction systems with learning methods requires sophisticated safeguard mechanisms due to the inherent stochastic and black-box natures of these models, such as Deep Neural Networks (DNNs). Vanilla Neural Networks (NNs) can be vulnerable to out-of-distribution data samples that can lead to sub-optimal decisions. To take a step towards a safe learning system in performance prediction, We propose vPALs that leverage well-correlated system metrics, and verification to produce safe performance prediction at runtime, providing an extra layer of safety to integrate learning techniques to cluster resource management systems. Our experiments show that vPALs can outperform vanilla NNs across our benchmark workload., Comment: presented at Deployable AI Workshop at AAAI-2024

This paper releases and analyzes two new Huawei cloud serverless traces. The traces span a period of over 7 months with over 1.4 trillion function invocations combined. The first trace is derived from Huawei's internal workloads and contains detailed per-second statistics for 200 functions running across multiple Huawei cloud data centers. The second trace is a representative workload from Huawei's public FaaS platform. This trace contains per-minute arrival rates for over 5000 functions running in a single Huawei data center. We present the internals of a production FaaS platform by characterizing resource consumption, cold-start times, programming languages used, periodicity, per-second versus per-minute burstiness, correlations, and popularity. Our findings show that there is considerable diversity in how serverless functions behave: requests vary by up to 9 orders of magnitude across functions, with some functions executed over 1 billion times per day; scheduling time, execution time and cold-start distributions vary across 2 to 4 orders of magnitude and have very long tails; and function invocation counts demonstrate strong periodicity for many individual functions and on an aggregate level. Our analysis also highlights the need for further research in estimating resource reservations and time-series prediction to account for the huge diversity in how serverless functions behave. Datasets and code available at https://github.com/sir-lab/data-release

While renormalization group theory is a fully established method to capture equilibrium phase transitions, the applicability of RG theory to universal non-equilibrium behavior remains elusive. Here we address this question by measuring the non-equilibrium dynamics triggered by a quench from superfluid to thermal phase across the Berezinskii-Kosterlitz-Thouless transition in a 2D Bose gas. We quench the system by splitting the 2D gas in two and probe the relaxation dynamics by measuring the phase correlation function and vortex density via matter-wave interferometry. The dynamics occur via a two-step process of rapid phonon thermalization followed by slow dynamic vortex unbinding. We demonstrate universal scaling laws for the algebraic exponents and vortex density, supported by classical-field simulations, and show their agreement with the real-time RG theory.

We probe local phase fluctuations of trapped two-dimensional (2D) Bose gases using matter-wave interferometry. This enables us to measure the phase correlation function, which changes from an algebraic to an exponential decay when the system crosses the Berezinskii-Kosterlitz-Thouless (BKT) transition. We determine the temperature dependence of the BKT exponent $\eta$ and find the critical value $\eta_c = 0.17(3)$ for our trapped system. Furthermore, we measure the local vortex density as a function of the local phase-space density, which shows a scale-invariant behaviour across the transition. Our experimental investigation is supported by Monte Carlo simulations and provides a comprehensive understanding of the BKT transition in a trapped system., Comment: 13 pages, 13 figures

Cloud providers offer end-users various pricing schemes to allow them to tailor VMs to their needs, e.g., a pay-as-you-go billing scheme, called \textit{on-demand}, and a discounted contract scheme, called \textit{reserved instances}. This paper presents a cloud broker which offers users both the flexibility of on-demand instances and some level of discounts found in reserved instances. The broker employs a buy-low-and-sell-high strategy that places user requests into a resource pool of pre-purchased discounted cloud resources. By analysing user request time-series data, the broker takes a risk-oriented approach to dynamically adjust the resource pool. This approach does not require a training process which is useful at processing the large data stream. The broker is evaluated with high-frequency real cloud datasets from Alibaba. The results show that the overall profit of the broker is close to the theoretical optimal scenario where user requests can be perfectly predicted.

Wireless sensor networks (WSN) are characterized by a network of small, battery powered devices, operating remotely with no pre-existing infrastructure. The unique structure of WSN allow for novel approaches to data reduction and energy preservation. This paper presents a modification to the existing Q-routing protocol by providing an alternate action of performing sensor data reduction in place thereby reducing energy consumption, bandwidth usage, and message transmission time. The algorithm is further modified to include an energy factor which increases the cost of forwarding as energy reserves deplete. This encourages the network to conserve energy in favor of network preservation when energy reserves are low. Our experimental results show that this approach can, in periods of high network traffic, simultaneously reduce bandwidth, conserve energy, and maintain low message transition times., Comment: 17 pages, NeTIOT 2020

Investigating out-of-equilibrium dynamics with two-dimensional (2D) systems is of widespread theoretical interest, as these systems are strongly influenced by fluctuations and there exists a superfluid phase transition at a finite temperature. In this work, we realise matter-wave interference for degenerate Bose gases, including the first demonstration of coherent splitting of 2D Bose gases using magnetic trapping potentials. We improve the fringe contrast by imaging only a thin slice of the expanded atom clouds, which will be necessary for subsequent studies on the relaxation of the gas following a quantum quench., Comment: 11 pages, 4 figures

Understanding and predicting the performance of big data applications running in the cloud or on-premises could help minimise the overall cost of operations and provide opportunities in efforts to identify performance bottlenecks. The complexity of the low-level internals of big data frameworks and the ubiquity of application and workload configuration parameters makes it challenging and expensive to come up with comprehensive performance modelling solutions. In this paper, instead of focusing on a wide range of configurable parameters, we studied the low-level internals of the MapReduce communication pattern and used a minimal set of performance drivers to develop a set of phase level parametric models for approximating the execution time of a given application on a given cluster. Models can be used to infer the performance of unseen applications and approximate their performance when an arbitrary dataset is used as input. Our approach is validated by running empirical experiments in two setups. On average the error rate in both setups is plus or minus 10% from the measured values., Comment: 8 pages, 10 figures

Radiofrequency (RF)-dressed potentials are a promising technique for manipulating atomic mixtures, but so far little work has been undertaken to understand the collisions of atoms held within these traps. In this work, we dress a mixture of 85Rb and 87Rb with RF radiation, characterize the inelastic loss that occurs, and demonstrate species-selective manipulations. Our measurements show the loss is caused by two-body 87Rb+85Rb collisions, and we show the inelastic rate coefficient varies with detuning from the RF resonance. We explain our observations using quantum scattering calculations, which give reasonable agreement with the measurements. The calculations consider magnetic fields both perpendicular to the plane of RF polarization and tilted with respect to it. Our findings have important consequences for future experiments that dress mixtures with RF fields., Comment: 12 pages, 7 figures, 1 table

We apply three machine learning strategies to optimize the atomic cooling processes utilized in the production of a Bose-Einstein condensate (BEC). For the first time, we optimize both laser cooling and evaporative cooling mechanisms simultaneously. We present the results of an evolutionary optimization method (Differential Evolution), a method based on non-parametric inference (Gaussian Process regression) and a gradient-based function approximator (Artificial Neural Network). Online optimization is performed using no prior knowledge of the apparatus, and the learner succeeds in creating a BEC from completely randomized initial parameters. Optimizing these cooling processes results in a factor of four increase in BEC atom number compared to our manually-optimized parameters. This automated approach can maintain close-to-optimal performance in long-term operation. Furthermore, we show that machine learning techniques can be used to identify the main sources of instability within the apparatus., Comment: 19 pages, 7 figures, 1 table

In this paper, we extend recent work on the functions that we call Bernstein-gamma to the class of bivariate Bernstein-gamma functions. In the more general bivariate setting, we determine Stirling-type asymptotic bounds which generalise, improve upon and streamline those found for the univariate Bernstein-gamma functions. Then, we demonstrate the importance and power of these results through an application to exponential functionals of L\'evy processes. In more detail, for a subordinator (a non-decreasing L\'evy process) $(X_s)_{s\geq 0}$, we study its \textit{exponential functional}, $\int_0^t e^{-X_s}ds $, evaluated at a finite, deterministic time $t>0$. Our main result here is an explicit infinite convolution formula for the Mellin transform (complex moments) of the exponential functional up to time $t$ which under very minor restrictions is shown to be equivalent to an infinite series. We believe this work can be regarded as a stepping stone towards a more in-depth study of general exponential functionals of L\'evy processes on a finite time horizon.

The Cloud has become integral to most Internet-based applications and user gadgets. This article provides a brief history of the Cloud and presents a researcher's view of the prospects for innovating at the infrastructure, middleware, and application and delivery levels of the already crowded Cloud computing stack., Comment: Accepted to IEEE Computer, 2019

We dress atoms with multiple-radiofrequency fields and investigate the spectrum of transitions driven by an additional probe field. A complete theoretical description of this rich spectrum is presented, in which we find allowed transitions and determine their amplitudes using the resolvent formalism. Experimentally, we observe transitions up to sixth order in the probe field using radiofrequency spectroscopy of Bose-Einstein condensates trapped in single- and multiple-radiofrequency-dressed potentials. We find excellent agreement between theory and experiment, including the prediction and verification of previously unobserved transitions, even in the single-radiofrequency case., Comment: 20 pages, 7 figures

How experiences gained in industry can improve academic research and teaching., Comment: Preprint which appears in Communications of the ACM, August 2018, Vol. 61 No. 9, Pages 31-33

Cloud computing is becoming an almost ubiquitous part of the computing landscape. For many companies today, moving their entire infrastructure and workloads to the cloud reduces complexity, time to deployment, and saves money. Spot Instances, a subset of Amazon's cloud computing infrastructure (EC2), expands on this. They allow a user to bid on spare compute capacity in Amazon's data centres at heavily discounted prices. If demand was ever to increase such that the user's maximum bid is exceeded, their instance is terminated. In this paper, we conduct one of the first detailed analyses of how location affects the overall cost of deployment of a spot instance. We analyse pricing data across all available Amazon Web Services regions for 60 days for a variety of spot instance types. We relate the data we find to the overall AWS region as well as to the Availability Zone within that region. We conclude that location does play a critical role in spot instance pricing and also that pricing differs depending on the granularity of that location - from a more coarse-grained AWS region to a more fine-grained Availability Zone within a region. We relate the pricing differences we find to the price's reliability, confirming whether one can be confident in the prices reported and subsequently, in the ensuing bids one makes. We conclude by showing that it is possible to run workloads on Spot Instances achieving both a very low risk of termination as well as paying very low amounts per hour., Comment: Extended version of CCGrid 2018 paper entitled "Location, Location, Location: Exploring Amazon EC2 Spot Instance Pricing Across Geographical Regions"

We study Markov processes conditioned so that their local time must grow slower than a prescribed function. Building upon recent work on Brownian motion with constrained local time in [5] and [33], we study transience and recurrence for a broad class of Markov processes. In order to understand the distribution of the local time, we determine the distribution of a non-decreasing L\'evy process (the inverse local time) conditioned to remain above a given level which varies in time. We study a time-dependent region, in contrast to previous works in which a process is conditioned to remain in a fixed region (e.g. [21,27]), so we must study boundary crossing probabilities for a family of curves, and thus obtain uniform asymptotics for such a family. Main results include necessary and sufficient conditions for transience or recurrence of the conditioned Markov process. We will explicitly determine the distribution of the inverse local time for the conditioned process, and in the transient case, we explicitly determine the law of the conditioned Markov process. In the recurrent case, we characterise the "entropic repulsion envelope" via necessary and sufficient conditions.

We characterize the anisotropic differential ac-Stark shift for the Dy $626$ nm intercombination transition, induced in a far-detuned $1070$ nm optical dipole trap, and observe the existence of a "magic polarization" for which the polarizabilities of the ground and excited states are equal. From our measurements we extract both the scalar and tensorial components of the dynamic dipole polarizability for the excited state, $\alpha_E^\text{s} = 188 (12)\,\alpha_\text{0}$ and $\alpha_E^\text{t} = 34 (12)\,\alpha_\text{0}$, respectively, where $\alpha_\text{0}$ is the atomic unit for the electric polarizability. We also provide a theoretical model allowing us to predict the excited state polarizability and find qualitative agreement with our observations. Furthermore, we utilize our findings to optimize the efficiency of Doppler cooling of a trapped gas, by controlling the sign and magnitude of the inhomogeneous broadening of the optical transition. The resulting initial gain of the collisional rate allows us, after forced evaporation cooling, to produce a quasi-pure Bose-Einstein condensate of $^{162}$Dy with $3\times 10^4$ atoms., Comment: 6 pages, 4 figures

Cloud computing has been widely adopted due to the flexibility in resource provisioning and on-demand pricing models. Entire clusters of Virtual Machines (VMs) can be dynamically provisioned to meet the computational demands of users. However, from a user's perspective, it is still challenging to utilise cloud resources efficiently. This is because an overwhelmingly wide variety of resource types with different prices and significant performance variations are available. This paper presents a survey and taxonomy of existing research in optimising the execution of Bag-of-Task applications on cloud resources. A BoT application consists of multiple independent tasks, each of which can be executed by a VM in any order; these applications are widely used by both the scientific communities and commercial organisations. The objectives of this survey are as follows: (i) to provide the reader with a concise understanding of existing research on optimising the execution of BoT applications on the cloud, (ii) to define a taxonomy that categorises current frameworks to compare and contrast them, and (iii) to present current trends and future research directions in the area., Comment: Accepted to Future Generation Computer Systems, 23 November 2017

The recent boom of big data, coupled with the challenges of its processing and storage gave rise to the development of distributed data processing and storage paradigms like MapReduce, Spark, and NoSQL databases. With the advent of cloud computing, processing and storing such massive datasets on clusters of machines is now feasible with ease. However, there are limited tools and approaches, which users can rely on to gauge and comprehend the performance of their big data applications deployed locally on clusters, or in the cloud. Researchers have started exploring this area by providing benchmarking suites suitable for big data applications. However, many of these tools are fragmented, complex to deploy and manage, and do not provide transparency with respect to the monetary cost of benchmarking an application. In this paper, we present Plug And Play Bench, an infrastructure aware abstraction built to integrate and simplify the deployment of big data benchmarking tools on clusters of machines. PAPB automates the tedious process of installing, configuring and executing common big data benchmark workloads by containerising the tools and settings based on the underlying cluster deployment framework. Our proof of concept implementation utilises HiBench as the benchmark suite, HDP as the cluster deployment framework and Azure as the cloud platform. The paper further illustrates the inclusion of cost metrics based on the underlying Microsoft Azure cloud platform., Comment: 8 pages, Published as a workshop paper in 2017 IEEE International Conference on Big Data in Boston Dec 11 - 14

This paper presents MAMoC, a framework which brings together a diverse range of infrastructure types including mobile devices, cloudlets, and remote cloud resources under one unified API. MAMoC allows mobile applications to leverage the power of multiple offloading destinations. MAMoC's intelligent offloading decision engine adapts to the contextual changes in this heterogeneous environment, in order to reduce the overall runtime for both single-site and multi-site offloading scenarios. MAMoC is evaluated through a set of offloading experiments, which evaluate the performance of our offloading decision engine. The results show that offloading computation using our framework can reduce the overall task completion time for both single-site and multi-site offloading scenarios., Comment: 8 pages, CloudCom 2017 Conference

Current HPC platforms do not provide the infrastructure, interfaces and conceptual models to collect, store, analyze, and access such data. Today, applications depend on application and platform specific techniques for collecting telemetry data; introducing significant development overheads that inhibit portability and mobility. The development and adoption of adaptive, context-aware strategies is thereby impaired. To facilitate 2nd generation applications, more efficient application development, and swift adoption of adaptive applications in production, a comprehensive framework for telemetry data management must be provided by future HPC systems and services. We introduce a conceptual model and a software framework to collect, store, analyze, and exploit streams of telemetry data generated by HPC systems and their applications. We show how this framework can be integrated with HPC platform architectures and how it enables common application execution strategies.

This work looks at the box-counting dimension of sets related to subordinators (non-decreasing L\'evy processes). It was recently shown in [Savov, 2014] that almost surely $\lim_{\delta\to0}U(\delta)N(t,\delta) = t$, where $N(t,\delta)$ is the minimal number of boxes of size at most $\delta$ needed to cover a subordinator's range up to time $t$, and $U(\delta)$ is the subordinator's renewal function. Our main result is a central limit theorem (CLT) for $N(t,\delta)$, complementing and refining work in [Savov, 2014]. Box-counting dimension is defined in terms of $N(t,\delta)$, but for subordinators we prove that it can also be defined using a new process obtained by shortening the original subordinator's jumps of size greater than $\delta$. This new process can be manipulated with remarkable ease in comparison to $N(t,\delta)$, and allows better understanding of the box-counting dimension of a subordinator's range in terms of its L\'evy measure, improving upon [Corollary 1, Savov, 2014]. Further, we shall prove corresponding CLT and almost sure convergence results for the new process., Comment: 20 pages

We present the first experimental demonstration of a multiple-radiofrequency dressed potential for the configurable magnetic confinement of ultracold atoms. We load cold $^{87}$Rb atoms into a double well potential with an adjustable barrier height, formed by three radiofrequencies applied to atoms in a static quadrupole magnetic field. Our multiple-radiofrequency approach gives precise control over the double well characteristics, including the depth of individual wells and the height of the barrier, and enables reliable transfer of atoms between the available trapping geometries. We have characterised the multiple-radiofrequency dressed system using radiofrequency spectroscopy, finding good agreement with the eigenvalues numerically calculated using Floquet theory. This method creates trapping potentials that can be reconfigured by changing the amplitudes, polarizations and frequencies of the applied dressing fields, and easily extended with additional dressing frequencies., Comment: 16 pages, 6 figures

A new class of Second generation high-performance computing applications with heterogeneous, dynamic and data-intensive properties have an extended set of requirements, which cover application deployment, resource allocation, -control, and I/O scheduling. These requirements are not met by the current production HPC platform models and policies. This results in a loss of opportunity, productivity and innovation for new computational methods and tools. It also decreases effective system utilization for platform providers due to unsupervised workarounds and rogue resource management strategies implemented in application space. In this paper we critically discuss the dominant HPC platform model and describe the challenges it creates for second generation applications because of its asymmetric resource view, interfaces and software deployment policies. We present an extended, more symmetric and application-centric platform model that adds decentralized deployment, introspection, bidirectional control and information flow and more comprehensive resource scheduling. We describe cHPC: an early prototype of a non-disruptive implementation based on Linux Containers (LXC). It can operate alongside existing batch queuing systems and exposes a symmetric platform API without interfering with existing applications and usage modes. We see our approach as a viable, incremental next step in HPC platform evolution that benefits applications and platform providers alike. To demonstrate this further, we layout out a roadmap for future research and experimental evaluation.

Using two-photon Bragg spectroscopy, we study the energy of particle-like excitations in a strongly interacting homogeneous Bose-Einstein condensate, and observe dramatic deviations from Bogoliubov theory. In particular, at large scattering length $a$ the shift of the excitation resonance from the free-particle energy changes sign from positive to negative. For an excitation with wavenumber $q$, this sign change occurs at $a \approx 4/(\pi q)$, in agreement with the Feynman energy relation and the static structure factor expressed in terms of the two-body contact. For $a \gtrsim 3/q$ we also see a breakdown of this theory, and better agreement with calculations based on the Wilson operator product expansion. Neither theory explains our observations across all interaction regimes, inviting further theoretical efforts., Comment: 4 pages, 4 figures

How can applications be deployed on the cloud to achieve maximum performance? This question is challenging to address with the availability of a wide variety of cloud Virtual Machines (VMs) with different performance capabilities. The research reported in this paper addresses the above question by proposing a six step benchmarking methodology in which a user provides a set of weights that indicate how important memory, local communication, computation and storage related operations are to an application. The user can either provide a set of four abstract weights or eight fine grain weights based on the knowledge of the application. The weights along with benchmarking data collected from the cloud are used to generate a set of two rankings - one based only on the performance of the VMs and the other takes both performance and costs into account. The rankings are validated on three case study applications using two validation techniques. The case studies on a set of experimental VMs highlight that maximum performance can be achieved by the three top ranked VMs and maximum performance in a cost-effective manner is achieved by at least one of the top three ranked VMs produced by the methodology., Comment: 14 pages, accepted to the IEEE Transactions on Cloud Computing on 31 July 2016

This paper presents the first framework for integrating procedural knowledge, or "know-how", into the Linked Data Cloud. Know-how available on the Web, such as step-by-step instructions, is largely unstructured and isolated from other sources of online knowledge. To overcome these limitations, we propose extending to procedural knowledge the benefits that Linked Data has already brought to representing, retrieving and reusing declarative knowledge. We describe a framework for representing generic know-how as Linked Data and for automatically acquiring this representation from existing resources on the Web. This system also allows the automatic generation of links between different know-how resources, and between those resources and other online knowledge bases, such as DBpedia. We discuss the results of applying this framework to a real-world scenario and we show how it outperforms existing manual community-driven integration efforts., Comment: The 19th International Conference on Knowledge Engineering and Knowledge Management (EKAW 2014), 24-28 November 2014, Link\"oping, Sweden

Existing benchmarking methods are time consuming processes as they typically benchmark the entire Virtual Machine (VM) in order to generate accurate performance data, making them less suitable for real-time analytics. The research in this paper is aimed to surmount the above challenge by presenting DocLite - Docker Container-based Lightweight benchmarking tool. DocLite explores lightweight cloud benchmarking methods for rapidly executing benchmarks in near real-time. DocLite is built on the Docker container technology, which allows a user-defined memory size and number of CPU cores of the VM to be benchmarked. The tool incorporates two benchmarking methods - the first referred to as the native method employs containers to benchmark a small portion of the VM and generate performance ranks, and the second uses historic benchmark data along with the native method as a hybrid to generate VM ranks. The proposed methods are evaluated on three use-cases and are observed to be up to 91 times faster than benchmarking the entire VM. In both methods, small containers provide the same quality of rankings as a large container. The native method generates ranks with over 90% and 86% accuracy for sequential and parallel execution of an application compared against benchmarking the whole VM. The hybrid method did not improve the quality of the rankings significantly., Comment: 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), 2016, Cartagena, Colombia. arXiv admin note: substantial text overlap with arXiv:1601.03872

The increasing amount of available Linked Data resources is laying the foundations for more advanced Semantic Web applications. One of their main limitations, however, remains the general low level of data quality. In this paper we focus on a measure of quality which is negatively affected by the increase of the available resources. We propose a measure of semantic richness of Linked Data concepts and we demonstrate our hypothesis that the more a concept is reused, the less semantically rich it becomes. This is a significant scalability issue, as one of the core aspects of Linked Data is the propagation of semantic information on the Web by reusing common terms. We prove our hypothesis with respect to our measure of semantic richness and we validate our model empirically. Finally, we suggest possible future directions to address this scalability problem.

With the availability of a wide range of cloud Virtual Machines (VMs) it is difficult to determine which VMs can maximise the performance of an application. Benchmarking is commonly used to this end for capturing the performance of VMs. Most cloud benchmarking techniques are typically heavyweight - time consuming processes which have to benchmark the entire VM in order to obtain accurate benchmark data. Such benchmarks cannot be used in real-time on the cloud and incur extra costs even before an application is deployed. In this paper, we present lightweight cloud benchmarking techniques that execute quickly and can be used in near real-time on the cloud. The exploration of lightweight benchmarking techniques are facilitated by the development of DocLite - Docker Container-based Lightweight Benchmarking. DocLite is built on the Docker container technology which allows a user-defined portion (such as memory size and the number of CPU cores) of the VM to be benchmarked. DocLite operates in two modes, in the first mode, containers are used to benchmark a small portion of the VM to generate performance ranks. In the second mode, historic benchmark data is used along with the first mode as a hybrid to generate VM ranks. The generated ranks are evaluated against three scientific high-performance computing applications. The proposed techniques are up to 91 times faster than a heavyweight technique which benchmarks the entire VM. It is observed that the first mode can generate ranks with over 90% and 86% accuracy for sequential and parallel execution of an application. The hybrid mode improves the correlation slightly but the first mode is sufficient for benchmarking cloud VMs., Comment: Accepted to the IEEE International Conference on Cloud Engineering (IEEE IC2E), Berlin, Germany, 2016 - 10 pages

Scheduling Bag-of-Tasks (BoT) applications on the cloud can be more challenging than grid and cluster environ- ments. This is because a user may have a budgetary constraint or a deadline for executing the BoT application in order to keep the overall execution costs low. The research in this paper is motivated to investigate task scheduling on the cloud, given two hard constraints based on a user-defined budget and a deadline. A heuristic algorithm is proposed and implemented to satisfy the hard constraints for executing the BoT application in a cost effective manner. The proposed algorithm is evaluated using four scenarios that are based on the trade-off between performance and the cost of using different cloud resource types. The experimental evaluation confirms the feasibility of the algorithm in satisfying the constraints. The key observation is that multiple resource types can be a better alternative to using a single type of resource., Comment: Visionary Track of the IEEE 11th World Congress on Services (IEEE SERVICES 2015)

Optimising the execution of Bag-of-Tasks (BoT) applications on the cloud is a hard problem due to the trade- offs between performance and monetary cost. The problem can be further complicated when multiple BoT applications need to be executed. In this paper, we propose and implement a heuristic algorithm that schedules tasks of multiple applications onto different cloud virtual machines in order to maximise performance while satisfying a given budget constraint. Current approaches are limited in task scheduling since they place a limit on the number of cloud resources that can be employed by the applications. However, in the proposed algorithm there are no such limits, and in comparison with other approaches, the algorithm on average achieves an improved performance of 10%. The experimental results also highlight that the algorithm yields consistent performance even with low budget constraints which cannot be achieved by competing approaches., Comment: 8th IEEE International Conference on Cloud Computing (CLOUD 2015)

Bag-of-Distributed-Tasks (BoDT) application is the collection of identical and independent tasks each of which requires a piece of input data located around the world. As a result, Cloud computing offers an ef- fective way to execute BoT application as it not only consists of multiple geographically distributed data centres but also allows a user to pay for what she actually uses only. In this paper, BoDT on the Cloud using virtually unlimited cloud resources. A heuristic algorithm is proposed to find an execution plan that takes budget constraints into account. Compared with other approaches, with the same given budget, our algorithm is able to reduce the overall execution time up to 50%.

A Cloud Services Brokerage (CSB) acts as an intermediary between cloud service providers (e.g., Amazon and Google) and cloud service end users, providing a number of value adding services. CSBs as a research topic are in there infancy. The goal of this paper is to provide a concise survey of existing CSB technologies in a variety of areas and highlight a roadmap, which details five future opportunities for research., Comment: Paper published in the 8th IEEE International Conference on Cloud Computing (CLOUD 2015)

This paper presents the first complete, integrated and end-to-end solution for ad hoc cloud computing environments. Ad hoc clouds harvest resources from existing sporadically available, non-exclusive (i.e. primarily used for some other purpose) and unreliable infrastructures. In this paper we discuss the problems ad hoc cloud computing solves and outline our architecture which is based on BOINC., Comment: Published in IEEE Cloud 2015 named "Ad hoc Cloud Computing"

Autonomously detecting and recovering from faults is one approach for reducing the operational complexity and costs associated with managing computing environments. We present a novel methodology for autonomously generating investigation leads that help identify systems faults, and extends our previous work in this area by leveraging Restricted Boltzmann Machines (RBMs) and contrastive divergence learning to analyse changes in historical feature data. This allows us to heuristically identify the root cause of a fault, and demonstrate an improvement to the state of the art by showing feature data can be predicted heuristically beyond a single instance to include entire sequences of information., Comment: Published and presented in the 11th IEEE International Conference and Workshops on Engineering of Autonomic and Autonomous Systems (EASe 2014)

Applications employed in the financial services industry to capture and estimate a variety of risk metrics are underpinned by stochastic simulations which are data, memory and computationally intensive. Many of these simulations are routinely performed on production-based computing systems. Ad hoc simulations in addition to routine simulations are required to obtain up-to-date views of risk metrics. Such simulations are currently not performed as they cannot be accommodated on production clusters, which are typically over committed resources. Scalable, on-demand and pay-as-you go Virtual Machines (VMs) offered by the cloud are a potential platform to satisfy the data, memory and computational constraints of the simulation. However, "Are clouds ready to accelerate ad hoc financial simulations?" The research reported in this paper aims to experimentally verify this question by developing and deploying an important financial simulation, referred to as 'Aggregate Risk Analysis' on the cloud. Parallel techniques to improve efficiency and performance of the simulations are explored. Challenges such as accommodating large input data on limited memory VMs and rapidly processing data for real-time use are surmounted. The key result of this investigation is that Aggregate Risk Analysis can be accommodated on cloud VMs. Acceleration of up to 24x using multiple hardware accelerators over the implementation on a single accelerator, 6x over a multiple core implementation and approximately 60x over a baseline implementation was achieved on the cloud. However, computational time is wasted for every dollar spent on the cloud due to poor acceleration over multiple virtual cores. Interestingly, private VMs can offer better performance than public VMs on comparable underlying hardware., Comment: Best paper nominee at the International Symposium on Big Data Computing (BDC 2014) in conjunction with IEEE/ACM Utility and Cloud Computing (UCC), 2014 London, UK

This paper argues that there are three fundamental challenges that need to be overcome in order to foster the adoption of big data technologies in non-computer science related disciplines: addressing issues of accessibility of such technologies for non-computer scientists, supporting the ad hoc exploration of large data sets with minimal effort and the availability of lightweight web-based frameworks for quick and easy analytics. In this paper, we address the above three challenges through the development of 'BigExcel', a three tier web-based framework for exploring big data to facilitate the management of user interactions with large data sets, the construction of queries to explore the data set and the management of the infrastructure. The feasibility of BigExcel is demonstrated through two Yahoo Sandbox datasets. The first dataset is the Yahoo Buzz Score data set we use for quantitatively predicting trending technologies and the second is the Yahoo n-gram corpus we use for qualitatively inferring the coverage of important events. A demonstration of the BigExcel framework and source code is available at http://bigdata.cs.st-andrews.ac.uk/projects/bigexcel-exploring-big-data-for-social-sciences/., Comment: 8 pages

How can applications be deployed on the cloud to achieve maximum performance? This question has become significant and challenging with the availability of a wide variety of Virtual Machines (VMs) with different performance capabilities in the cloud. The above question is addressed by proposing a six step benchmarking methodology in which a user provides a set of four weights that indicate how important each of the following groups: memory, processor, computation and storage are to the application that needs to be executed on the cloud. The weights along with cloud benchmarking data are used to generate a ranking of VMs that can maximise performance of the application. The rankings are validated through an empirical analysis using two case study applications; the first is a financial risk application and the second is a molecular dynamics simulation, which are both representative of workloads that can benefit from execution on the cloud. Both case studies validate the feasibility of the methodology and highlight that maximum performance can be achieved on the cloud by selecting the top ranked VMs produced by the methodology., Comment: 6 pages, 6th IEEE International Conference on Cloud Computing Technology and Science (IEEE CloudCom) 2014, Singapore

Bag of Distributed Tasks (BoDT) can benefit from decentralised execution on the Cloud. However, there is a trade-off between the performance that can be achieved by employing a large number of Cloud VMs for the tasks and the monetary constraints that are often placed by a user. The research reported in this paper is motivated towards investigating this trade-off so that an optimal plan for deploying BoDT applications on the cloud can be generated. A heuristic algorithm, which considers the user's preference of performance and cost is proposed and implemented. The feasibility of the algorithm is demonstrated by generating execution plans for a sample application. The key result is that the algorithm generates optimal execution plans for the application over 91\% of the time.

When orchestrating Web service workflows, the geographical placement of the orchestration engine(s) can greatly affect workflow performance. Data may have to be transferred across long geographical distances, which in turn increases execution time and degrades the overall performance of a workflow. In this paper, we present a framework that, given a DAG-based workflow specification, computes the op- timal Amazon EC2 cloud regions to deploy the orchestration engines and execute a workflow. The framework incorporates a constraint model that solves the workflow deployment problem, which is generated using an automated constraint modelling system. The feasibility of the framework is evaluated by executing different sample workflows representative of sci- entific workloads. The experimental results indicate that the framework reduces the workflow execution time and provides a speed up of 1.3x-2.5x over centralised approaches.

This paper proposes a novel framework for representing community know-how on the Semantic Web. Procedural knowledge generated by web communities typically takes the form of natural language instructions or videos and is largely unstructured. The absence of semantic structure impedes the deployment of many useful applications, in particular the ability to discover and integrate know-how automatically. We discuss the characteristics of community know-how and argue that existing knowledge representation frameworks fail to represent it adequately. We present a novel framework for representing the semantic structure of community know-how and demonstrate the feasibility of our approach by providing a concrete implementation which includes a method for automatically acquiring procedural knowledge for real-world tasks., Comment: 6th International Workshop on Web Intelligence & Communities (WIC14), Proceedings of the companion publication of the 23rd International Conference on World Wide Web (WWW 2014)

When orchestrating highly distributed and data-intensive Web service workflows the geographical placement of the orchestration engine can greatly affect the overall performance of a workflow. We present CloudForecast: a Web service framework and analysis tool which, given a workflow specification, computes the optimal Amazon EC2 Cloud region to automatically deploy the orchestration engine and execute the workflow. We use geographical distance of the workflow, network latency and HTTP round-trip time between Amazon Cloud regions and the workflow nodes to find a ranking of Cloud regions. This overall ranking predicts where the workflow orchestration engine should be deployed in order to reduce overall execution time. Our experimental results show that our proposed optimisation strategy, depending on the particular workflow, can speed up execution time on average by 82.25% compared to local execution., Comment: Extended Abstract for the ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC 2013) Poster Track