3,872 results on '"Software Framework"'
Search Results
2. Flexible IoT Agriculture Systems for Irrigation Control Based on Software Services.
- Author
-
Palomar-Cosín E and García-Valls M
- Subjects
- Humans, Agriculture, Computer Systems, Technology, Software, Computers
- Abstract
IoT technology applied to agriculture has produced a number of contributions in the recent years. Such solutions are, most of the time, fully tailored to a particular functional target and focus extensively on sensor-hardware development and customization. As a result, software-centered solutions for IoT system development are infrequent. This is not suitable, as the software is the bottleneck in modern computer systems, being the main source of performance loss, errors, and even cyber attacks. This paper takes a software-centric perspective to model and design IoT systems in a flexible manner. We contribute a software framework that supports the design of the IoT systems' software based on software services in a client-server model with REST interactions; and it is exemplified on the domain of efficient irrigation in agriculture. We decompose the services' design into the set of constituent functions and operations both at client and server sides. As a result, we provide a simple and novel view on the design of IoT systems in agriculture from a sofware perspective: we contribute simple design structure based on the identification of the front-end software services, their internal software functions and operations, and their interconnections as software services. We have implemented the software framework on an IoT irrigation use case that monitors the conditions of the field and processes the sampled data, detecting alarms when needed. We demonstrate that the temporal overhead of our solution is bounded and suitable for the target domain, reaching a response time of roughly 11 s for bursts of 3000 requests.
- Published
- 2022
- Full Text
- View/download PDF
3. Deep-Framework: A Distributed, Scalable, and Edge-Oriented Framework for Real-Time Analysis of Video Streams.
- Author
-
Sassu A, Saenz-Cogollo JF, and Agelli M
- Subjects
- Humans, Algorithms, Software
- Abstract
Edge computing is the best approach for meeting the exponential demand and the real-time requirements of many video analytics applications. Since most of the recent advances regarding the extraction of information from images and video rely on computation heavy deep learning algorithms, there is a growing need for solutions that allow the deployment and use of new models on scalable and flexible edge architectures. In this work, we present Deep-Framework, a novel open source framework for developing edge-oriented real-time video analytics applications based on deep learning. Deep-Framework has a scalable multi-stream architecture based on Docker and abstracts away from the user the complexity of cluster configuration, orchestration of services, and GPU resources allocation. It provides Python interfaces for integrating deep learning models developed with the most popular frameworks and also provides high-level APIs based on standard HTTP and WebRTC interfaces for consuming the extracted video data on clients running on browsers or any other web-based platform.
- Published
- 2021
- Full Text
- View/download PDF
4. A software framework for real-time multi-modal detection of microsleeps.
- Author
-
Knopp SJ, Bones PJ, Weddell SJ, and Jones RD
- Subjects
- Algorithms, Computer Systems, Electroencephalography, Humans, Signal Processing, Computer-Assisted, Task Performance and Analysis, User-Computer Interface, Sleep physiology, Software
- Abstract
A software framework is described which was designed to process EEG, video of one eye, and head movement in real time, towards achieving early detection of microsleeps for prevention of fatal accidents, particularly in transport sectors. The framework is based around a pipeline structure with user-replaceable signal processing modules. This structure can encapsulate a wide variety of feature extraction and classification techniques and can be applied to detecting a variety of aspects of cognitive state. Users of the framework can implement signal processing plugins in C++ or Python. The framework also provides a graphical user interface and the ability to save and load data to and from arbitrary file formats. Two small studies are reported which demonstrate the capabilities of the framework in typical applications: monitoring eye closure and detecting simulated microsleeps. While specifically designed for microsleep detection/prediction, the software framework can be just as appropriately applied to (i) other measures of cognitive state and (ii) development of biomedical instruments for multi-modal real-time physiological monitoring and event detection in intensive care, anaesthesiology, cardiology, neurosurgery, etc. The software framework has been made freely available for researchers to use and modify under an open source licence.
- Published
- 2017
- Full Text
- View/download PDF
5. A Health Surveillance Software Framework to deliver information on preventive healthcare strategies.
- Author
-
Macedo AA, Pollettini JT, Baranauskas JA, and Chaves JC
- Subjects
- Child, Chronic Disease, Diagnosis, Humans, Medical Records, Computer Systems, Health Status, Population Surveillance, Software
- Abstract
A software framework can reduce costs related to the development of an application because it allows developers to reuse both design and code. Recently, companies and research groups have announced that they have been employing health software frameworks. This paper presents the design, proof-of-concept implementations and experimentation of the Health Surveillance Software Framework (HSSF). The HSSF is a framework that tackles the demand for the recommendation of surveillance information aiming at supporting preventive healthcare strategies. Examples of such strategies are the automatic recommendation of surveillance levels to patients in need of healthcare and the automatic recommendation of scientific literature that elucidates epigenetic problems related to patients. HSSF was created from two systems we developed in our previous work on health surveillance systems: the Automatic-SL and CISS systems. The Automatic-SL system aims to assist healthcare professionals in making decisions and in identifying children with developmental problems. The CISS service associates genetic and epigenetic risk factors related to chronic diseases with patient's clinical records. Towards evaluating the HSSF framework, two new systems, CISS+ and CISS-SW, were created by means of abstractions and instantiations of the framework (design and code). We show that HSSF supported the development of the two new systems given that they both recommend scientific papers using medical records as queries even though they exploit different computational technologies. In an experiment using simulated patients' medical records, we show that CISS, CISS+, and CISS-SW systems recommended more closely related and somewhat related documents than Google, Google Scholar and PubMed. Considering recall and precision measures, CISS+ surpasses CISS-SW in terms of precision., (Copyright © 2016 Elsevier Inc. All rights reserved.)
- Published
- 2016
- Full Text
- View/download PDF
6. The GosipGUI Framework for Control and Benchmarking of Readout Electronics Front-Ends.
- Author
-
Adamczewski-Musch, Jorn and Kurz, Nikolaus
- Subjects
- *
OPTICAL fiber communication , *ACQUISITION of data , *MASS production , *MODULAR design , *GRAPHICAL user interfaces , *SOFTWARE frameworks , *SILICON detectors - Abstract
The gigabit optical serial interface protocol (GOSIP) provides communication via optical fibers between multiple kinds of front-end electronics and the KINPEX PCIe receiver board located in the readout host PC. In recent years, a stack of device driver software has been developed to utilize this hardware for several scenarios of data acquisition (DAQ). On top of this driver foundation, several graphical user interfaces (GUIs) have been created. These GUIs are based on the Qt graphics libraries and are designed in a modular way: All common functionalities, like generic I/O with the front-ends, handling of configuration files, and window settings are treated by a framework class GosipGUI. In the Qt workspace of such GosipGUI frame, specific subclasses may implement additional windows dedicated to operating different GOSIP front-end modules. For each kind of front-end, the GUIs allow to monitor specific register contents, set up the working configuration, and interactively change parameters like sampling thresholds during DAQ. The latter is extremely useful when qualifying and tuning the front-ends in the electronics lab or detector cave. Moreover, some of these GosipGUI implementations have been equipped with features for mostly automatic testing of ASICs in a prototype mass production. [ABSTRACT FROM AUTHOR]
- Published
- 2021
- Full Text
- View/download PDF
7. PSO-X: A Component-Based Framework for the Automatic Design of Particle Swarm Optimization Algorithms
- Author
-
Marco Dorigo, Christian Leonardo Camacho Villalón, and Thomas Stützle
- Subjects
Computer science ,MathematicsofComputing_NUMERICALANALYSIS ,Process (computing) ,Particle swarm optimization ,Ranging ,Object (computer science) ,computer.software_genre ,Theoretical Computer Science ,Variety (cybernetics) ,Task (project management) ,Software framework ,Computational Theory and Mathematics ,Component (UML) ,Algorithm ,computer ,Software - Abstract
The particle swarm optimization (PSO) algorithm has been the object of many studies and modifications for more than twentyfive years. Ranging from small refinements to the incorporation of sophisticated novel ideas, the majority of modifications proposed to this algorithm have been the result of a manual process in which developers try new designs based on their own knowledge and expertise. However, manually introducing changes is very time consuming and makes the systematic exploration of all the possible algorithm configurations a difficult process. In this paper, we propose to use automatic design to overcome the limitations of having to manually find performing PSO algorithms. We develop a flexible software framework for PSO, called PSO-X, which is specifically designed to integrate the use of automatic configuration tools into the process of generating PSO algorithms. Our framework embodies a large number of algorithm components developed over more than twentyfive years of research that have allowed PSO to deal with a large variety of problems, and uses irace, a state-of-the-art configuration tool, to automatize the task of selecting and configuring PSO algorithms starting from these components. We show that irace is capable of finding high performing instances of PSO algorithms never proposed before.
- Published
- 2022
8. Enabling Pulse-Level Programming, Compilation, and Execution in XACC
- Author
-
Thien Nguyen and Alexander McCaskey
- Subjects
Digital electronics ,Quantum Physics ,Quantum programming ,business.industry ,Computer science ,FOS: Physical sciences ,Cloud computing ,Computational Physics (physics.comp-ph) ,computer.software_genre ,Theoretical Computer Science ,Software framework ,Computational Theory and Mathematics ,Computer engineering ,Hardware and Architecture ,Scalability ,Use case ,Quantum Physics (quant-ph) ,Quantum information science ,business ,Physics - Computational Physics ,computer ,Software ,Quantum computer ,computer.programming_language - Abstract
Noisy gate-model quantum processing units (QPUs) are currently available from vendors over the cloud, and digital quantum programming approaches exist to run low-depth circuits on physical hardware. These digital representations are ultimately lowered to pulse-level instructions by vendor quantum control systems to affect unitary evolution representative of the submitted digital circuit. Vendors are beginning to open this pulse-level control system to the public via specified interfaces. Robust programming methodologies, software frameworks, and backend simulation technologies for this analog model of quantum computation will prove critical to advancing pulse-level control research and development. Prototypical use cases for this include error mitigation, optimal pulse control, and physics-inspired pulse construction. Here we present an extension to the XACC quantum-classical software framework that enables pulse-level programming for superconducting, gate-model quantum computers, and a novel, general, and extensible pulse-level simulation backend for XACC that scales on classical compute clusters via MPI. Our work enables custom backend Hamiltonian definitions and gate-level compilation to available pulses with a focus on performance and scalability. We end with a demonstration of this capability, and show how to use XACC for pertinent pulse-level programming tasks.
- Published
- 2022
9. Identifying Degree and Sources of Non-Determinism in MPI Applications Via Graph Kernels
- Author
-
Michela Taufer, Sanjukta Bhowmick, Nigel Tan, and Dylan Chapp
- Subjects
Graph kernel ,Degree (graph theory) ,Computer science ,business.industry ,media_common.quotation_subject ,Distributed computing ,computer.software_genre ,Software framework ,Kernel (linear algebra) ,Software ,Computational Theory and Mathematics ,Software bug ,Debugging ,Hardware and Architecture ,Signal Processing ,Leverage (statistics) ,business ,computer ,media_common - Abstract
As the scientific community prepares to deploy an increasingly complex and diverse set of applications on exascale platforms, the need to assess reproducibility of simulations and identify the root causes of reproducibility failures increases correspondingly. One of the greatest challenges facing reproducibility issues at exascale is the inherent non-determinism at the level of inter-process communication. The use of non-deterministic communication constructs is necessary to boost performance, but communication non-determinism can also hamper software correctness and result reproducibility. To address this challenge, we propose a software framework for identifying the percentage and sources of communication non-determinism. We model parallel executions as directed graphs and leverage graph kernels to characterize run-to-run variations in inter-process communication. We demonstrate the effectiveness of graph kernel similarity as a proxy for non-determinism, by showing that these kernels can quantify the type and degree of non-determinism present in communication patterns. To demonstrate our framework’s ability to link and quantify runtime non-determinism to root sources, demonstrate with present for an adaptive mesh refinement application, where our framework automatically quantifies the impact of function calls on non-determinism, and a Monte Carlo application, where our framework automatically quantifies the impact of parameter configurations on non-determinism.
- Published
- 2021
10. A Cluster of FPAAs to Recognize Images Using Neural Networks
- Author
-
Daniel García Moreno, Guillermo Botella, Alberto A. Del Barrio, and Jennifer Hasler
- Subjects
Adder ,Artificial neural network ,business.industry ,Computer science ,Analog computer ,Pattern recognition ,computer.software_genre ,law.invention ,Software framework ,Software ,law ,Feedforward neural network ,Artificial intelligence ,Electrical and Electronic Engineering ,business ,Field-programmable gate array ,computer ,MNIST database - Abstract
Analog computing has been recovering its relevance in the recent years. Field-Programmable Analog Arrays (FPAAs) are the equivalent to Field-Programmable Gate Arrays (FPGAs) but in the analog and mixed-signal domain. In order to increase the amount of analog resources, in this brief a cluster of 40 FPAAs is proposed. As a use case, a 19-8-6-4 feedforward Neural Network has been implemented on such cluster. With the help of a DCT-based software framework, this NN is able to classify $28 \times 28$ images from MNIST. Results show that the analog network is able to obtain similar results as the software baseline network.
- Published
- 2021
11. Improving the Reliability of Pick-and-Place With Aerial Vehicles Through Fault-Tolerant Software and a Custom Magnetic End-Effector
- Author
-
Soowon Kim, Gowtham Garimella, Matthew Sheckells, Marin Kobilarov, and Gabriel Baraban
- Subjects
Control and Optimization ,business.industry ,Computer science ,Mechanical Engineering ,Reliability (computer networking) ,Real-time computing ,Biomedical Engineering ,ComputerApplications_COMPUTERSINOTHERSYSTEMS ,Robotics ,computer.software_genre ,Software quality ,Computer Science Applications ,Task (project management) ,Human-Computer Interaction ,Software framework ,Software ,Artificial Intelligence ,Control and Systems Engineering ,Robustness (computer science) ,Software fault tolerance ,Computer Vision and Pattern Recognition ,Artificial intelligence ,business ,computer - Abstract
Aerial manipulation is an emerging field in robotics with various potential applications such as transport and delivery, agriculture, and, infrastructure inspection. To deploy aerial vehicles in the real world, the safety and reliability of these systems is paramount. Motivated by the need for safety and reliability, this work proposes a software framework that has built-in robustness to algorithmic failures and hardware faults. The framework allows users to build complex applications while reasoning about faults that can happen at different stages of an aerial manipulation task and specifying fallback actions to return to normal operating mode. The aerial manipulator is further endowed with a magnetic gripper that can handle positional errors arising from perception and control uncertainties. We also introduce a bias estimator for measuring the contact forces and sensor bias. We demonstrate how the estimator can be used to detect either completion or failures across several tasks. We demonstrate the reliability of the proposed framework on two tasks: package sorting task (e.g. as might be used in a distribution center) and sensor placement task (for infrastructure inspection). We show different failure modes that can occur and how our aerial manipulation system recovers from them.
- Published
- 2021
12. GraphAttack
- Author
-
Tyler Sorensen, Aninda Manocha, Opeoluwa Matthews, Margaret Martonosi, Juan L. Aragón, and Esin Tureci
- Subjects
Multi-core processor ,Speedup ,Memory hierarchy ,Computer science ,Data parallelism ,Parallel computing ,computer.software_genre ,Software framework ,Hardware and Architecture ,Scalability ,Graph (abstract data type) ,computer ,Queue ,Software ,Information Systems - Abstract
Graph structures are a natural representation of important and pervasive data. While graph applications have significant parallelism, their characteristic pointer indirect loads to neighbor data hinder scalability to large datasets on multicore systems. A scalable and efficient system must tolerate latency while leveraging data parallelism across millions of vertices. Modern Out-of-Order (OoO) cores inherently tolerate a fraction of long latencies, but become clogged when running severely memory-bound applications. Combined with large power/area footprints, this limits their parallel scaling potential and, consequently, the gains that existing software frameworks can achieve. Conversely, accelerator and memory hierarchy designs provide performant hardware specializations, but cannot support diverse application demands. To address these shortcomings, we present GraphAttack, a hardware-software data supply approach that accelerates graph applications on in-order multicore architectures. GraphAttack proposes compiler passes to (1) identify idiomatic long-latency loads and (2) slice programs along these loads into data Producer/ Consumer threads to map onto pairs of parallel cores. Each pair shares a communication queue; the Producer asynchronously issues long-latency loads, whose results are buffered in the queue and used by the Consumer. This scheme drastically increases memory-level parallelism (MLP) to mitigate latency bottlenecks. In equal-area comparisons, GraphAttack outperforms OoO cores, do-all parallelism, prefetching, and prior decoupling approaches, achieving a 2.87× speedup and 8.61× gain in energy efficiency across a range of graph applications. These improvements scale; GraphAttack achieves a 3× speedup over 64 parallel cores. Lastly, it has pragmatic design principles; it enhances in-order architectures that are gaining increasing open-source support.
- Published
- 2021
13. swFLOW: A large-scale distributed framework for deep learning on Sunway TaihuLight supercomputer
- Author
-
Mingfan Li, Qian Xiao, Junshi Chen, Rongfen Lin, Fei Wang, Guang R. Gao, Han Lin, Jose Monsalve Diaz, and Hong An
- Subjects
Information Systems and Management ,Speedup ,Computer science ,Distributed computing ,02 engineering and technology ,computer.software_genre ,Convolutional neural network ,Bottleneck ,Theoretical Computer Science ,Artificial Intelligence ,0202 electrical engineering, electronic engineering, information engineering ,Sunway TaihuLight ,business.industry ,Deep learning ,05 social sciences ,050301 education ,Supercomputer ,Computer Science Applications ,Software framework ,Stochastic gradient descent ,Control and Systems Engineering ,020201 artificial intelligence & image processing ,Artificial intelligence ,business ,0503 education ,computer ,Software - Abstract
Deep learning technology is widely used in many modern fields and a number of models and software frameworks have been proposed. However, it is still very difficult to process deep learning tasks efficiently on traditional high performance computing (HPC) systems. In this paper, we propose swFLOW: a large-scale distributed framework for deep learning on Sunway TaihuLight. Based on the performance analysis results of convolutional neural network (CNN), we optimize the convolutional layer , and get 10.42× speedup compared to the original version. As for distributed training, we use elastic averaging stochastic gradient descent (EASGD) algorithm to reduce communication. On 512 processes, we get a parallel efficiency of 81.01% with communication period τ = 8 . Particularly, a decentralized implementation of distributed swFLOW system is presented to alleviate bottleneck of the central server. By using distributed swFLOW system, we can scale the batch size up to 4096 among 1024 concurrent processes for cancerous region detection algorithm . The successful application on swFLOW reveals the great opportunity for joint combination of deep learning and HPC system.
- Published
- 2021
14. PlanAlyzer
- Author
-
Emma Tosch, J. Eliot B. Moss, Eytan Bakshy, Emery D. Berger, and David Jensen
- Subjects
FOS: Computer and information sciences ,General Computer Science ,Computer science ,02 engineering and technology ,Machine learning ,computer.software_genre ,Software ,020204 information systems ,0202 electrical engineering, electronic engineering, information engineering ,Internal validity ,Safety, Risk, Reliability and Quality ,Computer Science - Programming Languages ,business.industry ,020207 software engineering ,Software framework ,Software deployment ,Scripting language ,Scale (social sciences) ,Causal inference ,Key (cryptography) ,The Internet ,Artificial intelligence ,Precision and recall ,business ,computer ,Programming Languages (cs.PL) - Abstract
Online experiments are ubiquitous. As the scale of experiments has grown, so has the complexity of their design and implementation. In response, firms have developed software frameworks for designing and deploying online experiments. Ensuring that experiments in these frameworks are correctly designed and that their results are trustworthy---referred to as *internal validity*---can be difficult. Currently, verifying internal validity requires manual inspection by someone with substantial expertise in experimental design. We present the first approach for statically checking the internal validity of online experiments. Our checks are based on well-known problems that arise in experimental design and causal inference. Our analyses target PlanOut, a widely deployed, open-source experimentation framework that uses a domain-specific language to specify and run complex experiments. We have built a tool, PlanAlyzer, that checks PlanOut programs for a variety of threats to internal validity, including failures of randomization, treatment assignment, and causal sufficiency. PlanAlyzer uses its analyses to automatically generate *contrasts*, a key type of information required to perform valid statistical analyses over experimental results. We demonstrate PlanAlyzer's utility on a corpus of PlanOut scripts deployed in production at Facebook, and we evaluate its ability to identify threats to validity on a mutated subset of this corpus. PlanAlyzer has both precision and recall of 92% on the mutated corpus, and 82% of the contrasts it automatically generates match hand-specified data., 30 pages, hella long
- Published
- 2021
15. A neuron fuzzy identification system based on a complex nonlinear mathematical model
- Author
-
Hongying Luo, Jun Liu, and Xuebin Li
- Subjects
Hardware architecture ,Data stream ,Computer Networks and Communications ,business.industry ,Computer science ,computer.software_genre ,Fuzzy logic ,Software framework ,Identification (information) ,Software ,Continuous signal ,Electrical and Electronic Engineering ,business ,Algorithm ,computer ,Digital signal processing ,Information Systems - Abstract
During the implementation process of identification systems, multiple factors need to be considered at the same time. Due to the large amount of calculation and the randomness of the signal, the automatic identification capability of the machine is currently poor, especially for continuous signal. In order to solve this problem, a neuron fuzzy identification system was described based on a complex nonlinear mathematical model, which was designed from both hardware and software aspects of the system. The hardware architecture diagram was constructed based on the S3C:2440 microprocessor. The main modules include the power module, acquisition module, storage module, and output module. The DSP/BIOS system was used to construct the software framework diagram of the neuron fuzzy identification system to describe the identification process of video data stream, image data stream and control signal data. The software algorithm was designed based on the neuron fuzzy theory to establish a fuzzy similarity matrix and the best identification result was found by the maximum and minimum methods. The experimental results show that the designed system has higher recognition ability. When the frame length reaches 40 frames, the recognition rate increases to a larger value, the recognition rate is 75%, and the information is more accurate.
- Published
- 2021
16. A customisable pipeline for the semi-automated discovery of online activists and social campaigns on Twitter
- Author
-
Alexander Romanovsky, Alessandro Garcia, Flavio Primo, Paolo Missier, and Rafael Maiani de Mello
- Subjects
Focus (computing) ,Online activists ,Computer Networks and Communications ,Operational definition ,Computer science ,Context (language use) ,02 engineering and technology ,computer.software_genre ,Data science ,Pipeline (software) ,Article ,Influencer marketing ,Influence theories ,Software framework ,Online user discovery ,Online influencers ,Hardware and Architecture ,020204 information systems ,0202 electrical engineering, electronic engineering, information engineering ,Twitter analytics ,020201 artificial intelligence & image processing ,Social media ,Set (psychology) ,computer ,Software - Abstract
Substantial research is available on detectinginfluencerson social media platforms. In contrast, comparatively few studies exists on the role ofonline activists, defined informally as users who actively participate in socially-minded online campaigns. Automatically discovering activists who can potentially be approached by organisations that promote social campaigns is important, but not easy, as they are typically active only locally, and, unlike influencers, they are not central to large social media networks. We make the hypothesis that such interesting users can be found on Twitter within temporally and spatially localisedcontexts. We define these as small but topical fragments of the network, containing interactions about social events or campaigns with a significant online footprint. To explore this hypothesis, we have designed an iterative discovery pipeline consisting of two alternating phases of user discovery and context discovery. Multiple iterations of the pipeline result in a growing dataset of user profiles for activists, as well as growing set of online social contexts. This mode of exploration differs significantly from prior techniques that focus on influencers, and presents unique challenges because of the weak online signal available to detect activists. The paper describes the design and implementation of the pipeline as a customisable software framework, where user-defined operational definitions of online activism can be explored. We present an empirical evaluation on two extensive case studies, one concerning healthcare-related campaigns in the UK during 2018, the other related to online activism in Italy during the COVID-19 pandemic.
- Published
- 2021
17. A Programming Language for Data Privacy with Accuracy Estimations
- Author
-
Elisabet Lobo-Vesga, Marco Gaboardi, and Alejandro Russo
- Subjects
Information privacy ,Functional programming ,education.field_of_study ,Programming language ,Computer science ,Population ,020207 software engineering ,02 engineering and technology ,computer.software_genre ,Software framework ,Set (abstract data type) ,Taint checking ,020204 information systems ,Component (UML) ,0202 electrical engineering, electronic engineering, information engineering ,Differential privacy ,education ,computer ,Software - Abstract
Differential privacy offers a formal framework for reasoning about the privacy and accuracy of computations on private data. It also offers a rich set of building blocks for constructing private data analyses. When carefully calibrated, these analyses simultaneously guarantee the privacy of the individuals contributing their data, and the accuracy of the data analysis results, inferring useful properties about the population. The compositional nature of differential privacy has motivated the design and implementation of several programming languages to ease the implementation of differentially private analyses. Even though these programming languages provide support for reasoning about privacy, most of them disregard reasoning about the accuracy of data analyses. To overcome this limitation, we present DPella, a programming framework providing data analysts with support for reasoning about privacy, accuracy, and their trade-offs. The distinguishing feature of DPella is a novel component that statically tracks the accuracy of different data analyses. To provide tight accuracy estimations, this component leverages taint analysis for automatically inferring statistical independence of the different noise quantities added for guaranteeing privacy. We evaluate our approach by implementing several classical queries from the literature and showing how data analysts can calibrate the privacy parameters to meet the accuracy requirements, and vice versa.
- Published
- 2021
18. Grasping Robot Integration and Prototyping: The GRIP Software Framework
- Author
-
Brice Denoun, Miles Hansard, Beatriz Leon, and Lorenzo Jamone
- Subjects
0209 industrial biotechnology ,Computer science ,Process (engineering) ,business.industry ,02 engineering and technology ,computer.software_genre ,Computer Science Applications ,Software framework ,020901 industrial engineering & automation ,Software ,Control and Systems Engineering ,Human–computer interaction ,Research community ,Component-based software engineering ,Robot perception ,Task analysis ,Robot ,Electrical and Electronic Engineering ,business ,computer - Abstract
Robotic manipulation is fundamental to many realworld applications; however, it is an unsolved problem that remains a very active research area. New algorithms for robot perception and control are frequently proposed by the research community. These methods must be thoroughly evaluated in realistic conditions before they can be adopted by industry. This process can be extremely time consuming, mainly due to the complexity of integrating different hardware and software components.
- Published
- 2021
19. End-to-end privacy preserving deep learning on multi-institutional medical imaging
- Author
-
Marcus R. Makowski, Daniel Rueckert, Jason Mancuso, Dmitrii Usynin, Marc-Matthias Steinborn, Rickmer Braren, Andreas Saleh, Georgios Kaissis, Théo Ryffel, Andrew Trask, Ionésio Lima, Friederike Jungmann, Jonathan Passerat-Palmbach, and Alexander Ziller
- Subjects
0301 basic medicine ,Computer Networks and Communications ,Computer science ,business.industry ,Deep learning ,Inference ,Machine learning ,computer.software_genre ,Encryption ,Convolutional neural network ,Human-Computer Interaction ,Software framework ,03 medical and health sciences ,030104 developmental biology ,0302 clinical medicine ,End-to-end principle ,Artificial Intelligence ,Medical imaging ,Computer Vision and Pattern Recognition ,Applications of artificial intelligence ,Artificial intelligence ,business ,computer ,030217 neurology & neurosurgery ,Software - Abstract
Using large, multi-national datasets for high-performance medical imaging AI systems requires innovation in privacy-preserving machine learning so models can train on sensitive data without requiring data transfer. Here we present PriMIA (Privacy-preserving Medical Image Analysis), a free, open-source software framework for differentially private, securely aggregated federated learning and encrypted inference on medical imaging data. We test PriMIA using a real-life case study in which an expert-level deep convolutional neural network classifies paediatric chest X-rays; the resulting model’s classification performance is on par with locally, non-securely trained models. We theoretically and empirically evaluate our framework’s performance and privacy guarantees, and demonstrate that the protections provided prevent the reconstruction of usable data by a gradient-based model inversion attack. Finally, we successfully employ the trained model in an end-to-end encrypted remote inference scenario using secure multi-party computation to prevent the disclosure of the data and the model. Gaining access to medical data to train AI applications can present problems due to patient privacy or proprietary interests. A way forward can be privacy-preserving federated learning schemes. Kaissis, Ziller and colleagues demonstrate here their open source framework for privacy-preserving medical image analysis in a remote inference scenario.
- Published
- 2021
20. The Vision Behind MLPerf: Understanding AI Inference Performance
- Author
-
David Kanter, Christine Cheng, Vijay Janapa Reddi, Carole-Jean Wu, Peter Mattson, and Guenther Schmuelling
- Subjects
Computer science ,business.industry ,02 engineering and technology ,Benchmarking ,computer.software_genre ,020202 computer hardware & architecture ,Software framework ,Knowledge-based systems ,Software ,Hardware and Architecture ,0202 electrical engineering, electronic engineering, information engineering ,Benchmark (computing) ,Use case ,Performance measurement ,Electrical and Electronic Engineering ,Software engineering ,business ,computer ,Software measurement - Abstract
Deep learning has sparked a renaissance in computer systems and architecture. Despite the breakneck pace of innovation, there is a crucial issue concerning the research and industry communities at large: how to enable neutral and useful performance assessment for machine learning (ML) software frameworks, ML hardware accelerators, and ML systems comprising both the software stack and the hardware. The ML field needs systematic methods for evaluating performance that represents real-world use cases and useful for making comparisons across different software and hardware implementations. MLPerf answers the call. MLPerf is an ML benchmark standard driven by academia and industry (70+ organizations). Built out of the expertise of multiple organizations, MLPerf establishes a standard benchmark suite with proper metrics and benchmarking methodologies to level the playing field for ML system performance measurement of different ML inference hardware, software, and services.
- Published
- 2021
21. Programming framework and infrastructure for self-adaptation and optimized evolution method for microservice systems in cloud–edge environments
- Author
-
Xiaofei Xu, Zhongjie Wang, Zhiying Tu, and Xiang He
- Subjects
Service (systems architecture) ,Service system ,Computer Networks and Communications ,Computer science ,business.industry ,Distributed computing ,Quality of service ,020206 networking & telecommunications ,Cloud computing ,02 engineering and technology ,Microservices ,computer.software_genre ,Software framework ,Hardware and Architecture ,Software deployment ,Server ,0202 electrical engineering, electronic engineering, information engineering ,020201 artificial intelligence & image processing ,business ,Mobile device ,computer ,Software ,Edge computing - Abstract
Edge computing technologies facilitate the deployment of services on nearby edge servers with a large number of end users and their mobile devices to fulfill personalized demands. Owing to frequent changes in user mobility and demands, service systems deployed in an edge–cloud environment must continuously adapt to ensure that the quality of service (QoS) perceived by the end users is maintained at a stable and satisfactory level. As it is difficult for system operation engineers to manually deal with such frequent and large-scale evolution due to problems of cost and efficiency, self-adaptation of the system is essential. In this paper, we present a programming framework for microservices (EPF4M) and an infrastructure for self-adaptive microservice systems (EI4MS) for the cloud–edge environment based on microservice architecture. Our study follows a “monitoring–analyzing–planning–execution” control loop that empowers the service systems to redeploy the services according to changes in the QoS. A two-phase strategy is adopted to minimize the side effects of the loop on the performance of the service system. A prototype of this framework and infrastructure has been open-sourced and verified through experiments conducted in a real cloud–edge environment. The results demonstrate the usefulness and advantages of our approach.
- Published
- 2021
22. CogTool+
- Author
-
Shujun Li, Patrice Rusconi, and Haiyue Yuan
- Subjects
automation ,Cognitive modeling ,CogTool ,cyber security ,human performance evaluation ,parameterization ,simulation ,software ,user authentication ,QA75 ,Cognitive model ,Computer science ,BF ,02 engineering and technology ,computer.software_genre ,Software ,Human–computer interaction ,0202 electrical engineering, electronic engineering, information engineering ,0501 psychology and cognitive sciences ,QA76.9.H85 ,050107 human factors ,QA76.76 ,business.industry ,Scale (chemistry) ,05 social sciences ,020207 software engineering ,Automation ,Human-Computer Interaction ,Software framework ,Template ,Scalability ,User interface ,business ,computer - Abstract
Cognitive modeling tools have been widely used by researchers and practitioners to help design, evaluate, and study computer user interfaces (UIs). Despite their usefulness, large-scale modeling tasks can still be very challenging due to the amount of manual work needed. To address this scalability challenge, we propose CogTool+, a new cognitive modeling software framework developed on top of the well-known software tool CogTool. CogTool+ addresses the scalability problem by supporting the following key features: (1) a higher level of parameterization and automation; (2) algorithmic components; (3) interfaces for using external data; and (4) a clear separation of tasks, which allows programmers and psychologists to define reusable components (e.g., algorithmic modules and behavioral templates) that can be used by UI/UX researchers and designers without the need to understand the low-level implementation details of such components. CogTool+ also supports mixed cognitive models required for many large-scale modeling tasks and provides an offline analyzer of simulation results. In order to show how CogTool+ can reduce the human effort required for large-scale modeling, we illustrate how it works using a pedagogical example, and demonstrate its actual performance by applying it to large-scale modeling tasks of two real-world user-authentication systems.
- Published
- 2021
23. First application of the GPU-based software framework TIGRE for proton CT image reconstruction
- Author
-
Thomas Bergauer, Sepideh Hatamikia, Wolfgang Birkfellner, Dietmar Georg, Florian Pitters, Alexander Burker, Christian Irmler, Stefanie Kaser, Felix Ulrich-Pur, and Albert Hirtl
- Subjects
Computer science ,Monte Carlo method ,Biophysics ,Measure (physics) ,General Physics and Astronomy ,Iterative reconstruction ,computer.software_genre ,030218 nuclear medicine & medical imaging ,03 medical and health sciences ,0302 clinical medicine ,Software ,Image Processing, Computer-Assisted ,Humans ,Radiology, Nuclear Medicine and imaging ,Projection (set theory) ,Proton therapy ,Phantoms, Imaging ,business.industry ,General Medicine ,Software framework ,Simultaneous Algebraic Reconstruction Technique ,030220 oncology & carcinogenesis ,Protons ,Tomography, X-Ray Computed ,business ,Monte Carlo Method ,computer ,Algorithm ,Algorithms - Abstract
In proton therapy, the knowledge of the proton stopping power, i.e. the energy deposition per unit length within human tissue, is essential for accurate treatment planning. One suitable method to directly measure the stopping power is proton computed tomography (pCT). Due to the proton interaction mechanisms in matter, pCT image reconstruction faces some challenges: the unique path of each proton has to be considered separately in the reconstruction process adding complexity to the reconstruction problem. This study shows that the GPU-based open-source software toolkit TIGRE, which was initially intended for X-ray CT reconstruction, can be applied to the pCT image reconstruction problem using a straight line approach for the proton path. This simplified approach allows for reconstructions within seconds. To validate the applicability of TIGRE to pCT, several Monte Carlo simulations modeling a pCT setup with two Catphan® modules as phantoms were performed. Ordered-Subset Simultaneous Algebraic Reconstruction Technique (OS-SART) and Adaptive-Steepest-Descent Projection Onto Convex Sets (ASD-POCS) were used for image reconstruction. Since the accuracy of the approach is limited by the straight line approximation of the proton path, requirements for further improvement of TIGRE for pCT are addressed.
- Published
- 2021
24. Development and Testing of Algorithms for Vehicle Type Recognition and Car Tracking with Photo and Video Traffic Enforcement Cameras
- Author
-
M. A. Laptev, D. V. Nekrasov, and S. M. Staroletov
- Subjects
Traffic analysis ,business.industry ,Computer science ,ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION ,computer.software_genre ,Tracking (particle physics) ,Computer Graphics and Computer-Aided Design ,Convolutional neural network ,Task (project management) ,Software framework ,Haar-like features ,Software ,Pattern recognition (psychology) ,Computer Vision and Pattern Recognition ,business ,computer ,Algorithm - Abstract
The work is devoted to the research that was carried out within the framework of computer vision problems applicable to the analysis of images and video information with vehicles. We solve the problem of classifying vehicles. We analyze the drawbacks of Haar features and convolutional neural networks and test the obtained networks using the key point method; we construct an integral algorithm that includes several networks, and we further validate it on a large number of real photographs and types of vehicles. Next, we solve the task to develop a software framework for tracking vehicles by analyzing adjacent photographs from a video sequence. After that, we consider the tracking task in more detail. We analyze modern tracking algorithms using machine learning and describe our implemented tracker with support for the appearance of obstacles between the camera and a moving vehicle. As a result, we propose algorithms and open-source software that, after being configured for specific cameras, can be used in traffic analysis systems.
- Published
- 2021
25. A Vector Finder Toolkit for Track Reconstruction in MPD ITS
- Author
-
Dmitry Zinchenko, V. Vasendina, E. G. Nikonov, and A.I. Zinchenko
- Subjects
Computer science ,Physics::Instrumentation and Detectors ,QC770-798 ,computer.software_genre ,01 natural sciences ,law.invention ,Computational science ,Software ,law ,Nuclear and particle physics. Atomic energy. Radioactivity ,0103 physical sciences ,Nuclear Experiment ,010306 general physics ,Collider ,silicon pixel detector ,vertex reconstruction ,Time projection chamber ,Pixel ,010308 nuclear & particles physics ,business.industry ,Detector ,track reconstruction ,Tracking system ,heavy-ion collisions ,Software framework ,Upgrade ,High Energy Physics::Experiment ,business ,computer - Abstract
As a part of the future upgrade program of the Multi-Purpose Detector (MPD) experiment at the Nuclotron-Based Ion Collider Facility (NICA) complex, an Inner Tracking System (ITS) made of Monolitic Active Pixel Sensors (MAPSs) is proposed between the beam pipe and the Time Projection Chamber (TPC). It is expected that the new detector will enhance the experimental potential for the reconstruction of short-lived particles—in particular, those containing the open charm particle. To study the detector performance and select its best configuration, a track reconstruction approach based on a constrained combinatorial search was developed and implemented as a software toolkit called Vector Finder. This paper describes the proposed approach and demonstrates its characteristics for primary and secondary track finding in ITS, ITS-to-TPC track matching and hyperon reconstruction within the MPD software framework. The results were obtained on a set of simulated central gold–gold collision events at sNN=9 GeV with an average multiplicity of ∼1000 charged particles in the detector acceptance produced with the Ultra-Relativistic Quantum Molecular Dynamics (UrQMD) generator.
- Published
- 2021
26. JCOGIN: a programming framework for particle transport on combinatorial geometry
- Author
-
Baoyin Zhang, Aiqing Zhang, Wang Xin, Gang Li, Xiaolin Cao, Wang Wei, and Zeyao Mo
- Subjects
Multi-core processor ,Parallelism (rhetoric) ,Application programming interface ,Computer science ,Discrete geometry ,Domain decomposition methods ,Parallel computing ,computer.software_genre ,Bottleneck ,Theoretical Computer Science ,Software framework ,Data model ,Hardware and Architecture ,computer ,Software ,Information Systems - Abstract
Domain-specific programming frameworks are usually effective to simplify the development of large-scale applications on supercomputers. This paper introduces a parallel programming framework named JCOGIN for particle transport on combinatorial geometry. JCOGIN provides a combinatorial geometry data model and a patch-based parallel computing model to manage the data distribution in parallel computing and implements the hybrid parallelism of the domain decomposition and the particle parallelism on MPI/OpenMP to overcome the bottleneck of huge memory demand and long computational time. The application programming interface of JCOGIN can support users to quickly develop their parallel particle transport applications. Based on this framework, users only need to write serial codes for large-scale numerical simulations on modern supercomputers. The parallel efficiency of applications based on JCOGIN can reach up to 80% on hundreds of thousands of CPU cores.
- Published
- 2021
27. A Framework for Comparative Evaluation of High-Performance Virtualized Networking Mechanisms
- Author
-
Gabriele Ara, Luca Abeni, Leonardo Lai, Carlo Vitucci, and Tommaso Cucinotta
- Subjects
Computer science ,Cloud computing ,Throughput ,02 engineering and technology ,computer.software_genre ,Article ,Containers ,NFV ,Software ,020204 information systems ,0202 electrical engineering, electronic engineering, information engineering ,DPDK ,Kernel bypass ,Netmap ,Latency (engineering) ,Virtual network ,business.industry ,020206 networking & telecommunications ,Software framework ,Computer architecture ,Scalability ,Key (cryptography) ,business ,computer - Abstract
This paper presents an extension to a software framework designed to evaluate the efficiency of different software and hardware-accelerated virtual switches, each commonly adopted on Linux to provide virtual network connectivity to containers in high-performance scenarios, like in Network Function Virtualization (NFV). We present results from the use of our tools, showing the performance of multiple high-performance networking frameworks on a specific platform, comparing the collected data for various key metrics, namely throughput, latency and scalability, with respect to the required computational power.
- Published
- 2021
28. A Highly Modular Software Framework for Reducing Software Development Time of Nanosatellites
- Author
-
Mohammed I. Awad, Prashanth Reddy Marpu, Aisha El Allam, Abdul-Halim Jallad, and Maen Takruri
- Subjects
software architecture ,General Computer Science ,business.industry ,Computer science ,CubeSat ,General Engineering ,Software development ,flight software ,computer.software_genre ,Modularity ,nanosatellites ,Software quality ,TK1-9971 ,Software framework ,Software portability ,Software ,Systems engineering ,General Materials Science ,Electrical engineering. Electronics. Nuclear engineering ,Electrical and Electronic Engineering ,Software architecture ,business ,computer ,Agile software development - Abstract
The standardization of the physical aspects of nanosatellites (also known as CubeSats) and their wide adoption in academia and industry has made the mass production and availability of off-the-shelf components possible. While this has led to a significant reduction in satellite development time, the fact remains that a considerable amount of mission development time and effort continues to be spent on flight software development. The CubeSat’s agile development environment makes it challenging to utilize the advantages of existing software frameworks. Such an adoption is not straightforward due to the added complexity characterized by a steep learning curve. A well-designed flight software architecture mitigates possible sources of failure and increases mission success rate while maintaining moderate complexity. This paper presents a novel approach to a flight software framework developed specifically for nanosatellites. The software framework is characterized by simplicity, reliability, modularity, portability, and real-time capability. The main features of the proposed framework include providing a standardized and explicit skeleton for each module to simplify their construction, offering standardized interfaces for all modules to simplify communication, and providing a collection of ready-to-use common services open for further enhancement by CubeSat software developers. The framework efficiency was demonstrated through a software developed for the MeznSat mission that was successfully launched into Low Earth Orbit in September 2020. The proposed software framework proved to simplify software development for the application developer while significantly enhancing software modularity.
- Published
- 2021
29. Palabos: Parallel Lattice Boltzmann Solver
- Author
-
Christos Kotsalos, Federico Brogi, Christophe Coreixas, Sébastien Leclaire, Orestis Malaspinas, Remy Petkantchin, Sha Li, Yann Thorimbert, Mohamed Ben Belgacem, Bastien Chopard, Francesco Marson, Dimitrios Kontaxakis, Jonas Latt, Raphaël Conradin, Joel Beny, Franck Raynaud, Jonathan Lemus, Daniel Lagrava, and Andrea Parmigiani
- Subjects
business.industry ,Lattice Boltzmann methods ,Computational fluid dynamics ,Solver ,computer.software_genre ,Data structure ,01 natural sciences ,010305 fluids & plasmas ,Computational science ,Software framework ,Computational Mathematics ,Software ,Computational Theory and Mathematics ,Modeling and Simulation ,0103 physical sciences ,Programming paradigm ,Benchmark (computing) ,010306 general physics ,business ,computer ,Mathematics - Abstract
We present the scope, concepts, data structures and application programming models of the open-source Lattice Boltzmann library Palabos. Palabos is a C++ software platform developed since 2010 for Computational Fluid Dynamics simulations and Lattice Boltzmann modeling, which specifically targets applications with complex, coupled physics. The software proposes a very broad modeling framework, capable of addressing a large number of applications of interest in the Lattice Boltzmann community, yet exhibits solid computational performance. The article describes the philosophy of this programming framework and lists the models already implemented. Finally, benchmark simulations are provided which serve as a proof of quality of the implemented core functionalities.
- Published
- 2021
30. A Skill Programming Method Based on Assembly Motion Primitive for Modular Assembly System
- Author
-
Chao Shao, Pengyue Guo, Dongsheng Zhu, Zhijing Zhang, Yan Liu, and Yujun Liu
- Subjects
Engineering drawing ,General Computer Science ,business.industry ,Computer science ,General Engineering ,Process (computing) ,Modular design ,computer.software_genre ,modular assembly system ,Assembly motion primitive ,TK1-9971 ,Software framework ,Software ,Task analysis ,Robot ,General Materials Science ,Electrical engineering. Electronics. Nuclear engineering ,skill learning ,business ,Hidden Markov model ,hidden Markov model ,computer ,Graphical user interface - Abstract
To improve the programming efficiency of automatic assembly system, a novel skill programming framework based on task learning is proposed for modular assembly system in this paper. In this framework, the motion sequence of assembly skills can be modeled by demonstration data. And the assembly task is represented hierarchically. A complete assembly process of a part is divided into several skills, and each skill is divided into several sequential assembly motion primitives (AMP) of multiple modules. Then, a learning method of assembly motion sequence based on Hidden Markov Model is proposed, and the maximum probability method is used to generate the optimal sequential AMP. Each AMP is input to the assembly system in the form of instruction to complete the assembly. Aiming at the problem of accurate positioning and trajectory planning, visual guidance and direct teaching method are used to settle this problem. To evaluate the viability of the proposed framework, a customized modular assembly system is used to acquire the demonstration data, and a graphical user interface (GUI) software is designed. Five assembly skills are learned. Experimental are conducted to validate the effectiveness of the proposed method.
- Published
- 2021
31. A software framework for real-time multi-modal detection of microsleeps
- Author
-
Knopp, SJ, Bones, PJ, Weddell, SJ, and Jones, RD
- Published
- 2017
- Full Text
- View/download PDF
32. Towards enhanced MRI by using a multiple back end programming framework
- Author
-
Jesus Carretero, J. Daniel Garcia, David del Rio Astorga, Javier Garcia-Blas, Comunidad de Madrid, European Commission, and Ministerio de Economía y Competitividad (España)
- Subjects
Informática ,Computer Networks and Communications ,Data stream mining ,Computer science ,Distributed computing ,Perspective (graphical) ,GRPPI ,020206 networking & telecommunications ,Task-level parallelism ,02 engineering and technology ,Data-level parallelism ,computer.software_genre ,MRI reconstruction ,Task (project management) ,Software framework ,Data stream processing ,Hardware and Architecture ,0202 electrical engineering, electronic engineering, information engineering ,Parallelism (grammar) ,020201 artificial intelligence & image processing ,computer ,Software - Abstract
In recent years, on-line processing of data streams (DaSP) has been established as a major computing paradigm. This is due mainly to two reasons: first, more and more data that are generated in near real-time need to be processed; the second reason is given by the need of efficient parallel applications. However, the above-mentioned areas expose a tough challenge over traditional data-analysis techniques, which have been forced to evolve to a stream perspective. In this work, we apply a novel multiple back end programming framework for stream data and task based parallelism to a multi-staged diffusion magnetic resonance imaging (MRI) toolkit, named pHARDI. The results demonstrate the benefits of using our framework in terms of performance and memory usage. The evaluation carried out also depicts that the speed-up of our parallel framework increases with the problem size. This work was supported by the EU project \ASPIDE: Exascale Programming Models for Extreme Data Processing" under grant 801091 and project TIN2016-79637-P \Towards unification of HPC and Big Data Paradigms" from the Spanish Ministry of Economy and Competitiveness of Spain. This research was partially supported by Madrid regional Government (Spain) under the grant \Convergencia Big data-Hpc: de los sensores a las Aplicaciones. (CABAHLA-CM)" Ref: S2018/TCS-4423.
- Published
- 2020
33. Curious Containers: A framework for computational reproducibility in life sciences with support for Deep Learning applications
- Author
-
Peter Hufnagl, Felix Bartusch, Jonas Annuscheit, Michael Witt, Bruno Schilling, Christian Herta, Dagmar Krefting, Klaus Strohmenger, and Christoph Jansen
- Subjects
Decision support system ,Reproducibility ,Computer Networks and Communications ,business.industry ,Computer science ,Deep learning ,Interoperability ,020206 networking & telecommunications ,02 engineering and technology ,computer.software_genre ,Software framework ,Software ,Workflow ,Hardware and Architecture ,Container (abstract data type) ,0202 electrical engineering, electronic engineering, information engineering ,020201 artificial intelligence & image processing ,Artificial intelligence ,Reference implementation ,business ,Software engineering ,computer - Abstract
In clinical scenarios, there is an increasing interest in complex computational experiments, as for example the training of Deep Learning models. Reproducibility is an essential property of such experiments, especially if the result contributes to a patient’s treatment. This paper introduces Curious Containers, a software framework for computational reproducibility that treats data, software and runtime environment as decentralized network resources. All experiment resources are described in a single file, using a new format that is compatible with a subset of the Common Workflow Language. Docker is used to deploy the experiment software in a container image, including arbitrary data transmission programs to connect with existing storage solutions. The framework supports Deep Learning applications, that have a high demand in storage and processing capabilities. Large datasets can be mounted inside containers via network filesystems like SSHFS based on the filesystem in user-space technology. The Nvidia-Container-Toolkit enables GPU usage. Curious Containers has been tested in two biomedical scenarios. The first use case is a Deep Learning application for tumor classification in images that requires a large dataset and a GPU. In this context, a prototypical integration of the framework with the existing Data Version Control system for exploratory Deep Learning modeling has been developed. The second use case extends an existing container image, including a scientific workflow for detection and comparison of human protein in mass spectrography data. The container image was originally developed for an archiving platform and could be extended to be compatible with both Curious Containers and cwltool, the Common Workflow Language reference implementation. The presented solution allows for consistent description and execution of computational experiments, while trying to be both flexible and interoperable with existing software and standards. Support for Deep Learning experiments is gaining importance as such systems are increasingly validated as medical decision support systems.
- Published
- 2020
34. A Power- and Performance-Aware Software Framework for Control System Applications
- Author
-
Bonnie Ferri, Aldo A. Ferri, Michael Giardino, and Eric Klawitter
- Subjects
Situation awareness ,business.industry ,Computer science ,Mobile robot ,Control engineering ,02 engineering and technology ,computer.software_genre ,020202 computer hardware & architecture ,Theoretical Computer Science ,Software framework ,Architecture framework ,Software ,Computational Theory and Mathematics ,Hardware and Architecture ,Control theory ,Control system ,0202 electrical engineering, electronic engineering, information engineering ,Robot ,business ,computer - Abstract
This article describes the development of a software architectural framework for implementing compute-aware control systems, where the term “compute-aware” describes controllers that can modify existing low-level computing platform power managers in response to the needs of the physical system controller. This level of interaction means that high-level decisions can be made as to when to operate the computing platform in a power-savings mode or a high-performance mode in response to situation awareness of the physical system. The framework is demonstrated experimentally on a mobile robot platform. In this example, a situation-aware governor is developed that adjusts the speed of the processor based on the physical performance of the robot as it traverses a path through obstacles. The results show that the situation-aware governor results in overall power savings of up to 38.9 percent with 1.3 percent degradation in performance compared to the static high-power strategy.
- Published
- 2020
35. TuneMPC—A Tool for Economic Tuning of Tracking (N)MPC Problems
- Author
-
Moritz Diehl, Mario Zanon, and Jochem De Schutter
- Subjects
0209 industrial biotechnology ,Control and Optimization ,business.industry ,Computer science ,Computation ,Stability (learning theory) ,02 engineering and technology ,computer.software_genre ,System dynamics ,Tracking error ,Software framework ,Model predictive control ,020901 industrial engineering & automation ,Software ,020401 chemical engineering ,Control and Systems Engineering ,Control theory ,0204 chemical engineering ,business ,computer ,Numerical stability - Abstract
Economic nonlinear model predictive control (NMPC) is a variant of NMPC that directly optimizes an economic performance index instead of a tracking error. Although economic NMPC can achieve excellent closed-loop performance, the associated computational effort as well as the difficulty of guaranteeing stability in practice are its main drawbacks. Motivated by these difficulties, a formal procedure was developed that tunes a tracking (non)linear MPC scheme so that it is first-order equivalent to economic NMPC. This letter introduces TuneMPC, a new open-source software framework that closes the gap between the underlying theory and practical application of this tuning procedure. For user-provided system dynamics, constraints and economic objective, TuneMPC enables automated computation of optimal steady states and periodic trajectories, and returns the corresponding tuned stage cost matrices. To demonstrate the potential of the tool, we apply the technique to the challenging example of an autonomous tethered aircraft flying periodic orbits for airborne wind energy harvesting.
- Published
- 2020
36. Mining Design Pattern Use Scenarios and Related Design Pattern Pairs: A Case Study on Online Posts
- Author
-
Zhilei Ren, Zhong-Tian Long, He Jiang, Guojun Gao, and Dong Liu
- Subjects
Structure (mathematical logic) ,Information retrieval ,Computer science ,Design pattern ,020207 software engineering ,Context (language use) ,Sample (statistics) ,02 engineering and technology ,computer.software_genre ,Computer Science Applications ,Theoretical Computer Science ,Variety (cybernetics) ,Software framework ,Computational Theory and Mathematics ,Hardware and Architecture ,Software design pattern ,0202 electrical engineering, electronic engineering, information engineering ,computer ,Software - Abstract
In common design pattern collections, e.g., design pattern books, design patterns are documented with templates that consist of multiple attributes, such as intent, structure, and sample code. To adapt to modern developers, the depictions of design patterns, especially some specific attributes, should advance with the current programming technologies, for example, “known uses”, which exemplifies the use scenarios of design patterns in practice, and “related patterns”, which describes the relatedness between a design pattern and the others within a context. However, it is not easy to update the contents of these attributes manually due to the diversity of the programming technologies. To address this problem, in this work, we conducted a case study to mine design pattern use scenarios and related design pattern pairs from Stack Overflow posts to enrich the two attributes. We first extracted the question posts relevant to each design pattern by identifying the design pattern tags. Then, the topics of the posts were discovered by applying topic modeling techniques. Finally, by analyzing the topics specified for each design pattern, we detected 195 design pattern use scenarios and 70 related design pattern pairs, involving 61 design patterns totally. These findings are associated with a variety of popular software frameworks and programming techniques. They could complement the existing design pattern collections and help developers better acknowledge the usage and relatedness of design patterns in today’s programming practice.
- Published
- 2020
37. A Visual Programming Approach for Co-designed Robots
- Author
-
Francisco Ramos, Andrés S. Vázquez, Tomás Calvo, and Raul Fernandez
- Subjects
Self-reconfiguring modular robot ,Control and Optimization ,Computer science ,General Mathematics ,Programming complexity ,020101 civil engineering ,02 engineering and technology ,computer.software_genre ,0201 civil engineering ,Human–computer interaction ,0202 electrical engineering, electronic engineering, information engineering ,Visual programming language ,business.industry ,Mechanical Engineering ,Robotics ,Usability ,Automation ,Computer Science Applications ,Software framework ,Control and Systems Engineering ,Modeling and Simulation ,Robot ,020201 artificial intelligence & image processing ,Artificial intelligence ,business ,computer ,Software - Abstract
SUMMARYThis paper proposes an approach for the high-level programming of co-designed robots that reduces programming complexity. Particularly, the work presented focuses on the programming framework of an intelligent system, based on the IEEE Standard Ontologies for Robotics and Automation, which allows users the automatic design of robots and the automatic implementation of controllers in the Robot Operating System (ROS). In our approach, the co-designed robot functionalities are automatically translated into visual programming blocks allowing non-expert users an easy robot programming by means of a visual programming language. Several robot configurations and three case studies are provided as a proof of concept. The validation, in terms of usability, of the framework has been carried out with inexperienced users showing promising results.
- Published
- 2020
38. Analyzing system software components using API model guided symbolic execution
- Author
-
Ken Yihang Bai and Tuba Yavuz
- Subjects
Programming language ,business.industry ,Computer science ,020207 software engineering ,Linux kernel ,02 engineering and technology ,Symbolic execution ,computer.software_genre ,Software framework ,Protocol stack ,Software ,Component (UML) ,Component-based software engineering ,0202 electrical engineering, electronic engineering, information engineering ,business ,computer ,System software - Abstract
Analyzing real-world software is challenging due to complexity of the software frameworks or APIs they depend on. In this paper, we present a tool, PROMPT, that facilitates the analysis of software components using API model guided symbolic execution. PROMPT has a specification component, PROSE, that lets users define an API model, which consists of a set of data constraints and life-cycle rules that define control-flow constraints among sequentially composed API functions. Given a PROSE model and a software component, PROMPT symbolically executes the component while enforcing the specified API model. PROMPT has been implemented on top of the KLEE symbolic execution engine and has been applied to Linux device drivers from the video, sound, and network subsystems and to some vulnerable components of BlueZ, the implementation of the Bluetooth protocol stack for the Linux kernel. PROMPT detected two new and four known memory vulnerabilities in some of the analyzed system software components.
- Published
- 2020
39. A framework for Model-Driven Engineering of resilient software-controlled systems
- Author
-
Fulvio Patara, Jacopo Parri, Enrico Vicario, and Samuele Sampietro
- Subjects
0209 industrial biotechnology ,Process (engineering) ,Computer science ,02 engineering and technology ,computer.software_genre ,Theoretical Computer Science ,020901 industrial engineering & automation ,Systems Modeling Language ,0202 electrical engineering, electronic engineering, information engineering ,Adaptation (computer science) ,computer.programming_language ,System of systems ,Numerical Analysis ,business.industry ,020207 software engineering ,Functional requirement ,computer.file_format ,Computer Science Applications ,Software framework ,Computational Mathematics ,Computational Theory and Mathematics ,Executable ,Model-driven architecture ,Software engineering ,business ,computer ,Software - Abstract
Emergent paradigms of Industry 4.0 and Industrial Internet of Things expect cyber-physical systems to reliably provide services overcoming disruptions in operative conditions and adapting to changes in architectural and functional requirements. In this paper, we describe a hardware/software framework supporting operation and maintenance of software-controlled systems enhancing resilience by promoting a Model-Driven Engineering (MDE) process to automatically derive structural configurations and failure models from reliability artifacts. Specifically, a reflective architecture developed around digital twins enables representation and control of system Configuration Items properly derived from SysML Block Definition Diagrams, providing support for variation. Besides, a plurality of distributed analytic agents for qualitative evaluation over executable failure models empowers the system with runtime self-assessment and dynamic adaptation capabilities. We describe the framework architecture outlining roles and responsibilities in a System of Systems perspective, providing salient design traits about digital twins and data analytic agents for failure propagation modeling and analysis. We discuss a prototype implementation following the MDE approach, highlighting self-recovery and self-adaptation properties on a real cyber-physical system for vehicle access control to Limited Traffic Zones.
- Published
- 2020
40. AN HRM SYSTEM FOR SMALL AND MEDIUM ENTERPRISES (SME)S BASED ON CLOUD COMPUTING TECHNOLOGY
- Author
-
Subhi R. M. Zeebaree, Karwan Jacksi, Pavel Y. Abdullah, and Rizgar R. Zeabri
- Subjects
Human resource management system ,Knowledge management ,business.industry ,Computer science ,05 social sciences ,Cloud computing ,02 engineering and technology ,computer.software_genre ,Software framework ,Software ,Work (electrical) ,Human resource management ,0502 economics and business ,0202 electrical engineering, electronic engineering, information engineering ,020201 artificial intelligence & image processing ,Small and medium-sized enterprises ,Human resources ,business ,computer ,050203 business & management - Abstract
Technology has changed our life and the way we work; however, technology has affected several methods of working in Small and Medium Enterprises (SME)s. Human Resource (HR) is one of the core components in businesses, and nowadays most businesses are using technology for daily basis tasks. However, it still is not used all over the world. In Kurdistan Region-Iraq (KRI), most of the SMEs still use the old way of working and follow the paper-based method for their daily basis tasks. According to a survey, more than seventy percent of SMEs in Kurdistan are not using software to manage human resource management tasks. However, some big companies are using HRMS; but even then, there is a lack of use of Cloud Technology. In this study, a model of the Enterprise Human Resource Management System (EHRMS) is proposed and implemented to solve the HR problems in this area using Cloud Technology. The proposed system consists of sixteen standard modules which used usually with famous HRM systems. The system has been developed by using several technologies such as CodeIgniter as a software framework. The system is launched and deployed on Amazon Web Service (AWS) Elastic Compute Cloud (EC2).
- Published
- 2020
41. Smart heating in collaborative and reasoning-enabled housing units
- Author
-
George Kousiouris, Panagiotis Bourelos, Achilleas Marinakis, Theodora Varvarigou, Vrettos Moulos, and Orfefs Voutyras
- Subjects
Computer Networks and Communications ,business.industry ,Computer science ,020206 networking & telecommunications ,02 engineering and technology ,Modular design ,Social learning ,computer.software_genre ,Field (computer science) ,Knowledge sharing ,Software framework ,Work (electrical) ,Hardware and Architecture ,11. Sustainability ,0202 electrical engineering, electronic engineering, information engineering ,020201 artificial intelligence & image processing ,Case-based reasoning ,Software engineering ,business ,computer ,Software - Abstract
In aiming to create added intelligence for Things interfacing with IoT principles in the form of Services, Smart Heating Management is consistently a field of promising research. In the currently demonstrated work, an approach will be described for an operational software framework of modular components that can create intelligence in Housing Units as those presented in our scenario. The framework is based on the lightweight Case Based Reasoning approach and principles in order to describe the Problem and Solution of heating management, as well as to extract knowledge from generic historical data. Collaboration between houses is included through the sharing of anonymized high level problem-solution data, as an instantiation of the same social learning principles that govern human behavior and enable Knowledge diffusion. The approach is validated through historical data acquired in two time periods from the Camden (London) community residencies and demonstrates an average of 22% savings in boiler active time. Knowledge sharing is also analyzed along with the benefits and pitfalls it might produce.
- Published
- 2020
42. Massively Parallel CFD Simulation Software: CCFD Development and Optimization Based on Sunway TaihuLight
- Author
-
Zhonghua Lu, Wenpeng Ma, Xiazhen Liu, Jian Zhang, and Wu Yuan
- Subjects
Article Subject ,business.industry ,Computer science ,Domain decomposition methods ,010103 numerical & computational mathematics ,Parallel computing ,Load balancing (computing) ,computer.software_genre ,Grid ,01 natural sciences ,010305 fluids & plasmas ,Computer Science Applications ,Software framework ,QA76.75-76.765 ,Multigrid method ,Software ,0103 physical sciences ,Graph (abstract data type) ,Computer software ,0101 mathematics ,business ,computer ,Massively parallel - Abstract
A parallel framework software, CCFD, based on the structure grid, and suitable for parallel computing of super-large-scale structure blocks, is designed and implemented. An overdecomposition method, in which the load balancing strategy is based on the domain decomposition method, is designed for the graph subdivision algorithm. This method takes computation and communication as the limiting condition and realizes the load balance between blocks by dividing the weighted graph. The fast convergence technique of a high-efficiency parallel geometric multigrid greatly improves the parallel efficiency and convergence speed of CCFD software. This paper introduces the software structure, process invocations, and calculation method of CCFD and introduces a hybrid parallel acceleration technology based on the Sunway TaihuLight heterogeneous architecture. The results calculated by Onera-M6 and DLR-F6 standard model show that the software structure and method in this paper are feasible and can meet the requirements of a large-scale parallel solution.
- Published
- 2020
43. AutoMATiC: Code Generation of Model Predictive Control Algorithms for Microcontrollers
- Author
-
Patryk Chaber and Maciej Lawrynczuk
- Subjects
business.industry ,Computer science ,Process (computing) ,computer.software_genre ,Computer Science Applications ,Software framework ,Model predictive control ,Microcontroller ,Software ,Control and Systems Engineering ,Process control ,Code generation ,Software system ,Electrical and Electronic Engineering ,business ,computer ,Algorithm ,Information Systems - Abstract
This article describes the AutoMATiC software system that generates the C code of software implementation of model predictive control algorithms for a chosen target microcontroller. The following components of the AutoMATiC tool are described: the system structure, workflow, and software framework. The system includes a transcompiler, a simulator, and a profiler. To discuss effectiveness of the system, a dynamic process with two inputs and two outputs is considered. The following advantages of the AutoMATiC system are emphasized: simplicity of use, the possibility of activating/deactivating online different control algorithms in a seamless way, the possibility of adding new control algorithms in a straightforward way, and code efficiency.
- Published
- 2020
44. A three-dimensional software framework for environmental system monitoring and decision support in Poyang lake basin
- Author
-
Karsten Rink, Gang Zhao, Lars Bilke, Changqing Yan, Tianxiang Yue, and Olaf Kolditz
- Subjects
Decision support system ,010504 meteorology & atmospheric sciences ,Land use ,business.industry ,Computer science ,Environmental resource management ,Terrain ,010502 geochemistry & geophysics ,computer.software_genre ,01 natural sciences ,Environmental data ,Visualization ,Software framework ,Software ,Data visualization ,General Earth and Planetary Sciences ,business ,computer ,0105 earth and related environmental sciences - Abstract
With the development of remote sensing and large-scale environmental modelling, large amount of environmental data are continuously becoming available. An intuitive and comprehensive visualization of these data could facilitate data exploration, communication and collaboration between the stakeholders for informed decisions making. In Poyang lake basin regions, we demonstrate how to develop a software platform that can visualize three-dimensionally environmental data layers including terrain, weather, river net, water level, land use changes and interrelations between these data layers for environmental system monitoring and decision supporting. The tool is built by combining several prevailing projects including Unity, Paraview, and hydrological models, etc. We develop an open standardized framework for the software tool to host environmental data layers permitting the application of the tool, which could be employed for the intuitive and comprehensive data visualization in other regions facing environmental challenges.
- Published
- 2020
45. The early history of F#
- Author
-
Don Syme
- Subjects
Functional programming ,Object-oriented programming ,Java ,Computer science ,business.industry ,Programming language ,020207 software engineering ,02 engineering and technology ,computer.software_genre ,Object (computer science) ,Software framework ,Software ,020204 information systems ,0202 electrical engineering, electronic engineering, information engineering ,Pattern matching ,Safety, Risk, Reliability and Quality ,business ,computer ,Period (music) ,computer.programming_language - Abstract
This paper describes the genesis and early history of the F# programming language. I start with the origins of strongly-typed functional programming (FP) in the 1970s, 80s and 90s. During the same period, Microsoft was founded and grew to dominate the software industry. In 1997, as a response to Java, Microsoft initiated internal projects which eventually became the .NET programming framework and the C# language. From 1997 the worlds of academic functional programming and industry combined at Microsoft Research, Cambridge. The researchers engaged with the company through Project 7, the initial effort to bring multiple languages to .NET, leading to the initiation of .NET Generics in 1998 and F# in 2002. F# was one of several responses by advocates of strongly-typed functional programming to the "object-oriented tidal wave" of the mid-1990s. The development of the core features of F# 1.0 happened from 2004-2007, and I describe the decision-making process that led to the "productization" of F# by Microsoft in 2007-10 and the release of F# 2.0. The origins of F#'s characteristic features are covered: object programming, quotations, statically resolved type parameters, active patterns, computation expressions, async, units-of-measure and type providers. I describe key developments in F# since 2010, including F# 3.0-4.5, and its evolution as an open source, cross-platform language with multiple delivery channels. I conclude by examining some uses of F# and the influence F# has had on other languages so far.
- Published
- 2020
46. Computer vision algorithms acceleration using graphic processors NVIDIA CUDA
- Author
-
Yahia Said, Mouna Afif, and Mohamed Atri
- Subjects
Computer Networks and Communications ,Computer science ,020206 networking & telecommunications ,02 engineering and technology ,Parallel computing ,computer.software_genre ,Image (mathematics) ,Software framework ,Acceleration ,CUDA ,0202 electrical engineering, electronic engineering, information engineering ,Parallelism (grammar) ,020201 artificial intelligence & image processing ,Central processing unit ,General-purpose computing on graphics processing units ,computer ,Implementation ,Software ,ComputingMethodologies_COMPUTERGRAPHICS - Abstract
Using graphic processing units (GPUs) in parallel with central processing unit in order to accelerate algorithms and applications demanding extensive computational resources has been a new trend used for the last few years. In this paper, we propose a GPU-accelerated method to parallelize different Computer vision tasks. We will report on parallelism and acceleration in computer vision applications, we provide an overview about the CUDA NVIDIA GPU programming language used. After that we will dive on GPU Architecture and acceleration used for time consuming optimization. We introduce a high-speed computer vision algorithm using graphic processing unit by using the NVIDIA’s programming framework compute unified device architecture (CUDA). We realize high and significant accelerations for our computer vision algorithms and we demonstrate that using CUDA as a GPU programming language can improve Efficiency and speedups. Especially we demonstrate the efficiency of our implementations of our computer vision algorithms by speedups obtained for all our implementations especially for some tasks and for some image sizes that come up to 8061 and 5991 and 722 acceleration times.
- Published
- 2020
47. RESEARCH AND DESIGN OF INSPECTION CLOUD PLATFORM FRAMEWORK FOR SURVEYING AND MAPPING PRODUCTS
- Author
-
Z. Li, L. B. Zhang, and Han Y. H. Chen
- Subjects
lcsh:Applied optics. Photonics ,lcsh:T ,business.industry ,Computer science ,Big data ,lcsh:TA1501-1820 ,Cloud computing ,computer.software_genre ,lcsh:Technology ,Software framework ,Spatial query ,Software ,Code refactoring ,lcsh:TA1-2040 ,Distributed data store ,Systems engineering ,lcsh:Engineering (General). Civil engineering (General) ,business ,computer - Abstract
With the continuous improvement of modern surveying and mapping technology and with the plentiful of achievements, traditional quality inspection software for single machine, single task and single data type, difficult to massive multi-source isomerization achievements, difficult to meet the requirement of rapid, accurate and efficient quality inspection. With the development of IT technology such as cloud computing, big data and artificial intelligence, the quality inspection software needs to combine cloud computing technology with quality inspection business, refactoring software framework. Facing to the storage and spatial query requirement of inspection for surveying and mapping products, the paper researches and designs the spatial data distributed storage and the spatial data distributed index in cloud platform. The Management of inspection rule is the core in cloud platform. Inspection rule is the minimum operating independent unit, which becomes inspection item by parameterization, the paper builds full run-time operating mechanism in cloud platform for inspection rule. Finally, Combining the inspection requirement for surveying and mapping products and business, the paper researches and design the cloud framework for surveying and mapping products.
- Published
- 2020
48. BornAgain: software for simulating and fitting grazing-incidence small-angle scattering
- Author
-
Celine Durniak, J.M. Carmona Loaiza, W. Van Herck, Marina Ganeva, Jonathan Fisher, Jan Burle, Dmitry Yurov, Gennady Pospelov, and Joachim Wuttke
- Subjects
Computer science ,Astrophysics::High Energy Astrophysical Phenomena ,02 engineering and technology ,Neutron scattering ,010402 general chemistry ,computer.software_genre ,01 natural sciences ,General Biochemistry, Genetics and Molecular Biology ,Computational science ,Computer Programs ,Condensed Matter::Materials Science ,Software ,grazing-incidence small-angle scattering (GISAS) ,Reflectometry ,ComputingMethodologies_COMPUTERGRAPHICS ,computer.programming_language ,business.industry ,Scattering ,software ,neutron scattering ,Python (programming language) ,X-ray scattering ,021001 nanoscience & nanotechnology ,simulation ,0104 chemical sciences ,Software framework ,ddc:540 ,Neutron reflectometry ,Small-angle scattering ,0210 nano-technology ,business ,computer - Abstract
BornAgain is a free and open-source multi-platform software framework for simulating and fitting X-ray and neutron reflectometry, off-specular scattering, and grazing-incidence small-angle scattering (GISAS). This paper concentrates on GISAS., BornAgain is a free and open-source multi-platform software framework for simulating and fitting X-ray and neutron reflectometry, off-specular scattering, and grazing-incidence small-angle scattering (GISAS). This paper concentrates on GISAS. Support for reflectometry and off-specular scattering has been added more recently, is still under intense development and will be described in a later publication. BornAgain supports neutron polarization and magnetic scattering. Users can define sample and instrument models through Python scripting. A large subset of the functionality is also available through a graphical user interface. This paper describes the software in terms of the realized non-functional and functional requirements. The web site https://www.bornagainproject.org/ provides further documentation.
- Published
- 2020
49. Extended validation and verification of XPS/AVL-Fire™, a computational CFD-DEM software platform
- Author
-
Dalibor Jajcevic, Johannes Khinast, Thomas Forgber, Peter Toson, Hermann Kureck, and Stefan Madlmeir
- Subjects
Class (computer programming) ,Computer science ,business.industry ,General Chemical Engineering ,Control engineering ,02 engineering and technology ,Experimental validation ,Numerical models ,021001 nanoscience & nanotechnology ,computer.software_genre ,Outcome (game theory) ,Software framework ,Software ,020401 chemical engineering ,Thermal coupling ,0204 chemical engineering ,0210 nano-technology ,business ,computer ,CFD-DEM - Abstract
The goal of this work is an comprehensive experimental validation and verification of a computational CFD-DEM software platform. The software platform's modules and performance requirements specifically address the needs of pharmaceutical industry in terms of batch size (i.e., number of particles) and related phenomena (i.e., physical complexity). Moreover, we critically assessed the numerical models implemented. In addition to the review of relevant literature and selection of validation experiments, novel analytical solutions for the spray class in question are presented. As the presented method is independent of the spray model, the solution can be used to prove the correct implementation of various available spray models. The final outcome of this work is a validated software framework, which can be further used to investigate large-scale processes by resolving single particle trajectories in a coupled environment, including heat, mass and momentum transfer.
- Published
- 2020
50. Towards a Novel Framework for Automatic Big Data Detection
- Author
-
Hameeza Ahmed and Muhammad Ali Ismail
- Subjects
Big data (3Vs) ,General Computer Science ,Computer science ,Big data ,detection ,02 engineering and technology ,computer.software_genre ,Software portability ,Software ,020204 information systems ,0202 electrical engineering, electronic engineering, information engineering ,General Materials Science ,Code generation ,Programmer ,business.industry ,General Engineering ,Usability ,Software framework ,Identification (information) ,machine learning ,LLVM ,020201 artificial intelligence & image processing ,lcsh:Electrical engineering. Electronics. Nuclear engineering ,Data mining ,business ,lcsh:TK1-9971 ,computer - Abstract
Big data is a ”relative” concept. It is the combination of data , application , and platform properties. Recently, big data specific technologies have emerged, including software frameworks, databases, hardware accelerators, storage technologies, etc. However, the automatic selection of these solutions for big data computations remains a non-trivial task. Presently, the big data tools are selected by analyzing the problem manually, or by using several performance prediction techniques. The manual identification is based on the data properties only, whereas the performance predictors only estimate basic execution metrics without linking them with big data (3Vs) thresholds. Hence, both ways of identification are mostly incorrect, which can lead to inefficient use of 3Vs optimizations, resulting into global inefficiency, reduced system performance, increasing power consumption, requiring greater effort on the part of the programming team, and misallocation of the hardware resources required for the task. In this regard, a novel framework has been proposed for automatic detection of 3Vs (Volume, Velocity, Variety) of big data, using machine learning. The detection is done through static code features , data , and platform properties , leading to relevant tool selection, and code generation, with minimal overheads, lesser programmer interventions, higher usability, and portability. Instead of handling each application with big data specialized solutions, or manually identifying the 3Vs, the framework can automatically detect and link the 3Vs to the relevant optimizations. Several standard applications have been tested using the proposed framework. In the case of volume, the average detection accuracy is up to 97.8% for seen and 95.9% for unseen applications. In the case of velocity, the average detection accuracy is up to 97.3% for seen and 92.6% for unseen applications. There is no margin of error in variety detection, as it has straightforward computations without any predictions. Furthermore, an airline recommendation system case study strengthens the effectiveness of the proposed approach.
- Published
- 2020
Catalog
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.