Search

Guyana's capacity to address the impacts of climate change on its coastal environment requires the ability to monitor, quantify and understand coastal change over short-, medium- and long- term. Understanding the drivers of change in coastal and marine environment can be achieved through the accurate measurement and critical analyses of morphologies, flows, processes and responses. This manuscript presents a strategy developed to create a central resource, database and web-based platform to integrate data and information on the drivers and the changes within Guyana coastal and marine environment. The strategy involves four complimentary work packages including data collection, development of a platform for data integration, application of the data for coastal change analyses and consultation with stakeholders. The last aims to assess the role of the integrated data systems to support strategic governance and sustainable decision-making. It is hoped that the output of this strategy would support the country's climate-focused agencies, organisations, decision-makers, and researchers in their tasks and endeavours.

The emerging area of the internet of things (IoT) generates a large amount of data from IoT applications such as health care, smart cities, etc. This data needs to be analyzed in order to derive useful inferences. Machine learning (ML) plays a significant role in analyzing such data. It becomes difficult to select optimal algorithm from the available set of algorithms/classifiers to obtain best results. The performance of algorithms differs when applied to datasets from different application domains. In learning, it is difficult to understand if the difference in performance is real or due to random variation in test data, training data, or internal randomness of the learning algorithms. This study takes into account these issues during a comparison of ML algorithms for binary and multivariate classification. It helps in providing guidelines for statistical validation of results. The results obtained show that the performance measure of accuracy for one algorithm differs by critical difference (CD) than others over binary and multivariate datasets obtained from different application domains.

Policy document mention is considered to indicate the significance and societal impact of scientific product. However, the accuracy of policy document altmetrics data needs to be evaluated to fully understand its strength and limitation. An in-depth coding analysis was conducted on sample policy documents records of Altmetric.com database. The sample consists of 2079 records from all 79 distinct policy document source platforms tracked by the database. Errors about mentioned publications in the policy documents (type A error) are found in 8% of the records, while errors about either the recorded policy documents or the mentioned publications in the altmetrics database (type B error) are found in 70% of the records. In type B error, policy document link error (5% of the records) could be attributable to the policy document website, transcription error (52% of the records) could be attributable to the third-party bibliographic data provider. These two categories of error are relatively minor and may have limited influence on altmetrics research and practices. False positive policy document mention (13% of the records), however, could be attributable to the Altmetric database and may diminish the validity of research based on the policy document altmetrics data. The underlying reasons remain to be further investigated. Considering the high complexity of extracting mentions of publications from various sources and formats of policy documents as well as its short history, Altmetric database has achieved excellent performance.

Moving objects such as people, animals, and vehicles have generated a large amount of spatiotemporal data by using location-capture technologies and mobile devices. This collected data needs to be processed, visualized and analyzed to transform raw trajectory data into useful knowledge. In this study, we build a system to deliver a set of traffic insights and recommendations by applying two techniques, clustering, and sequential pattern mining. This system has three stages, the first stage preprocesses and samples the dataset into 168 subsets, the second stage applies two clustering techniques, the hierarchical density-based spatial clustering (HDBSCAN) and the Random Swap clustering (RS). We compare these two clustering algorithms in terms of processing time and quality of clusters. In the comparative analysis, the Silhouette coefficient shows that RS clustering outperforms HDBSCAN in terms of clusters quality. Moreover, the analysis shows that RS outperforms K-means in terms of the mean of square error (MSE) reduction. After that, we use a Google Maps approach to label the traffic districts and apply sequential pattern mining to extract taxi trips flow. The system can detect 146 sequential patterns in different areas of the city. In the last stage, we visualize traffic clusters generated from the RS algorithm. Furthermore, we visualize the taxi trips heatmap per weekday and hour of the day in Porto city. This system can be integrated with the current traffic control applications to provide useful guidelines for taxi drivers, passengers, and transportation authorities.

India is a country with thousands of languages and dialects spoken across a billion-strong population. For multi-lingual content creation and accessibility, text-to-speech systems will play a crucial role. However, the current neural TTS systems are data-hungry and need about 20 hours of clean single-speaker speech data for each language and speaker. This is not scalable for the large number of Indian languages and dialects. In this work, we demonstrate three simple, yet effective pre-training strategies that allow us to train neural TTS systems with just about one-tenth of the data needs while also achieving better accuracy and naturalness. We show that such pre-trained neural TTS systems can be quickly adapted to different speakers across languages and genders with less than 2 hours of data, thus significantly reducing the effort for future expansions to the thousands of rare Indian languages. We specifically highlight the benefits of multi-lingual pre-training and its consistent impact across our neural TTS systems for 8 Indian languages.

54th CIRP Conference on Manufacturing Systems 2021 ��� "Towards Digitalized Manufacturing 4.0", CIRP-CMS 2021, online, 22 Sep 2021 - 24 Sep 2021; Procedia CIRP 104, 50-55 (2021). doi:10.1016/j.procir.2021.11.009 special issue: "54th CIRP CMS 2021 - Towards Digitalized Manufacturing 4.0 / Edited by Dimitris Mourtzis", Published by Elsevier, Amsterdam

Online healthcare is the next big thing and proper security mechanisms with privacy preservation techniques for sensitive data is the need of the hour. Handling sensitive datasets such as medical data needs ultimate security which covers confidentiality, integrity, authentication, and reversibility. This paper presents a novel approach for authentication using reversible data hiding (RDH) technique. Traditional RDH methods provide adequate security for sensitive image. The proposed RDH technique uses a combination of reversible data marking techniques in multiple levels to provide a robust authentication measure for medical images. Least significant bit (LSB) modification works as the base methodology for data marking to ensure complete reversibility.

This research describes some of the results from an unsupervised ER process using cluster entropy as a way to self-regulate linking. The experiments were performed using synthetic person references of varying quality. The process was able to obtain a linking accuracy of 93% for samples with moderate to high data quality. While results for low-quality references were much lower, there are many possible avenues of research that could further improve the results from this process. The purpose of this research is to allow ER processes to self-regulate linking based on cluster entropy. The results are very promising for entity references of relatively high quality; using this process for low-quality data needs further improvement. The best overall result obtained from the sample was just over 50% linking accuracy.

A common privacy issue in traditional machine learning is that data needs to be disclosed for the training procedures. In situations with highly sensitive data such as healthcare records, accessing this information is challenging and often prohibited. Luckily, privacy-preserving technologies have been developed to overcome this hurdle by distributing the computation of the training and ensuring the data privacy to their owners. The distribution of the computation to multiple participating entities introduces new privacy complications and risks. In this paper, we present a privacy-preserving decentralised workflow that facilitates trusted federated learning among participants. Our proof-of-concept defines a trust framework instantiated using decentralised identity technologies being developed under Hyperledger projects Aries/Indy/Ursa. Only entities in possession of Verifiable Credentials issued from the appropriate authorities are able to establish secure, authenticated communication channels authorised to participate in a federated learning workflow related to mental health data., Comment: MDPI Mach. Learn. Knowl. Extr. 2021, 3(2), 333-356; https://doi.org/10.3390/make3020017

The cheap availability of portable devices and processing elements has triggered the emergence of IOT. It is an integration of powerful computers with large number of tiny devices that support automation and smart decision making in all walks of life that may be agriculture, battle field, industry, office or home. Thus, IOT has very wide range of applications. However, the success of such applications heavily depends on efficient handling of numerous challenges related to computation and communication because huge volume of data needs to be aggregated, transported and processed at faster rate. We illustrate the potential of IOT and various technologies used to realize it and reap benefits. The paper also makes a mention of a futuristic application which is a wearable device that assists to protect victims from various threats.

Background Data from the 1000 Genomes project is quite often used as a reference for human genomic analysis. However, its accuracy needs to be assessed to understand the quality of predictions made using this reference. We present here an assessment of the genotyping, phasing, and imputation accuracy data in the 1000 Genomes project. We compare the phased haplotype calls from the 1000 Genomes project to experimentally phased haplotypes for 28 of the same individuals sequenced using the 10X Genomics platform. Results We observe that phasing and imputation for rare variants are unreliable, which likely reflects the limited sample size of the 1000 Genomes project data. Further, it appears that using a population specific reference panel does not improve the accuracy of imputation over using the entire 1000 Genomes data set as a reference panel. We also note that the error rates and trends depend on the choice of definition of error, and hence any error reporting needs to take these definitions into account. Conclusions The quality of the 1000 Genomes data needs to be considered while using this database for further studies. This work presents an analysis that can be used for these assessments. Electronic supplementary material The online version of this article (10.1186/s12864-019-5957-x) contains supplementary material, which is available to authorized users.

The huge amount of data that is committed to the databases, takes more time, than the time that is estimated which further creates trouble for database transactions. To untighten this process, data needs to be clustered based on common attributes, to commit data faster. Databases like Hadoop have structures like MapReduce, which are incompetent to handle the process alone. As a result, the accumulation of these transactions eventually slows down the process as well as adversely affects the performance of the application that creates havoc at the user end. The proposed system puts forth the idea of decomposing this data to hyper graphs based on correlating features between them. Furthermore, the use of Hungarian and Genetic Algorithm, achieves optimized transaction time.

In a big data environment, it is often necessary to ingest data from different sources into a unique storage. Because of low memory price, system distribution and failure tolerance, that storage is typically HDFS. It enables users to manipulate data with different tools from the Hadoop ecosystem. The process of data ingestion seems simple. However, because sources can be different database systems, structured, semi-structured and unstructured data complicate the ingestion procedure. It is usually not enough to just store everything. Data needs to be stored in such a way that enables users to quickly access and manipulate it. There are many ingestion-specific solutions in the big data ecosystem. This paper will describe an implemented system for data ingestion from MSSQL, MySQL and Postgres into a Hive database. The process starts with creating tables with corresponding metadata, continues with the ingestion process and ends with a description of how the process is automated. The implementation of Sqoop as an open-source tool and Hue, a web user interface from Cloudera, will be described.

In view of the rapidly growing number of synthesized nanoparticles as well as public concerns about their potential negative impacts on human health and the environment, there is an urgent need to address current risk assessment data gaps. Thus, the development of comprehensive computational methods (e.g., read-across methods) for filling data gaps that meet realistic data needs is crucial. The present study proposes a new quantitative read-across approach based on linear algebra (i.e., one/two-point-slope formula) and one of the most widely used unsupervised pattern recognition methods (i.e., principal component analysis). The applicability and usefulness of the newly developed read-across algorithm for pre-screening hazard assessment of nanomaterials are confirmed by using three literature nanotoxicity datasets. The findings from this study clearly indicate that the proposed read-across approach provides reasonably accurate and statistically significant results of estimations of nanotoxicity data. Therefore, the method can be used for prioritizing current and future nanoparticles for the purpose of further testing and risk assessment.

To utilize optical techniques for bio-medical applications, e.g. disease diagnostics, the data needs to be analyzed using machine learning. This translation requires automatic data pipelines presented in this contribution for selected application.

The emergence of more complex, data-intensive applications motivates a high demand of effective data modeling for graph databases to support efficient query answering. In this paper, we develop an intuitive graph data model for dynamic taxi ride sharing. We argue that our proposed data model meets the data needs imposed by three fundamental tasks associated with taxi ride sharing. An experiment consisting of a taxi ride sharing simulation with real-world data demonstrates the effectiveness of our modelling approach.

There is an increasing demand for real-time analysis of large volumes of data streams that are produced at high velocity. The most recent data needs to be processed within a specified delay target in order for the analysis to lead to actionable result. To this end, in this paper, we present an effective solution for detecting the correlation of such data streams within a micro-batch of a fixed time interval. Our solution, coined DCS, for Detection of Correlated Data Streams, combines (1) incremental sliding-window computation of aggregates, to avoid unnecessary re-computations, (2) intelligent scheduling of computation steps and operations, driven by a utility function within a micro-batch, and (3) an exploration policy that tunes the utility function. Specifically, we propose nine policies that explore correlated pairs of live data streams across consecutive micro-batches. Our experimental evaluation on a real world dataset shows that some policies are more suitable to identifying high numbers of correlated pairs of live data streams, already known from previous micro-batches, while others are more suitable to identifying previously unseen pairs of live data streams across consecutive micro-batches.

Data is Information that has not been processed or manipulated, which is not enough for some users. So that the info is useful, the data needs to be prepared and integrated first to become Information. Universitas Muhammadiyah Yogyakarta has a tool to monitor data on Alumni, namely Tracer Study. The data currently available is still not integrated and processed correctly. Now there is a solution for processing data and data by making base data. The purpose of creating this database is to facilitate users in checking data, data that will be stored neatly and integrated into the computer. The method in this study uses the Single DDS method. Methodology This research starts from determining the subject of investigation, primary needs, creating baseline data, analyzing results. The backstage of need is carried out with two data analyzes, and interviews with LPKA (Lembaga Pengelola Kemahasiswaan dan Alumni) or Students and Alumni Management Institutions. Baseline data making in this study is single DDS, which has two data stores, namely Stage and DDS. Data analysis is done by creating reports with Microsoft Office Excel according to user needs. The testing phase is carried out with two processes, namely: ETL Testing and Functional Testing.

Data mining is the current buzz of the data processing world. There are large number of applications which are producing the large amount of data. This data need to be processed for the extraction of the analyzed data. So that this analysis can be used in the decision making purpose in relevant organizations. But the data collected from the row sources are of imbalance in nature. This data needs to be processed to generate the balancing dataset. Because processing imbalance data will be highly in efficient process. There are various types of discrepancies lies into the dataset. In current research paper these different types of discrepancies are to be neutralized. So that processing and analysis of the dataset will become easy and more efficient. These discrepancies lies into the dataset are like small disjuncts, lack of density in data, overlapping between the classes, impact of noisy data, the borderline etc. using different techniques these problems into the dataset are being neutralized. So that balanced data can be inputted for the data mining purpose.

It is popular to use real-world data to evaluate data mining techniques. However, there are some disadvantages to use real-world data for such purposes. Firstly, real-world data in most domains is difficult to obtain for several reasons, such as budget, technical or ethical. Secondly, the use of many of the real-world data is restricted, those data sets do either not contain specific patterns that are easy to mine or the data needs special preparation and the algorithm needs very specific settings in order to find patterns in it. The solution to this could be the generation of synthetic, "meaningful data" (data with intrinsic patterns). This paper presents a novel approach for generating synthetic data by developing a tool, including novel algorithms for specific data mining patterns, and a user-friendly interface, which is able to create large data sets with predefined classification rules, multilinear regression patterns. A preliminary run of the prototype proves that the generation of large amounts of such "meaningful data" is possible. Also the proposed approach could be extended to a further development for generating synthetic data with other intrinsic patterns.

International audience; Frequent Itemsets mining is a key concept in Association Rule Mining task, it aims to discover the frequent itemsets in a transactional dataset.Nowadays large amounts of data needs to be analysed, thus the use of traditional approaches for mining frequent itemsets suffer from computational time and memory requirement, beside the difficulty for the user to provide an appropriate minimum support value. For these reasons, we propose to tackle the problem of mining frequent itemsets by mean of Chemical Reaction Optimization metaheuristic without the need to specify minsup threshold . The results show that the proposed approach gives good results and it can be used as alternative to mine frequent itemsets.

Due to the limited computational resources of data owners and the developments in cloud computing, more and more data owners choose to store data on the cloud to reduce their own storage burden. When the data owner wants to re-select a new cloud service provider, the data needs to be transmitted from Cloud A to Cloud B. How to ensure that data has been safely transferred to Cloud B and Cloud A honestly implementing data deletion has become a hot topic for many scholars. To solve these problems, Liang et al. proposed a provable data transfer protocol based on provable data possession and deletion for secure cloud storage. However, we find a security flaw in their scheme. In this paper, we first review their scheme and then present our attack in detail. Finally, we propose an improved scheme which can resist the attack well.

Geospatial technologies have transformed the way we visualize and understand social phenomena and physical environments. There are significant advantages in using these technologies and data however, their use also presents ethical dilemmas such as privacy and security concerns as well as the potential for stigma and discrimination resulting from being associated with particular locations. Therefore, the use of geospatial technologies and resulting data needs to be critically assessed through an ethical lens prior to implementation of programmes, analyses or partnerships. This paper examines the benefits, risks and ethical considerations when undertaking evidence generation using geospatial technologies. It is supplemented by a checklist that may be used as a practical tool to support reflection on the ethical use of geospatial technologies.

Every student in all Universities who is already at the final level of course will report the results in the form of presentation in the presence of mentors and examiners. Students will usually wait for a schedule that has been determined by the coordinator in the Faculty. This scheduling sometimes makes students anxious because they have to wait until the time is appropriate and often they don’t get a suitable schedule. This is also happen with the students in Faculty of Information Technology, Maranatha Christian University. The coordinators often experience difficulties in the allocation of mentors or examiners to be scheduled for council. In this research will make an application that help in conducting scheduling of the council, so that the council process can held smoothly and students will not be afraid to miss the schedule. Applications will be built using JavaFX desktop programming, JSON type network services, and data reporting through JasperReport. Data were collected in literature and interviews with related parties. Applications will accommodate starting from data entry, role assignment determination, scheduling and reporting. Applications use data exchange methods through network services that facilitate the development of data needs. This prototype has been tested by black box method. The test results state that this prototype works through testing with all of the possibilities.

Recent water quality decrease has caused difficult in finding clean water source for people and their daily life. Monitoring on water quality had been carried out many times, from up stream to down stream. It’s necessary to do Online Monitoring on ground and underground water quality continuously, so that the effect of water quality decrease could be detected earlier and handle directly. The output of water quality data needs to be processed so that the society and the decision makers could see the information publicly. So, we need a design of structured database of online and real-time water quality data processing. Water quality data management using structured data base system could make water source data retracing easier. Katakunci : database struktur, online monitoring, real time monitoring

Missing traffic data are inevitable due to detector failure or communication failure. Currently, most of imputation methods estimated the missing traffic values by using spatial-temporal information as much as possible. However, it ignores an important fact that spatial-temporal information of the traffic missing data is often incomplete and unavailable. Moreover, most of the existing methods are verified by traffic data from freeway, and their applicability to urban road data needs to be further verified. In this paper, a hybrid method for missing traffic data imputation is proposed using FCM optimized by a combination of PSO algorithm and SVR. In this method, FCM is the basic algorithm and the parameters of FCM are optimized. Firstly, the patterns of missing traffic data are analyzed and the representation of missing traffic data is given using matrix-based data structure. Then, traffic data from urban expressway and urban arterial road are used to analyze spatial-temporal correlation of the traffic data for the determination of the proposed method input. Finally, numerical experiment is designed from three perspectives to test the performance of the proposed method. The experimental results demonstrate that the novel method not only has high imputation precision, but also exhibits good robustness.

Indoor positioning systems answer the need for ubiquitous localisation systems. Frequently, indoor positioning relies on machine learning models developed based on the training data composed of WiFi received signal strength (RSS) vectors observed in different indoor locations. However, this requires expensive collection of RSS vectors in precisely measured locations. In this study, we propose a semi-supervised method, which can reduce the volume of expensive labelled training data and exploit the availability of unlabelled signal strength measurements. The method relies, inter alia, on the measures of similarity among nearest neighbours of unlabelled vectors. Tests performed with a number of testbed areas confirm that the method improves the accuracy of random forest models used to estimate indoor location of mobile terminals.

Curriculum learning is a learning technique in which a classifier learns from easy samples first and then from increasingly difficult samples. On similar lines, a curriculum based feature selection framework is proposed for identifying most useful features in a dataset. Given a dataset, first, easy and difficult samples are identified. In general, the number of easy samples is assumed larger than difficult samples. Then, feature selection is done in two stages. In the first stage a fast feature selection method which gives feature scores is used. Feature scores are then updated incrementally with the set of difficult samples. The existing feature selection methods are not incremental in nature; entire data needs to be used in feature selection. The use of curriculum learning is expected to decrease the time needed for feature selection with classification accuracy comparable to the existing methods. Curriculum learning also allows incremental refinements in feature selection as new training samples become available. Our experiments on a number of standard datasets demonstrate that feature selection is indeed faster without sacrificing classification accuracy.

The improvement in the technology and the desperate need to store huge data has been increasing steadily. Hadoop uses MapReduce framework for processing big datasets, which consists of namenode and datanode. The hadoop framework technology has been extensively used to store, process, and use the enormous data that are being placed into server. Handling of huge small files is becoming laborious since the namenode has to manage the filenames and its corresponding metadata. For the purpose of fault tolerance, the data needs to be replicated on the data nodes. In order to handle the mammoth number of small files and to reduce the out of memory burden on the namenode, several techniques are being approached which involves HDFS, EHDFS, SFS, HAR, NHAR. In this paper we discuss about the different techniques that are developed to handle the storage of small files and propose a new method to solve the storage issue in hadoop. The new method takes the priority of the file as a distinguishing factor which further helps in reducing the memory usage of namenode.

Decision making and business support is an important process in data mining and this can be achieved by means of pattern classification and extraction. Since the huge volume of data needs starving knowledge to process and organisation faces many issues in solving those issues. Clustering is an effective technology available to analyse and convert the datasets into meaningful patterns. Clustering in data mining uses various attributes to compute large dataset and meet out the real time issues. The proposed model uses Bayesian hierarchical clustering model with Dirichlet model to resolve the issues in large dataset analysis. Experimental results prove that proposed model experience better clustering efficiency than conventional complete link agglomerative clustering by achieving 92% of clustering accuracy.

In this paper, on the basis of the data warehouse integration mechanisms, use Object-oriented analysis methods, take the ITIS’s functions and data needs as the starting point, promote and optimize the ETL method and form a Tourism-oriented ITIS data integration model. The “wisdom tourism” is explained elaborately. Then it shows what data integration is and how to use integration.

Construct validation is an important step in the formulation of theory and its testing. Any information infers from the construct and its related hypothesis, without the validation, might be misleading the decision. In the literature, the construct validation guidelines for primary data are available but the exponential usage of secondary data mining approaches in recent times has created its need for secondary data users. However, it is a less addressed domain, specifically for medical sciences due to underdeveloped theoretical foundations of the field. Usually, an assessment of the validity of the secondary data is not performed because primary users already use validation process especially in the tool development phase. Further, secondary data users are more concerned with data mining that is a data-driven approach rather than theoretical knowledge of the field. Therefore, if researchers want to explore the hidden structures of data and try some different combinations of items which were not tested by primary users, then they should validate this newly explored group of items for the creation of knowledge of the particular field. In this paper, a guideline of the construct validation process for secondary data users with its practical issues especially validity and reliability coefficients is discussed. This paper concludes that the construct validation based on secondary data needs special attention of data mining experts and medical researchers while designing the studies to maximize the benefits of the secondary data.

Social networks provide a large amount of social network data, which is gathered and released for various purposes. Since social network data usually contains much sensitive information of individuals, the data needs to be anonymized before releasing. To protect privacy of individuals in released social network, many anonymizing methods have been proposed. However, most of them were proposed for general purpose, and suffered the over-information loss problem when they were used for specific purposes. In this paper, we focus on the problem of preserving structure information in anonymized social network data, which is the most important knowledge for community analysis. Furthermore, we propose a novel local-perturbation technique that can reach the same privacy requirement of k-anonymity, while minimizing the impact on community structure. We evaluate the performance of our method on real-world data. Experimental results show that our method has less community structure information loss compared with existing techniques.

dbpedia data is largely generated from extracting and parsing the wikitext from the infoboxes of Wikipedia. This generation process is handled by the dbpedia Extraction Framework (dbpedia ef). This framework currently consists of data transformations, a series of custom hard-coded steps which parse the wikitext, and schema transformations, which model the resulting rdf data. Therefore, applying changes to the resulting rdf data needs both Semantic Web expertise and development within the dbpedia ef. As such, the current dbpedia data is being shaped by a small amount of core developers. However, by describing both schema and data transformations declaratively, we shape and generate dbpedia data using solely declarations, splitting the concerns between implementation and modeling. The parsing functions development is decoupled from the dbpedia ef, and other data transformation functions can easily be integrated during dbpedia data generation. This demo showcases an interactive Web application that allows non-technical users to (re-)shape the dbpedia data and use external data transformation functions, solely by editing a mapping document via html controls.

With the availability of large volumes of multimedia data in P2P file sharing systems, locating data needs to address a multi-modal retrieval. In this approach, the paper proposes to make use of indexing as a mechanism for organizing files and the use of distributional features for searching of a file using natural language text. Then it trains the nodes onto a P2P network using Backpropagation algorithm and also make use of a heuristic search technique. Experimental results show improved efficiency in providing fast and accurate answers in P2P systems.

With the advancement of technology, today's world is a digital world where digital technology is generating new world of possibilities and opportunities and we are abandoned with digital data over the network that needs to be exchanged with many organisations, devices and users. This data needs to be a secure data and malware is one type of threat to this data. There are many forms of malware that can damage this sensitive data. This paper includes various malware detection techniques to detect various known and unknown binaries and presents a detailed analysis of current methods of malware detection and a malware analyser tools.

Data mining techniques have been widely used in many research disciplines such as medicine, life sciences, and social sciences to extract useful knowledge (such as mining models) from research data. Research data often needs to be published along with the data mining model for verification or reanalysis. However, the privacy of the published data needs to be protected because otherwise the published data is subject to misuse such as linking attacks. Therefore, employing various privacy protection methods becomes necessary. However, these methods only consider privacy protection and do not guarantee that the same mining models can be built from sanitized data. Thus the published models cannot be verified using the sanitized data. This article proposes a technique that not only protects privacy, but also guarantees that the same model, in the form of decision trees or regression trees, can be built from the sanitized data. We have also experimentally shown that other mining techniques can be used to reanalyze the sanitized data. This technique can be used to promote sharing of research data.

It is possible to obtain accurate temperature and salinity profiles of the oceans using a SBE 911plus CTD and accompanying data conversion packages. To obtain highly accurate results, CTD data needs to be carefully processed in addition to proper and regular maintenance of the CTD itself. Since the manufacturer of the CTD provides tools that are necessary for post processing, it is possible to conduct proper processing without too much effort. Some users, however, are not familiar with all of the processes and inadvertently ignore some of these processes at the expense of data quality. To draw attention to these and other similar issues, we show how it is possible to improve data quality by utilizing a few extra processes to the standard or default data process procedures with CTD data obtained from the equatorial Eastern Pacific between 2001 and 2005, and 2007. One easy step that is often ignored in the standard data process procedure is “wild edit”, which removes abnormal values from the raw data. If those abnormal values are not removed, the abnormality could spread vertically during subsequent processes and produce abnormal salinity in a range much wider than that of the raw data. To remove spikes in salinity profiles the “align CTD” procedure must be carried out not with the default values included in the data processing software but with a proper time constant. Only when “cell thermal mass” correction is conducted with optimal parameters, we can reduce the difference between upcast and downcast, and obtain results that can satisfy the nominal accuracy of the CTD.

In Part 2 of this book, we will be introducing topics in order of appearance in the source code of our trading platform. This chapter will discuss acquisition, storage, and updating of data using free APIs. Our trading platform will contain various processes that run automatically throughout the day, and our data needs to be in our R environment for us to work with it. We may want to run R and RStudio to do unrelated jobs between those times, which means we need a way to store the data in files in our computer so that they can be fetched when the analysis starts. In this chapter, we will explore various ways to fetch, store, and load data. We will ultimately settle on the most efficient method to execute the following algorithm. This algorithm will be referred to as “the algorithm” throughout the chapter.

It is popular to use real-world data to evaluate or teach data mining techniques. However, there are some disadvantages to use real-world data for such purposes. Firstly, real-world data in most domains is difficult to obtain for several reasons, such as budget, technical or ethical. Secondly, the use of many of the real-world data is restricted or in the case of data mining, those data sets do either not contain specific patterns that are easy to mine for teaching purposes or the data needs special preparation and the algorithm needs very specific settings in order to find patterns in it. The solution to this could be the generation of synthetic, â??meaningful dataâ?? (data with intrinsic patterns). This paper presents a framework for such a data generator, which is able to generate datasets with intrinsic patterns, such as decision trees. A preliminary run of the prototype proves that the generation of such â??meaningful dataâ?? is possible. Also the proposed approach could be extended to a further development for generating synthetic data with other intrinsic patterns.

This article presents a method of steganographic keys generation for case of embedding many secrets in single container. The most important requirements of proposed algorithm are as follows. The generated keys cannot overlap what means that the positions they indicate in the cover have to be different. An authorized user with known key should be able to easily retrieve any secret from carrier file without necessity of extracting all other messages. Additionally secret data needs to be scattered in carrier randomly. The proposed algorithm can be applied to any method in which the capacity is fixed and can be expressed as a sequence of positions. It is suitable to use with multi-secret steganography in which many messages are embedded in one container, especially with techniques of substitution or cover lookup.

In this work we describe the steps required to render scientific data for real-time immersive displays. First, the input data needs to be taken into account for further processing. As a second step, the data needs to be reduced without losing significant information in order to enable real-time interaction. Therefore, we describe different reduction techniques and discuss advantages and disadvantages of those. As a third step, the data is integrated in a framework that is able to render the information on different display technologies. We elaborate the general workflow on a geoscientific dataset that illustrates the temperature distribution of the Earth's mantle.

Agencies have been working to improve their metering data collection, management, and analysis efforts over the last decade (since EPAct 2005) and will continue to address these challenges as new requirements and data needs come into place. Unfortunately there is no “one-size-fits-all” solution. As agencies continue to expand their capabilities to use metered consumption data to reducing resource use and improve operations, the hope is that shared knowledge will empower others to follow suit. This paper discusses the Federal metering data analysis needs and some existing tools.

In medical endoscopy more and more surgeons record videos of their interventions in a long-term storage archive for later retrieval. In order to allow content-based search in such endoscopic video archives, the video data needs to be indexed first. However, even the very basic step of content-based indexing, namely content segmentation, is already very challenging due to the special characteristics of such video data. Therefore, we propose to use instrument classification to enable semantic segmentation of laparoscopic videos. In this paper, we evaluate the performance of such an instrument classification approach. Our results show satisfying performance for all instruments used in our evaluation.

Many computer systems; especially in corporations, contain large amount of documents such as letters, reports and presentations. Many such documents are present in several versions. Such data needs to be synchronized with branch offices and mobile devices, often over slow and expensive connections. However, as many documents are stored in an already compressed format, it is difficult to compress them further by exploiting the hidden redundancies. We present a novel approach named RepoZip which improves the compression of an existing compression algorithm over a document collection, by exploiting the inter-document meta-data and content-level redundancies. It concentrates on compressing OOXML documents that have been constructed through the archival of a hierarchy of meta-data files and PDF documents which include deflated content streams. Therefore, the RepoZip approach achieves larger compression gains over OOXML document collections or PDF document collections by exploiting usually undetected meta-data level similarities.

Mining frequent patterns with a frequent pattern tree (FP-tree in short) avoids costly candidate generation and repeatedly occurrence frequency checking against the support threshold. It therefore achieves much better performance and efficiency than Apriori-like algorithms. However, the database still needs to be scanned twice to get the FP-tree. This can be very time-consuming when new data is added to an existing database because two scans may be needed for not only the new data but also the existing data. In this research we propose a new data structure, the pattern tree (P-tree in short), and a new technique, which can get the P-tree through only one scan of the database and can obtain the corresponding FP-tree with a specified support threshold. Updating a P-tree with new data needs one scan of the new data only, and the existing data does not need to be re-scanned. Our experiments show that the P-tree method outperforms the FP-tree method by a factor up to an order of magnitude in large datasets.

Research was done to evaluate the extent to which the pedestrian data collection efforts of transportation agencies in the United States are addressing pedestrian safety factors. There was also consideration of how pedestrian data collection can be improved to facilitate the monitoring of these factors. Fifteen pedestrian safety issues are identified based on a literature review and examination of pedestrian–vehicle crashes in Utah. A 2001 survey of U.S. transportation agencies indicated that 45 (75%) of the 60 respondents were counting pedestrians at various locations. Hand counting, the recording of push-button use, and video cameras were methods used to collect data. Automated systems, such as position sensors and image processing, were not used to count pedestrians. The use of advanced data collection technologies is not critical to the resolution of pedestrian safety concerns, although permanent counting installations might increase data collection efficiency. Only 4 of the 15 pedestrian safety issues were specifically being addressed by the agencies’ data collection efforts. Their existing methods could, however, be used to target seven additional safety factors. The development of a pedestrian data monitoring guide is recommended; an outline is proposed. Several agencies admitted that pedestrian volumes did not affect their pedestrian treatments. Evidently, some transportation agencies could benefit from direction on how to relate pedestrian demand and behavior data to safety improvements.

Systems chemical biology offers a novel way of approaching drug discovery by developing models that consider the global physiological environment of protein targets and their perturbations by drugs. However, the integration of all these data needs curation and standardization with an appropriate representation in order to get relevant interpretations. In this mini review, we present some databases and services, which integrated together with computational tools and data standardization, could assist scientists in decision making during the different drug development process.

This paper aims at the development and implementation of a methodology to extract specifications of products from HTML pages containing product details from various e-commerce portals. The extracted resultant data needs to be in a standardised uniform format without any reflection of its initial structure in source format. The most significant problem in designing a solution is the source of the data itself. Since the data is fetched from not just one but many different portals, the sheer variety of it is an obstacle as the format and structure vary for every single portal. The paper considers two subproblems of data available in structured as well as unstructured format. The methodology developed for structured data makes use of the information pattern contained in the underlying tree structure of the page's HTML content from which data is sourced in order to perform extraction. And pattern matching using regular expressions is the concept used for cases where data is unstructured. Implementation has been carried out using Python as the programming language with the usage of tools like Scrapy and LXML.