Search

Implementation of Master Data Management (MDM) in an organization aims to help the process of consolidating and integrating various sources of master data into a solitary truth source. MDM is also able to help to solve the complexity of data that occurs in the synchronization, consolidation and cleaning process of data from redundancy, and to increase the value of operational efficiency through the standardized content which represents business-based process. The goal of this study is to assess the level of MDM maturity at Secretariat of Presidential Advisory Council during the COVID-19 pandemic era. The assessment was done using Master Data Management Maturity Model (MD3M) by Spruit-Pietzka. The master data management maturity level in the organization was assessed using a questionnaire filled out by SMEs in a group discussion. The result of the assessment showed that MDM maturity level at Secretariat of Presidential Advisory Council is 1. There are 61.29% (38 of total 62) implemented capabilities. This means that organization already has awareness in the management of master data. The organization can improve its MDM maturity to a higher level by implementing the missing capabilities.

This paper describes a taxonomy which allows to assess and compare different implementations of master data objects. A systematic breakdown of core entities provides a framework to tell apart four subdividing categories of master data objects: independent and dependent objects, relational objects, and reference objects that serve to attribute information. This supports the preparation of data migrations from one system to another.

Master data is the blood of industry. Due to the complicated environment of materials flow - materials are frequently exchanged from one place to another, collecting or cleaning data from such environment becomes a challenging task. Unlike the common NLP tasks which already have massive labeled data, labeled data only share a small proportion compared to thoes unlabeled dirty data in the task of cleaning industry master data. Workers who take response in recording data are inclined to make the work done as fast as possible, this may cause the data records vary in thousands ways: abbreviation of material's name, lack of basic properties, wrong measurement unit or number, etc. This paper introduces some practical methods to handle the data cleaning problem. In data preprocessing and alignment, traditional NLP technics such as trie [1], Aho-Corasick(AC) [2] and wordnet [3] can take advantage of hidden statistical informations; we use tree search to help solving data correcting problem which can construct all possible combinations of data records; finally lattice-LSTM [4] are applied in splitting and reconstucting data properties.

Master data management is a process to integrate several data sources into one for good, consistent, and evenly standardized data. Several companies have been successful in creating a good quality data source for their data. However, some companies yet to have this system. A government agency in Indonesia finds difficulties in managing their data source. The problem lies in data duplication, lack of control on data reference, and the lack of application to monitor all data sources. This paper is carried out to fix the MDM monitoring dashboard that has been created in advance. The monitoring dashboard is created with the pureshare method. This paper designed a monitoring dashboard that can monitor the process and quality of master data. The benefit of this research is to get good quality and consistent master data, which can be used in all organizations, and its authenticity can be guaranteed.

PPATK established the APUPPT Training Center to increase the knowledge and competence of PPATK stakeholders and human resources through education and training in Anti Money Laundering (AML) and the Prevention of Terrorism Funding. It is able to develop and improve service quality, manage education and training through standardization of competent institutions and conduct training on certification of expertise and competence in AML and Terrorism Financing Prevention. In order to fulfill the needs of the Education and Training Center, this study design information system using the user-centered design (UCD) method that will be used internally by Education and Training Center employees called the Internal Information System of the Education and Training Center APUPPT (SINPU-PPT). SINPUT-PPT has the main goal of automating business processes and reducing the use of office stationery. The design identifies system requirements using the Unified Modeling Language (UML), designs the user interface using a Graphical User Interface (GUI) and designs the database. The results of this research is an information system design. The information system have 9 main menu tabs Home, Room Scheduling, Class Settings, Access Rights and Users, Master Data, Catering Arrangements, Training Program, Adding Letters and Receiving Letters. The results of this design can be used as a structured reference for coding programming and testing the development of the SINPU-PPT information system.

Open Government Data (OGD) is one of the government initiatives to gain public trust. It involves the citizens using or reuse the data so that they can make benefits from it. To implement OGD, the government shall implement several principles; there are standard data, metadata, interoperability, and reference code or master data. However, current problems and technology need to be found for a lesson learned in the next implementation. A systematic literature review was used as a methodology to find the information. And the results are 19 problems encountered and divided into seven categories: the OGD portal, engagement with the public, data quality, data utilization, organization data privacy, and data interoperability. For technology, there are seven technologies has found regarding OGD: UTAT model and trust factor, Legal open data framework towards data governance, linked open data, RST API GeoServer, Open data maturity model, Research model of post-adoption in OGD, Blockchain, and Semantic Government Vocabulary (SGoV). Findings in this research are several fundamental problems in managing portal for OGD, like updating the data and checking for the features in the portal, and the assessment toward OGD might be about corruption.

Master data construction is at the core of the enterprise information strategy, and it is the collection place of data to ensure the consistency and uniqueness of the target system data. In this paper, the identification indicators are determined from the characteristics of master data. The indicator weight is based on the business importance, so as to construct the master data identification score system. According to the data survey and census results, identify the master data and clarify the scope of master data attributes, and finally build the business owner master data model, which lay a good foundation for later data governance.

There are three types of user in the Electronic Veterinary Management (EVM) : employee; veterinarian; and manager. The employee role is the weakest authority and the manager is the strongest. Employees can manage the master data in the system that are the pet’s owner, pet data, treatment fee, and the queue system. Veterinarians can modify the treatment data. The final users, the manager can manipulate the user authority data and access the revenue of the clinic. The architecture of this system is model, view, and controller by servlet, JSP and Java language, MySQL is the database management system and iReport is the report generating tool. The average EVM time for getting pet’s data is 25.14 seconds and a standard deviation value is 10.02. While as the average time of the original process is 37.09 seconds, s.d. is 23.24. Therefore EVM is faster accessing than the original pet record procedure.

Studies have been conducted in an attempt to address the roles of Master Data Quality Management (MDQM) to improve service delivery. However, many of these studies concentrated on data quality dimensions and technical issues rather than addressing MDQM in a holistic manner. They lacked proper guidance on how the demographic variables could affect the improvement of service delivery through master data. This study contributes theoretically by filling the gap of the MDQM factors that specifically influence service delivery improvement and practically by identifying the effects of demographic variables that can be used not only by government departments but also by other organizations to make decisions relating to the improvement of master data quality management, as well as enhancing service delivery. This study recommends that future research should investigate the interacting effects for moderating factors that may have influence on users' perceptions towards technology as time varies. The study also recommends the use of a mixed methods approach or the triangulation of data collection and analysis methods to give a more holistic view of the influence of MDQM on service delivery.

Master data is required as a reference to perform data processing. The primary purpose of Master Data Management is to ensure that the master data reflects the current state of the business. However, many companies do not have a data management strategy, especially large companies which have large amounts of data. Master Data Management Maturity Model (MD3M) can be used to conduct an assessment of the maturity of the MDM in a company. The purpose of this research is to measure the maturity of the existing master data management at PT XYZ using Master Data Maturity Management Model (MD3M). This research used MD3M model by Spruit and Pietzka which has five key topics and 13 focus areas. A questionnaire is used to identify the level of master data management maturity. The questionnaire contains two sets of questions. The first set of questions tests the influential factors, and the second set of questions is about the capabilities to be assessed. The Subject Matter Experts (SMEs) who has a position as manager and Settlement Service Department and Reporting Department staff will answer the questionnaire given. The questionnaire will be mapped into a matrix to identify the maturity level. Measurement conducted in this study showed that the level of maturity at PT XYZ is 0, even though the findings in this study found that 54 out of 65 capabilities have been implemented that is equal to 83%. This is because the maturity level in the key topic data quality is still 0 (zero). By improving the capabilities that have not been implemented, the organization will achieve a higher level of maturity.

Geospatial data is important in supporting census and survey in Statistics Indonesia. Some problems exist about it are inaccurate maps, the existence of incomplete data attribute, and there are many versions exist in silo systems. They can impact the result of census or survey data quality. Improvement applied to overcome problems is managing geospatial data by adopting geodatabase. To improve the application in the future, it needs to be evaluated. By the reason, this research aims to measure the maturity level of master data management of geospatial data and arrange strategy to improve master data management of geospatial data in the future. This research used Master Data Management Maturity Model to measure the level of maturity and mapped the result to reference and master data management activity of Data Management Body of Knowledge. The result is Statistics Indonesia has not achieved any level yet for geospatial master data management and must concern to apply five activities for improvement in the future namely are to understand reference and master data integration needs, to identify reference and master data sources and contributors, to define and maintain the data integration architecture, to define and maintain match rules, and to establish golden record.

The approach to create power system digital twins is presented by the example of energy supply of a geographically localized R&D facility. In this paper, the six-layer digital twin architecture is proposed and its prototype software implementation is described. The architecture consists of an ontological model, a digital single line diagram, electronic documentation, master data, load measurement data, and mathematical models and simulations. The paper describes problems and principles of ontological modeling of the prosumer infrastructure, including customer load, low-voltage distribution network sections, small-scale generation equipment, and electric energy storage devices. The optimal configuration of the hybrid power supply system with renewable energy sources was computed using the digital twin that was composed according to the presented approach. For machine-readable representation of the digital twin, the ontological modeling language OWL is used.

Data is an important asset that used as a key for making decisions for the company’s operations and business needs. The data is not only taken from one database source but from various databases that exist to be a single entity. However, obstacles arise when the process is carried out, which is the existence of duplication data with different formats, but the data is same. Problems that can be reduced or even eliminated by implementing good data governance. One process that exists in data governance is Reference and Master Data. This process can be the foundation for making a master data management tools using the open source platform. This research focuses on analyzing existing master data management architectures and designing new architectures. So, design of master data management tools architecture for open source systems that can be applied at PT XYZ.

Data is the source of any information. Quality of data will provide accurate information. There are still many companies, especially State-Owned Enterprises that have not been able to maintain the quality of data. There is still a lot of duplication and redundancy of data in each application. Actually, the data remains the same even when the app is different. This problem can be solved by applying Data Governance processes, that is Reference and Master Data. In this process requires a Master Data Management Tool to manage, so the data will no longer be duplication or redundancy of data. In this research focuses on the analysis and design of master data synchronization algorithms, update master data algorithms, saving master data algorithms and using open source application platform that is Pentaho Data Integration. The results of this research are the matching data algorithm, saving master data algorithms, change master data algorithm and synchronization and integration master data algorithm. Although use open source-based application, MDM algorithm in this research can be made of data on the PT. XYZ can be synchronized and integrated between applications and can help PT. XYZ to maintain the quality of their data.

Statistics Indonesia is known as Badan Pusat Statistik or BPS in Indonesia, has a mission to provide high quality statistical data through an integrated statistics by implementing national and international standards. To enables practical application of standard statistical units and their classifications, BPS initiated create Master Data to integrated source of frame for all economic survey, that’s called statistical business registers (SBR). Main objective of SBR become the primary and only establishment and enterprise directory. But, the current situation, several directories of establishments exist, maintained by various SMEs. Therefore, assessment to SBR in BPS is important, because it becomes a benchmark for the success of SBR in BPS. The measurement was conducted using Master Data Management Maturity Model (MD3M), by providing questionnaires for the two SMEs at BPS, filled with interview. From the domains assessed, the result shows that the maturity level rate of SBR is at level 1 on a score from 1 to 5. This means that a first awareness for issues regarding the topic of MDM has been raised on an operational level. Initial steps are initialized.

Alignment is mandatory for any inertial navigation system (INS) before running in the navigation phase. It is a process, that the INS finds its own attitude including roll, pitch and heading. During the motion, the transfer alignment (TA) is used. Transfer alignment is a method of finding the initial attitude of a Slave INS by using a Master INS, which is a more precise external INS system. The paper presents a simulation platform for designing the TA algorithm. The platform contains the Master INS and Slave INS. The Master and the slave are mounted to different places in a vehicle and also have different orientations, so the Master data and Slave data are different. The core of the platform is transfer functions, which describe the relationship between the Master data and the Slave data. In this paper, the transfer functions will be mathematically demonstrated and verified by simulation. The simulation platform provides all sensor data and navigation data, which are needed for designing the TA algorithm.

Analytical workflows are heavily used in large and data intensive companies. An important application of such workflows in Siemens is equipment analytics when equipment KPIs and reports are computed by aggregating equipment’s operational, master, and analytical data. In Siemens this data satisfies big data dimensions and this dependence poses significant challenges in authoring, reuse, and maintenance of analytical workflows by engineers and data scientists. In this work we propose to address these problems by relying on semantic technologies: we use ontologies to give a high level representation of equipment’s operational and master data and offer a high level language to express KPIs over ontologies. We implemented our approach, integrated it with KNIME, and evaluated at Siemens. This is a preliminary work and we are excited about its further extensions.

The Master Data Management (MDM) empowers the organization to consolidate and integrate multiple master data sources into a single source of truth. Successful implementations of Master Data Management enable the organization to have standardized content that reflects business-based logic and rules processing. The goal of most MDM is to ensure that master data includes reference details that reflect the current state of the business. Master data management maturity model is a measurement of the ability of an organization for continuous improvement in master data management. The aim of this study is to measure MDM implemented in case organization with Master Data Management Maturity Model (MD3M). MD3M has five key topics and 13 focus areas. Those key topics and focus areas are designed to measure master data management maturity, each focus area has packed capabilities to be accomplished. The measurement showed that its maturity level is 0. Nevertheless, it not means that the organization did not implement MDM. The findings showed that almost 50% of the capabilities in MD3M had been implemented. By implementing further improvement about the missing capabilities, the organization will achieve higher maturity.

Domain verticals such as healthcare have a strong use case for externalizing and leveraging knowledge and experience, with a goal to administering high quality service at a minimal cost. With an ever-diminishing profit margin, these enterprises have to continually harness data and insights gained through a combination of administration experience and best practices captured in the form of externalized business rules, transactional data and master data. A major factor in achieving the above goal hinges on the ability to intuitively discover actionable insights based on diverse information. Another important factor to achieving the above goal consistently over time is to build in mechanisms to aid in the continual governance and management of vast sources of externalized knowledge and rule bases. This paper describes a case study of incorporating features of intelligent discovery and correlation to Enterprise Decision and Master Data Management Systems that constitute the knowledge backbone of a highly automated, evolving enterprise.

Nowadays, with the rapid development of the Internet of things (IoT) technology, various smart complex software systems (Smart CSS) are becoming important software paradigms for realizing the integration of human-cyber-physical ternary world. However, the characteristics of Smart CSS, including members heterogeneous, boundary openness, behavioral emergence, and continuous evolution, make the existing software analysis and design methods difficult to adapt to their development requirements. This paper proposed a data oriented analysis and design (DOAD) method for Smart CSS of IoT. It focuses on data closed loop of perception - analysis - decision - execution, including basic digital virtual objects (VO) data model design, the metadata design and standard for data exchange and catalog management, and the design of subject domain master data model. Through the implementation of DOAD Framework and its practice in the Smart CSS analysis and design of Smart City, it is demonstrated that this method can effectively help the development team to solve the existing data island and system silos problems, and improve the complexity management capability and resources reusability of the design team. In the future, we will continue to optimize it in practice by applying the DOAD method to the analysis and design of more Smart CSS of IoT.

In the management of its information system, XYZ Institute has launched several applications related to personnel affairs. Use of these various applications is necessary because of the specific features in each of these applications. However, problems such as redundancy and inconsistancy of employee data arose due to the absence of master data management as a single source of truth. Related to that, the research was conducted to plan the master data management in accordance with the needs of XYZ Institute in managing personnel data. Master data management planning is done by using data management body of knowledge guidance (DMBOK) which is published by Dama International in 2009 as a method. The research succeeded in formulating the need for master data management and master data source of employement at XYZ Institute. Such personnel data can be integrated using collaborative usage methods and transaction hub architecture patterns. Master data management is expected to help XYZ Institute in managing personnel data to improve data quality.

Smart cities generate huge amounts of data, which are often shared openly by multi-domain service providers. However, due to the lack of common standard interfaces, data models, and global ID management, smart city services suffers from fragmentation. It prevents data from being readily shared and accessible in a seamless way and restricts various services from being developed and discovered. To tackle this problem, we propose an open smart city platform, Oliot-OpenCity, built on the top of global standards ratified by GS1. The platform uses standardized event/master data to capture and store smart city data into an independently distributed repositories called EPC Information System (EPCIS), which can provide standard open interfaces to share resources and creates smart city data hub. Through Object Name Service (ONS) and Discovery Service (DS), it offers service and resource discovery to resolve distributed repositories and services from the various smart city domains. Moreover, to integrate the legacy city platforms, mediation gateways are proposed, which harmonize the fragmented key management and different data model into GS1 standards-based globally unique identifier and event/master data respectively. To show the feasibility of Oliot-OpenCity, we developed bus and vehicle information use cases which demonstrated its interoperability with other legacies, and the standardized data collection and sharing.

Master data management (MDM) of employee is used by the Information System of Personnel (SIKEP) and the Integrated Supreme Court Information System (SIMARI) in Supreme Court of Indonesia Republic (Supreme Court). SIKEP not yet used by all echelon I units in the Supreme Court. Therefore, it is necessary to identify problems where much data in SIKEP is not yet complete and valid. It is necessary to manage the master data of employees in the Supreme Court. The purpose of master data management assessment is to find out the extent of maturity of master data management (MDM) employee of Supreme Court Indonesia Republic (Supreme Court). This assessment is using management master data maturity model (MD3M) base on other research. The objective of this research paper is a practical nature. From a corporate point of view, the objective is to give organization to assess their employee MDM maturity. The result of assessment can identify potential improvement areas of Employee MDM. Final result of the maturity of the master data management of the Supreme Court is as follows; for business function data model and data quality maturity level is 1. The use and ownership and maturity level is 2. Business security function even reach the level of maturity 4. Therefore, the overall result of the maturity level of master data management in the Supreme Court is 1. This means that the Supreme Court already has awareness in the management of master data.

The world is moving toward the next industrial revolution - Industry 4.0; consequently manufacturing enterprises have to prepare for becoming smart factories. The development of information systems for manufacturing enterprises is a crucial approach to increase productivities and to gain competitive advantages. This paper presents system analysis and design of manufacturing information system employing radio frequency identification technology (RFID) for real time information tracking for plastic product manufacture. According to the analysis, the manufacturing information system is composed of master data and four main modules which are production, ordering, searching and tracking, and report. The practical framework and design of RFID in production process are proposed together with main challenges for implementing RFID. Furthermore, the test is conducted to confirm the feasibility of RFID application.

In centralized based IP surveillance architectures, the information pertaining to the event captured in each camera is sent to the master data base that is located in central control server. Later for crime or theft detection in surveillance region, the recorded data is accessed and analyzed. As on today, with ongoing acts of vandalism, it is highly essential for surveillance to be reactive rather than proactive. The objective is fulfilled by incorporating on board processing features such as face detection and recognition with in the sensor node(s).Soon person enters the vicinity of camera, features are captured, processed and then recognized with in the node.In order to demonstrate this on board processing capability, we came up with a single Tier homogeneous sensor network design featuring the real time face recognition without using database(s) and also optimal transmission in the network using image dimensionality reduction technique. In order to assess the similarity in face recognition statistical measures like mean square error and correlation coefficient are taken for consideration and the system is tested and verified with a confidence level of 98 % in a resource constrained environment.

Master data management and real-time data warehousing are gaining increased prominence within the worlds of business and technology. Past research efforts have shed light towards development of many master data management approaches. On the other hand, growth of technology has demanded real-time analytics and real-time processing of data. This trend has shed light in developing multiple real-time data warehousing approaches to perform real-time analytics based on an organization's requirements. With the evolution of real-time data warehousing, Master Data Management was an issue for large organizations' when both systems are working in the same business environment. Since both systems focus on real-time integration, similar, duplicated and parallel data extraction processes were executed by these applications. This was due to the fact that master Data Management was designed to focus on the operational aspect and real-time data warehouse was designed to focus on analysis aspect of the organization. Hence, each had its own ways of managing master data. These duplicated extractions caused data quality issues in these parallel applications. This research provides a framework that combines both master data management and real-time data warehousing and ultimately proposing to build a Hybrid Real-Time Data Warehousing Architecture in order to achieve enterprise wide master data management.

Systematic Literature Review (SLR) is a structured way of conducting a review of existing research works produced by the earlier researchers. The application of right data analysis technique during the SLR evaluation stage would give an insight to the researcher in achieving the SLR objective. This paper presents how descriptive analysis and text analysis can be applied to achieve one of the common SLR objectives which is to study the progress of specific research domain. These techniques have been demonstrated to synthesis the progress of Master Data Management research domain. Using descriptive analysis technique, this study has identified a trend of related literary works distribution by years, sources, and publication types. Meanwhile, text analysis shows the common terms and interest topics in the Master Data Management research which are 1) master data, 2) data quality, 3) business intelligence, 4) business process, 5) data integration, 6) big data, 7) data governance, 8) information governance, 9) data management and 10) product data. It is hoped that other researchers would be able to replicate these analysis techniques in performing SLR for other research domains.

In the EPCglobal standards for RFID architecture frameworks and interfaces, the Electronic Product Code Information System (EPCIS) acts as a standard repository storing event and master data that are well suited to Supply Chain Management (SCM) applications. Oliot-EPCIS broadens its scope to a wider range of IoT applications in a scalable and flexible way to store a large amount of heterogeneous data from a variety of sources. However, this expansion poses data security challenge for IoT applications including patients' ownership of events generated in mobile healthcare services. Thus, in this paper we propose Secure-EPCIS to deal with security issues of EPCIS for IoT applications. We have analyzed the requirements for Secure-EPCIS based on real-world scenarios and designed access control model accordingly. Moreover, we have conducted extensive performance comparisons between EPCIS and Secure-EPCIS in terms of response time and throughput, and provide the solution for performance degradation problem in Secure-EPCIS.

Near-real-time data warehousing is an emerging area of research in order to meet the high and up-to-date demands of business organizations. This means customers transactions executed at data source level need to reflect into the data warehouse immediately that requires semi-stream join between a stream of customer transactions and disk-based master data. For this purpose a well-known algorithm called CACHEJOIN was presented in the literature. Due to the sequential execution of both of its phases, called stream-probing phase and disk-probing phase, the algorithm cannot exploit the memory and CPU resources optimally and consequently it leaves it suboptimal. We present an algorithm P-CACHEJOIN that aims to optimize the performance of CACHEJOIN. P-CACHEJOIN allows parallel execution of two phases of CACHEJOIN which maximizes number of joined stream tuples and consequently improves service rate of the algorithm significantly. This paper presents the performance analysis of P-CACHEJOIN with existing CACHEJOIN empirically using synthetic dataset.

Software clustering techniques have been studied and applied to analyze and visualize the actual structure of legacy applications, which have used program information, e.g., dependencies, as input. However, business data also play an important role in a business system. Revealing which programs actually use data in the current system can give us a key insight when analyzing a long-lived complicated system. In this paper, we calculate indexes indicating how a data entity is used, making use of software clustering, which can be used to detect problematic or characteristic parts of the system. The developed technique can reveal the characteristics of a data entity; i.e., it is used like master data. We applied this technique to two business systems used for many years and found that our technique can help us understand the systems in terms of business data usage. Through case studies, we evaluated the validity of the indexes and showed that software visualization with the indexes can be used to investigate a system in an exploratory way.

Data cleaning is a critical part of the data transformation stage in data warehousing where the extracted data from relational databases are usually unclean. This may affect critical tasks in different organizations such as data analysis and decision making. Current techniques of data cleaning generally deal with one or two quality aspects. The techniques assume the availability of master data, or that users are involved in data cleaning such as manually placing confidence scores that represent the correctness of the values of data. In this paper, we present a uniform framework and algorithms to integrate data deduplication with inconsistent data repairing and discovering of the accurate values in data. We utilize the embedded density information in data to fix errors based on data density where tuples that are close to each other are packed together. We present a weight model to assign confidence scores that are based on the density of data. The assignments are automated and no user is involved in the process. We consider the inconsistent data in terms of violations with respect to a set of functional dependencies (FDs), as these violations are common in practice. We present a cost model for data repairing that is based on the weight model. We experimentally verify the quality and the scalability of our algorithms. We use synthetic and real datasets.

In the Computer System we have basically three types of Resources; they are Software, Hardware and Data. Data is the most important resource of computer system, because whatever computing we are doing is just because of data. Data Science deals with large amount of data to infer knowledge from the data sets to rationalize the information to achieve business values. Traditionally information was in structured form, means that was generally generated by business transactions. This information was easily process with our traditional data management system. In past few years the amount of digital data generated was grown exponentially. This large amount of data was unstructured, which cannot be processed and extracted efficiently from our traditional system. It includes Text files, sensor data, log data, web data, social networking data etc. The major reason behind the generation of this unstructured data is various applications used via internet e.g. Smart devices, web, mobile, social media and sensor Devices. For achieving the business goals this data is essential to mine. This large amount of unstructured data is called BIG DATA [1]. It's in large volume, its varying. There are various tools are available for processing of this large amount of data. Hadoop is one of the popular and efficient tool for the processing of Big Data. Hadoop provide a framework that allows us distributed computing and run tasks in parallel such that such type of complex data can be processed efficiently with respect to time, performance and resources. This paper covers the major resources used by Hadoop cluster.

In the process of high education informationization, unstructured data sharing plays an important part in the business integration. Due to the diversity and large volume of unstructured data, it is very difficult to share unstructured data. In this paper, we investigated on the problems of data sharing among different departments in high education information integration, and provided an unstructured data sharing model based education information integration.

An enriching of stream data with disk-based master data is common in many applications. Real-time data warehousing is one of these applications where stream data is customers transactions producing by operational data source(s). This stream data needs to enrich by disk-based master data before loading this into the data warehouse. To implement this enrichment operation a join operator is required. Normally we call it semi-stream join as it is performed between stream data and disk data. The join operator typically works under limited main memory and this memory is generally not large enough to hold the whole disk-based master data. Therefore, the relation is loaded into memory in partitions. A well-known join algorithm called MESHJOIN (Mesh Join) has been presented in the literature to implement the semi-stream join operation. However, the algorithm suffers with some limitations. In particular, the algorithm can be improved based on the characteristics of stream data e.g. skew in stream data. In this paper we address this issue by presenting a novel algorithm called Cached-based Stream-Disk Join (CSDJ). The algorithm exploits skew characteristic in stream data more appropriately and over performs existing MESHJOIN. We present results for Zip-fian distributions of the type that appear in many applications. We evaluate our algorithm using synthetic, TPC-H and real datasets. Our experiments show that CSDJ performs significantly better than MESHJOIN.

Business Intelligence plays an important role in the organization for collecting, integrating, analyzing and transforming data to be useful for effective decision making process. Nowadays, organizations are flooded with various kinds of unstructured data such as e-mail, images, reports, maps, charts, publications. An effective and efficient business model of these data could help in decision making. Currently, there is no study done on the business intelligence model for managing unstructured data that can fulfil the organization needs. Therefore, the purpose of this paper is to improve the organization's business intelligence process through the exploitation of unstructured data that is owned by the organization. In this study, unstructured data are classified, enriched and complemented with diversity of data through the process of creating metadata for each unstructured data. Four main processes are proposed to transform unstructured data to structured data which are extraction, classification, storage and mapping of data classes. Each process and its activities are combined to produce an effective and efficient business intelligence model for unstructured data management. This model helps in generating new data and information that is more comprehensive and collective to help business intelligence through advanced analysis, decision-making process and planning new research areas. Output from this study is to make unstructured data as renewable assets that is easily accessible and used as a reference and foundation in business intelligence and decision making process.

Recent efforts in data cleaning of structured data have focused exclusively on problems like data deduplication, record matching, and data standardization; none of these focus on fixing incorrect attribute values in tuples. Correcting values in tuples is typically performed by a minimum cost repair of tuples that violate static constraints like CFDs (which have to be provided by domain experts, or learned from a clean sample of the database). In this paper, we provide a method for correcting individual attribute values in a structured database using a Bayesian generative model and a statistical error model learned from the noisy database directly. We thus avoid the necessity for a domain expert or clean master data. We also show how to efficiently perform consistent query answering using this model over a dirty database, in case write permissions to the database are unavailable. We evaluate our methods over both synthetic and real data.

Stream-based join algorithms are a promising technology for modern real-time data-warehouses. A particular category of stream-based joins is a semi-stream join where a single stream is joined with a disk based master data. The join operator typically works under limited main memory and this memory is generally not large enough to hold the whole disk-based master data. Recently, a seminal join algorithm called MESHJOIN (Mesh Join) has been proposed in the literature to process semi-stream data. MESHJOIN is a candidate for a resource-aware system setup. However, MESHJOIN is not very selective. In particular, MESHJOIN does not consider the characteristics of stream data and its performance is suboptimal for skewed stream data. In this paper I propose a novel Cached-based Semi-Stream Join (CSSJ) using a cache module. The algorithm is more appropriate for skewed distributions, and I present results for Zipfian distributions of the type that appear in many applications. I conduct a rigorous experimental study to test our algorithm. Our experiments show that CSSJ outperforms MESHJOIN significantly. I also present the cost model for our CSSJ and validate it with experiments.

At the arrival of big data era, traditional geological industries are still using the traditional way to produce and collect data, and geosciences information is represented as unstructured data in various forms. These data is often categorized together according to a relatively simple way, thus forming a number of datasets with complex internal structure. However, this is not a good expression of rich geoscience information carried by unstructured data and it is also inconvenient to express complex relationships among the information, even against to find in-depth knowledge across datasets. Meanwhile, existence forms of such data also impeded the application of advanced technological methods. In an attempt to solve the problem, this paper proposes a multi-granularity content tree model and pay-as-you-go mode to support evolvement data modeling. These features help to split the data model, position data content precisely and to expand the dimensions of the main features that described according to the data subject, and then gradually discover data contained information and relationships among the information. Considering the large size of the data features, this paper designs data persistence mode based on HBase, so as to achieve the purpose of data processing by using technologies within the Hadoop system. This article also presents data content extraction and content tree initial state algorithms under MapReduce framework, and dynamic loading and local caching algorithms of content tree, thus forming a basic extract-store-load process. An application example of the model about the geological industries is given at the end.

Requirements such as integrated view of the customer or global business process integration make enterprise wide management of master data a prerequisite for achieving business goals. The master data application architecture, as a part of enterprise master data management, plays a critical role in enterprises. Choosing the right master data application architecture is a controversial subject in many enterprises. Unfortunately, the current state of the art in research does not provide sufficient guidance for enterprises dealing with this subject. The paper aims at overcoming this gap in research by presenting a decision model for supporting enterprises in the decision-making process regarding the choice of the right master data application architecture. To design the model, Multiple-Criteria Decision Analysis and Design Science Research Methodology were applied.

Emergence of heterogeneous format of unstructured data is vital for an organization to consider as a basis to support organization function. Thus, there is an urgency of a suitable management process of unstructured data through collecting, manage, transfer and transform these data to a meaningful and informed data. Effective and efficient mapping processes of unstructured data to structured data could help organization to understand the significance of exploited and transformed data in supporting decision making process. Currently there is no study done on mapping unstructured data to structured data that can full fill the organization needs. This paper will explained four (4) main processes which are extraction, classification, repositories development and mapping of data with intention to help in generating new data and information which are more structured, comprehensive, collective and varied in topics to serve organizational needs. Output from this study is to ensure the mapping process of unstructured data to structured data will denote unstructured data as renewable assets that is useful, well informed and meaningful to serve organizational functions.

In the field of real-time data warehousing semi-stream processing has become a potential area of research since last one decade. One important operation in semi-stream processing is to join stream data with a slowly changing disk-based master data. A join operator is usually required to implement this operation. This join operator typically works under limited main memory and this memory is generally not large enough to hold the whole disk-based master data. Recently, a seminal join algorithm called MESHJOIN (Mesh Join) has been proposed in the literature to process semi-stream data. MESHJOIN is a candidate for a resource-aware system setup. However, MESHJOIN is not very selective. In particular, MESHJOIN does not consider the characteristics of stream data and its performance is suboptimal for skewed stream data. In this paper we propose a novel Semi-Stream Join (SSJ) using a new cache module. The algorithm is more appropriate for skewed distributions, and we present results for Zipfian distributions of the type that appears in many applications. We conduct a rigorous experimental study to test our algorithm. Our experiments show that SSJ outperforms MESHJOIN significantly. We also present the cost model for our SSJ and validate it with experiments.

The amount of business data is large & keeps on evolving leading to heterogeneous information base. The challenge is to access, analyze & integrate various data sources for making intelligent decisions. Business data can be structured or unstructured. Structured data attains the row-column format easily while unstructured data (USD) is the one that poses problem in such kind of tabular storage. Owing to the fact that USD is more than three times of structured data, and that it is more resourceful business wise and helps in charting out strategies and making decisions, it becomes important to devise methods for handling USD in data warehouse. Since the importance of USD has been realized, various authors have discussed different ways to manage it and extract useful information from it. In this paper, we have first comprehensively reviewed & surveyed the representative research works of various authors that have demonstrated how unstructured data can be handled in the warehouse. Finally, we have manifested & sorted them on various parameters & provided the same in tabular form.

Master data management (MDM) provides an access to the consistent views of an organization's master data. Yet the establishment of MDM function, i.e. a department that attempts to ensure master data consistency is not an easy task as several stakeholders have different interests and expectations, among many other reasons. This paper is based on a year-long MDM project, from which we identify the process for establishing MDM functions in an organization, and what are the different steps and interdependencies that should be taken into account. These steps and their dependencies, and other incidental issues help organizations when establishing MDM function, and complement the scarce MDM literature.

We are in an age repeatedly referred to the information age. In this age, because we suppose that information leads to power and achievement, and credit to sophisticated techniques such as computers, satellites etc. we include collecting large amount of information similar to business transaction, scientific data, medical data satellite data, surveillance video a pictures world wide web and many more. With the enormous amount of data stored in files, data base and this technique hold large amount of data is called data mining. Data mining is the method of extracting valuable information from the enormous amount of data saved in the database and files. At that time basically two important reasons that are used in data mining. First our capacity to accumulate and store the large amount of data is fast increase day by day, and second but the most imperative basis is the need to turn such data into useful information and knowledge. Association rule mining is a vital procedure to find out hidden interaction with items in the transaction. This paper describes new technique cluster based Master-Slave architecture. It uses Hybrid technique, the grouping of bottom up and top down technique for search repetitive item sets. It reduces the time in use to get out the support count of the item sets. The Prime number show present offers the give for verify rules and provides decrease in the data complexity.

With high quality of data, enterprises will add value of business. How-ever, poor data has resulted in waste of resource, low service efficiency and high costs in every area. In this paper, we firstly focus on basic issues of data quality, like where the quality problems come from and how to describe it. Then we study some cases of data quality management from a holistic enterprise perspective to the details of perspective, that is, hierarchical management architecture, frameworks, approaches and algorithm. Quality assessment of data is also an important theme, which is used to show whether data is good enough and to help people master credibility of data quality. We study some assessment algorithm and models. When problem occur, what tools should be used in a project is also included in the paper. Finally, we provide some outstanding research topics and unresolved issues for future.

The profusion of unstructured data forced organizations to manage and take advantage of such data especially in the decision making process. The feasibility of integrating or mapping unstructured data to a data warehouse is becoming significant to bridge this gap and take the full potential of these data. In this paper, we propose a multi-layer schema for mapping structured data stored in a data warehouse and unstructured data in business-related documents. The multi-layer schema facilitates the mapping between the two different data. Linguistically correlated data is identified using Word Net to enable the integration between both data sources. We also propose a generic XML schema for business-related unstructured documents to assist the mapping. The use Word Net to identify the matching result is promising in the absence of schema-instance and without the need to domain specific knowledge.

Manufacturing control is an increasingly challenging task in many domains due to the complexity of production processes. In many Manufacturing Execution Systems, rule-based components therefore steadily become harder to analyze and to maintain as the number of rules usually grows with the age of the factory. As a result, given large rule sets accumulating over the years of operation, the behavior of such systems is hard to predict and changes to rule sets or referenced master data may lead to undesired and often very costly behavior. This paper describes an approach to rule-driven decisions based on knowledge about the factory stored in ontologies. Expressing rules and master data in OWL and reasoning on these knowledge bases allows better understanding of the control systems and more intuitive rules. Most importantly, several features of the rule sets and the corresponding master data can be verified whenever changes are necessary thus leading to a more predictable control system even with large rule sets.

Putting robotic automation to work in a mid-nineties vintage 8” wafer fab is without a doubt a challenging maneuver. Therefore it is not at all surprising that any rigorous project scope puts all its momentum into getting the robots automatically moving wafers based on the existing standard workflows — in other words — executing the non exceptional everyday workflows. In the case of manufacturing exceptions, however, wafer lots are set aside for human interaction, undermining the value of robotic automation. This paper will describe how we address exception workflows at our robotic metrology clusters under the strict rules of automotive device manufacturing. We will introduce our robotic automation environment and present different metrology clusters. The basic workflow to improve robotic tool utilization and the efforts to implement it will be presented. Benefits and prospects of the selected approach will be discussed.

The amount of data in our industry and the world is exploding. Data is being collected and stored at unprecedented rates. The challenge is not only to store and manage the vast volume of data (“big data”), but also to analyze and extract meaningful value from it. There are several approaches to collecting, storing, processing, and analyzing big data. The main focus of the paper is on unstructured data analysis. Unstructured data refers to information that either does not have a pre-defined data model or does not fit well into relational tables. Unstructured data is the fastest growing type of data, some example could be imagery, sensors, telemetry, video, documents, log files, and email data files. There are several techniques to address this problem space of unstructured analytics. The techniques share a common character tics of scale-out, elasticity and high availability. MapReduce, in conjunction with the Hadoop Distributed File System (HDFS) and HBase database, as part of the Apache Hadoop project is a modern approach to analyze unstructured data. Hadoop clusters are an effective means of processing massive volumes of data, and can be improved with the right architectural approach.