Physical Integration of Heterogeneous Web Based Data

With the ever-growing quantity of data available on the Internet and the development of new techniques to gain novel insights on the web of data, more attention must be given to the integration methods used. While the data extraction techniques have developed considerably with the rise of new automation tools that can be used even by persons without a software engineering background, the problem arises on integrating this data required for the analysis. In order to be able to gain valuable knowledge not only from a local perspective, but also from a global, enterprise-wide point of view and even from outside the borders of the organization, this data must first be integrated to a common storage capability of the business. The problem arises from the way most of the database management systems are designed, which in the early days of their development, it wasn’t anticipated such an explosion of information from so many heterogeneous data sources. Moreover, there is an ongoing race on tapping into new information in places unexplored before, thus gaining valuable competitive advantage over other businesses. The problem of data integration is even more challenging in the case of web-based data sources due to the high frequency of changes that occur, meaning that solutions which works today are not guaranteed to work properly tomorrow.