Search

In the field of time series data mining, the concept of the Perceptually Important Point (PIP) identification process is proposed for financial time series pattern matching and it is then found suitable for time series dimensionality reduction and representation. Its strength is on preserving the overall shape of the time series by identifying the salient points in it. With the rise of Big Data, time series data contributes a major proportion, especially on the data which generates by sensors in the Internet of Things (IoT) environment. According to the nature of PIP identification and the successful applications in the past years, it is worth to further explore the opportunity to apply PIP in time series “Big Data”. However, the performance of PIP identification is always considered as the limitation when dealing with “Big” time series data. In this paper, two improvement algorithms namely Caching and Splitting algorithms are proposed. Significant improvement in term of speed is obtained by these improvement algorithms.

This research examines two different applications of the Genetic Algorithms (GA) in portfolio management. GA is adopted to determine the optimized parameters setting of different technical indicators and portfolio weighting. Besides the Traditional GA, the Hierarchical GA is also adopted in this research. Different algorithms and the usage of different numbers of technical indicators are evaluated in different economic situations. GA shows its optimization power over different tasks in portfolio management.

Time series is an important class of temporal data objects and it can be easily obtained from scientific and financial applications. A time series is a collection of observations made chronologically. The nature of time series data includes: large in data size, high dimensionality and necessary to update continuously. Moreover time series data, which is characterized by its numerical and continuous nature, is always considered as a whole instead of individual numerical field. The increasing use of time series data has initiated a great deal of research and development attempts in the field of data mining. The abundant research on time series data mining in the last decade could hamper the entry of interested researchers, due to its complexity. In this paper, a comprehensive revision on the existing time series data mining research is given. They are generally categorized into representation and indexing, similarity measure, segmentation, visualization and mining. Moreover state-of-the-art research issues are also highlighted. The primary objective of this paper is to serve as a glossary for interested researchers to have an overall picture on the current time series data mining development and identify their potential research direction to further investigation.

Time series visualization is a fundamental task in most financial applications. A framework that can reduce the dimensionality of the time series data, sufficiently accurate so that it can capture the actual shape of the time series but, at the same time, salient points will not be smoothed out can take advantage of charting the raw time series data. On the other hand, it is preferable that the representation framework can handle the multi-resolution problem rather than reduce the dimension to a fixed level only. In this paper, a framework that represents and visualizes time series data based on data point importance is proposed. Furthermore, discovering frequently appearing and surprising patterns are non-trivial tasks in financial applications. A method for discovering patterns across different resolutions is proposed. The proposed method is based on a modified version of VizTree. By converting the time series to symbol string based on data point importance, the potential patterns with different lengths can be encoded in the VizTree for visual pattern discovery while the important points and the overall shape of the time series patterns can be preserved even under a high compression ratio. Various experiments were conducted to evaluate the performance of the proposed framework. One may find it particularly attractive in financial applications like stock data analysis.

Recently, the increasing use of time series data has initiated various research and development attempts in the field of data and knowledge management. Time series data is characterized as large in data size, high dimensionality and update continuously. Moreover, the time series data is always considered as a whole instead of individual numerical fields. Indeed, a large set of time series data is from stock market. Stock time series has its own characteristics over other time series. Moreover, dimensionality reduction is an essential step before many time series analysis and mining tasks. For these reasons, research is prompted to augment existing technologies and build new representation to manage financial time series data. In this paper, financial time series is represented according to the importance of the data points. With the concept of data point importance, a tree data structure, which supports incremental updating, is proposed to represent the time series and an access method for retrieving the time series data point from the tree, which is according to their order of importance, is introduced. This technique is capable to present the time series in different levels of detail and facilitate multi-resolution dimensionality reduction of the time series data. In this paper, different data point importance evaluation methods, a new updating method and two dimensionality reduction approaches are proposed and evaluated by a series of experiments. Finally, the application of the proposed representation on mobile environment is demonstrated.

One of the major duties of financial analysts is technical analysis. It is necessary to locate the technical patterns in the stock price movement charts to analyze the market behavior. Indeed, there are two main problems: how to define those preferred patterns (technical patterns) for query and how to match the defined pattern templates in different resolutions. As we can see, defining the similarity between time series (or time series subsequences) is of fundamental importance. By identifying the perceptually important points (PIPs) directly from the time domain, time series and templates of different lengths can be compared. Three ways of distance measure, including Euclidean distance (PIP-ED), perpendicular distance (PIP-PD) and vertical distance (PIP-VD), for PIP identification are compared in this paper. After the PIP identification process, both template- and rule-based pattern-matching approaches are introduced. The proposed methods are distinctive in their intuitiveness, making them particularly user friendly to ordinary data analysts like stock market investors. As demonstrated by the experiments, the template- and the rule-based time series matching and subsequence searching approaches provide different directions to achieve the goal of pattern identification.

Time series data, due to their numerical and continuous nature, are difficult to process, analyze, and mine. However, these tasks become easier when the data can be transformed into meaningful symbols. Most recent works on time series only address how to identify a given pattern from a time series and do not consider the problem of identifying a suitable set of time points for segmenting the time series in accordance with a given set of pattern templates (e.g., a set of technical patterns for stock analysis). However, the use of fixed-length segmentation is an oversimplified approach to this problem; hence, a dynamic approach (with high controllability) is preferable so that the time series can be segmented flexibly and effectively according to the needs of the users and the applications. In view of the fact that this segmentation problem is an optimization problem and evolutionary computation is an appropriate tool to solve it, we propose an evolutionary time series segmentation algorithm. This approach allows a sizeable set of pattern templates to be generated for mining or query. In addition, defining similarity between time series (or time series segments) is of fundamental importance in fitness computation. By identifying the perceptually important points directly from the time domain, time series segments and templates of different lengths can be compared and intuitive pattern matching can be carried out in an effective and efficient manner. Encouraging experimental results are reported from tests that segment both artificial time series generated from the combinations of pattern templates and the time series of selected Hong Kong stocks.

It is always expected that a correlation exists between the movement of stock prices (technical analysis) and news sentiment (fundamental analysis). If we can determine such a correlation, further interesting research directions will certainly be generated. In this paper, a system prototype is proposed for investigating the correlation between stock prices and news sentiment. Our primary target market is Hong Kong and the system is customized for Chinese language. Different methods and the impacts of various design parameters are tested in the experiments.

SB-Tree is a data structure proposed to represent time series according to the importance of the data points. Its advantages over traditional time series representation approaches include: representing time series directly in time domain (shape preservation), retrieving time series data according to the importance of the data points and facilitating multi-resolution time series retrieval. Based on these benefits, one may find this representation particularly attractive in financial time series domain and the corresponding data mining tasks, i.e. categorization and clustering. In this paper, an investigation on the size of the SB-Tree is reported. Two SB-Tree optimization approaches are proposed to reduce the size of the SB-Tree while the overall shape of the time series can be preserved. As demonstrated by various experiments, the proposed approach is suitable for different categorization and clustering applications.

SB-Tree is a binary tree data structure proposed to represent time series according to the importance of data points. Its use in stock data management is distinguished by preserving the critical data points' attribute values, retrieving time series data according to the importance of data points and facilitating multi-resolution time series retrieval. As new stock data are available continuously, an effective updating mechanism for SB-Tree is needed. In this paper, a study of different updating approaches is reported. Three families of updating methods are proposed. They are periodic rebuild, batch update and point-by-point update. Their efficiency, effectiveness and characteristics are compared and reported.

Nowadays, financial applications spread over various devices in both e-commerce and m-commerce. One of the major tasks in this kind of application is viewing the historical price movement of a stock for the market players before making any decision. However, due to the divergence of the configurations among different devices and platforms, adaptation is necessary for time series data delivery and visualization. In this paper, an adaptive framework of financial time series delivery and visualization is proposed to achieve this goal. The core system adapts the concept of data point importance and data points reordering for time series representation. The application of the proposed framework on stock price time series delivery and visualization is demonstrated in different devices using Web service.

In our previous work, a time series representation framework, specialized binary tree (SB-tree) has been proposed for representing the stock time series data effectively and efficiently. By putting a set of SB-trees together, a time series database is formed while we termed it as a specialized binary tree-forest (i.e. SBT-forest). By manipulating the SBT-forest, different time series query and mining processes can be facilitated. However, the major challenge is how to locate a SB-tree in the forest efficiently. Therefore, the development of an indexing approach for the SB-trees is of fundamental importance for maintaining an acceptable speed for query. In this paper, a time series indexing approach, based on transforming the SB-trees to symbol strings first and then indexing the symbol strings by a trie data structure, is proposed. The proposed approach is efficient and effective as well. As demonstrated in the experiments, the proposed approach speeds up the time series query process. The proposed approach can handle the problem of updating new entries to the database without any difficulty.

Most of the existing commercial NIDS products are signature-based but not adaptive. In this paper, an adaptive NIDS using data mining technology is developed. Data mining approaches are used to accurately capture the actual behavior of network traffic, and portfolio mined is useful for differentiating "normal" and "attack" traffics. On the other hand, most of the current researches are using only one engine for detection of various attacks; the proposed system is constructed by a number of agents, which are totally different in both training and detecting processes. Each of the agents has its own strength on capturing a kind of network behavior and hence the system has strength on detecting different types of attack. In addition, its ability on detecting new types of attack as well as a higher tolerant to fluctuations were shown. The experimental results showed that the frequent patterns mined from the audit data could be used as reliable agents, which outperformed from traditional signature-based NIDS.

Discovery of interesting or frequently appearing time series patterns is one of the important tasks in various time series data mining applications. However, recent research criticized that discovering subsequence patterns in time series using clustering approaches is meaningless. It is due to the presence of trivial matched subsequences in the formation of the time series subsequences using sliding window method. The objective of this paper is to propose a threshold-free approach to improve the method for segmenting long stock time series into subsequences using sliding window. The proposed approach filters the trivial matched subsequences by changing Perceptually Important Point (PIP) detection and reduced the dimension by PIP identification.

Stock data in the form of multiple time series are difficult to process, analyze and mine. However, when they can be transformed into meaningful symbols like technical patterns, it becomes easier. Most recent work on time series queries concentrates only on how to identify a given pattern from a time series. Researchers do not consider the problem of identifying a suitable set of time points for segmenting the time series in accordance with a given set of pattern templates (e.g., a set of technical patterns for stock analysis). On the other hand, using fixed length segmentation is a primitive approach to this problem; hence, a dynamic approach (with high controllability) is preferred so that the time series can be segmented flexibly and effectively according to the needs of users and applications. In view of the fact that such a segmentation problem is an optimization problem and evolutionary computation is an appropriate tool to solve it, we propose an evolutionary time series segmentation algorithm. This approach allows a sizeable set of stock patterns to be generated for mining or query. In addition, defining the similarity between time series (or time series segments) is of fundamental importance in fitness computation. By identifying perceptually important points directly from the time domain, time series segments and templates of different lengths can be compared and intuitive pattern matching can be carried out in an effective and efficient manner. Encouraging experimental results are reported from tests that segment the time series of selected Hong Kong stocks.

Time series data are difficult to manipulate. When they can be transformed into meaningful symbols, it becomes an easy task to query and understand them. While most recent works in time series query only concentrate on how to identify a given pattern from a time series, they do not consider the problem of identifying a suitable set of time points based upon which the time series can be segmented in accordance with a given set of pattern templates, e.g., a set of technical analysis patterns for stock analysis. On the other hand, using fixed length segmentation is only a primitive approach to such kind of problem and hence a dynamic approach is preferred so that the time series can be segmented flexibly and effectively. In view of the fact that such a segmentation problem is actually an optimization problem and evolutionary computation is an appropriate tool to solve it, we propose an evolutionary segmentation algorithm in this paper. Encouraging experimental results in segmenting the Hong Kong Hang Seng Index using 22 technical analysis patterns are reported.

Most of the existing commercial Network Intrusion Detection System (NIDS) products are signature-based but not adaptive. In this paper, an adaptive NIDS using data mining technology is developed. Data mining approaches are used to accurately capture the actual behaviour of network traffic, and the portfolio mined is useful for differentiating 'normal' and 'attack' traffics. On the other hand, most of the current researches use only one engine for detection of various attacks; the proposed system, which is constructed by a number of agents, is totally different in both training and detecting processes. Each of the agents has its own strength in capturing a kind of network behaviour and hence the system has strength in detecting different types of attack. In addition, its ability in detecting new types of attack and its higher tolerance to fluctuations were shown. The experimental results showed that the frequent patterns mined from the audit data could be used as reliable agents, which outperformed the traditional signature-based NIDS.