Anomaly detection and predictive analytics for financial risk management

In the big data era, the digital revolution has driven the entire financial industry to collect, store and analyze massive volumes of data nowadays than it ever has in history. With the overwhelming scale of data, new technologies are needed to derive competitive advantage and unlock the power of the data, including the approaches people use for financial risk management. In this dissertation, we study how advanced data mining techniques can play essential roles in financial risk management. Specifically, we provide case studies to apply data mining techniques in three application scenarios for financial risk management. The first study exploits a special type of fraudulent trading ring pattern in the financial market, and defines the so-called blackhole and volcano patterns to identify the fraud. A blackhole mining framework consisting of two pruning schemes is developed. The first pruning scheme is to exploit the concept of combination dominance to reduce the exponential growth search space. The second pruning scheme is an approximate approach, which can strike a balance between the efficiency and the completeness of blackhole mining. The second study exploits the problem of contract risk management. In particular, how IT service providers can leverage the experiences and lessons learnt from historical contracts to prevent similar issues from reoccurring in the future, in order to mitigate the project risks, ensure smooth delivery and continuous profitability. Along this line, we investigate how to predict potential risks for new contracts based on their similarities with existing ones, and develop a new approach as an extension of the Mahalanobis distance metric learning framework to solve the problem. The third study examines the application of cluster analysis in bankruptcy pattern learning and financial statement fraud detection. By leveraging the domain knowledge in accounting area, valuable features from financial statement can be extracted. Clustering technique is then applied to identify the clustering effect of bankrupt companies in different business sectors. Finally, the most indicative financial features for the bankrupt companies in the business sector can be uncovered from the hidden data and validated by significant tests.