In order to extract the features of fault signals of dynamic system polluted and modulated by background noise, and solve anomaly detection problem of slight fault based on the generated normal patterns, this paper considers the one-class classification problem of insufficient fault samples and class imbalanced in intelligent monitoring and diagnosis for dynamic systems. Besides, it is well known that the conventional feature selection methods always depend on specific fault types. If some features are chosen to identify one fault, their performance may be poor for other fault cases. Compared with other classifiers, support vector data description (SVDD) is only based on samples at normal state, and its optimization hyperplane is independent of specific fault types. Chaos and fractal are the nonlinear phenomena which widely exist in nature and socio-economic system. Chaos and fractal can help to find out the internal regularity of system, and it is a new branch and the important fields of nonlinear science, whose subjects include unsmoothed, nondifferentiable geometry and long-range correlated structure in nonlinear system, method to judge if one system has chaos and fractal phenomena that mainly uses the largest Lyapunov index and fractal dimension to judge. Study has showed that the chaos and fractal theory can reflect the essence of the dynamic systems as complex system. Hence, a new abnormal identification and feature selection method of dynamic systems that combines chaos fractal feature and SVDD is presented. In this method, the receiver operating characteristic (ROC) curve for 2 classifying errors i.e. misdiagnosis and missed diagnosis in classification field based on SVDD is synthesized in the proposed method to make the selected chaos fractal and features more effective. The best combination of the largest Lyapunov exponent and correlation dimension is selected through analyzing chaos and fractal features, which include: the largest Lyapunov exponent, box dimension, correlation dimension and the mean of generalized dimension. To ensure good performance in the environment of complex dynamic systems, anti-noise robustness analysis is conducted. The data samples of different normal states are trained to obtain the optimal enclosing spheres respectively. And the relative distances between diagnostic samples and the distributed spheres are introduced to decide which class they belongs to. In this way SVDD classifier of normal condition samples has been proposed based on combined features as the input of SVDD classifier. In addition, SVDD is a robust data domain description method; its performance, however, is strongly influenced by kernel parameter. To improve the low fraction of target class that is accepted by the SVDD in the case of atypical target training data, experimental studies for seeking optimum parameters of kernel functions which are used to improve classification precision have been discussed. The penalty coefficient B is fixed to the value of 0.4, and in this case, the optimal kernel parameter value is suggested as a value of 0.6, which can prevent effectively the phenomenon of over-fitting and has good fault sensitivity and generalization. Abnormal identification model is set up based on the normal history data of the system to detect the abnormal condition by comparing the projection on the chaos fractal feature space to which the real measurement data and normal data were projected. Using SVDD-ROC as feature extraction method, the problem that traditional feature extraction methods depend on specific fault samples is avoided. The experiments with single-point fault data simulated on bearing vibration test-bed have been implemented to evaluate the efficiency and performance of the clustering, and results of test show that the proposed method only requires the samples at normal state to identify condition between normal states and fault states, and it is able to well distinguish unknown fault types. The study can provide theoretical and design basis for the modeling and detecting of dynamic systems with fault conditions, which is helpful to prevent incident, save maintenance cost, improve the utilization rate of the system, protect the running of production, and thus reduce costs. [ABSTRACT FROM AUTHOR]