Search

Min Li,1 Hongzhen Xie,2 Qing Luo,3 Guangbin Huang,1 Guoxian Xu,1 Ye Cheng,1 Jun Li4 1Department of Traumatology, Chongqing University Central Hospital, Chongqing Emergency Medical Center, Chongqing, People’s Republic of China; 2Department of Health Medicine, General Hospital of Southern Theatre Command, Guangzhou, People’s Republic of China; 3Department of Neurosurgery, General Hospital of Southern Theatre Command, Guangzhou, People’s Republic of China; 4Chongqing Metropolitan College of Science and Technology, Chongqing, People’s Republic of ChinaCorrespondence: Jun Li, Chongqing Metropolitan College of Science and Technology, No. 368, Guangcai Avenue, Yongchuan District, Chongqing, 402160, People’s Republic of China, Tel +86 18084067947, Email lijun20228@163.comBackground: International stroke care guidelines recommend the routine assessment and management of psychological and emotional problems in patients and their families after a stroke. This study aimed to explore the humanistic nursing needs of patients with stroke and their families and provide a basis for the construction of humanistic nursing practice guidelines for advanced stroke.Methods: From September to October 2019, using the descriptive phenomenological research method, a semi-structured interview outline was formed based on a literature review and subject group discussions. A targeted sampling method was used to investigate 21 patients with stroke and 21 family members, respectively, and their care needs were investigated in depth. Colaizzi’s thematic cluster analysis was used to analyse the data, and a total of 6 themes and 14 sub-themes were summarised.Results: The 6 themes and 14 sub-themes were as follows: ①psychological care, ② security concerns (professional personal integrity, timely response), ③ emotional care (service consciousness, positive incentives, empathy, effective communication, provision of a communication platform, personalised care), ④ respect for rights, ⑤ rehabilitation care (rehabilitation programme, professional guidance, rehabilitation configuration, continuation of services) and ⑥ family care (physical and psychological support, care guidance).Conclusion: Among the humanistic care needs of patients with stroke and their families, the most basic physiological care accounts for the largest proportion of emotional and rehabilitation care, followed by safety care, respect for rights and family care. Based on the actual humanistic care needs of patients with stroke and their families, the practical effect of humanistic care in stroke wards can be improved. This study provides a reference for the construction of humanistic nursing practice guidelines for late stroke.Keywords: stroke, humanistic care, qualitative research

This paper explores the changes in EEG when subjects performed a modified Test of Variables of Attention (TOVA), compared to open eye resting (baseline) state. To recognize these two different brain states, two machine learning algorithms, i.e. extreme learning machine (ELM) and support vector machine (SVM), were applied and compared, using 3 statistical features and 4 power spectral density per channel. The results showed that using all 14 channels, ELM and SVM achieved similar test accuracy of 94.6% and 95.1% respectively (McNemar’s test p = 0.8 > 0.05). Using recursive channel selection, 9 channels (ELM) and 8 channels (SVM) were selected from 14 channels. After channel selection, ELM outperformed SVM significantly (McNemar’s test p = 0.0005 < 0.01) with average test accuracy of 95.0% and 92.5% respectively. The channel rank of each subject was weighted and merged using analytic hierarchical process to obtain a cross-subject ranking, which revealed the close correlation between TOVA and the visual pathway in brain.

Up to now many neural network models have been proposed. In our study we focus on two kinds of feedforward networks: strictly feedforward networks and Kohonen's self-organizing mappings where lateral competition is introduced. The two kinds of feedforward networks have played a fundamental role in neural networks research and application. Doctor of Philosophy (EEE)

Resource management (RM) plays a critical role for QoS provisioning in the design of mobile wireless networks. In this thesis, we focus on the research of two key RM issues: call admission control (CAC) and bandwidth allocation (BA), for three main RM optimization problems in multi-service mobile wireless networks. We first study the MAXU problem, which is defined as maximizing system utilization subject to constraints on call blocking probabilities. Since it is difficult to promptly adjust bandwidth allocation on uplink and downlink according to the change of traffic load in the system, the mismatch of bandwidth allocation and traffic load results in low bandwidth utilization. In such environment, traditional CAC may admit superfluous Real-Time (RT) calls or Non-Real-Time (NRT) calls and thus lead to bandwidth waste. We propose and evaluate two new CAC schemes to address the low bandwidth utilization problems in such bandwidth asymmetry networks. By determining the admissible regions for RT calls and NRT calls, the proposed schemes prevent the calls of a specific class from overusing the bandwidth resources. Next, we focus on minimizing average system cost (MINCost) problem in multi-service mobile wireless networks. By modeling the admission control problem into a Markov decision process (MDP) and analyzing the corresponding value function, we obtain some monotonicity properties of the optimal policy. These properties suggest that the optimal admission control policy for the bandwidth asymmetry mobile wireless networks have a threshold structure and the threshold specified for a class of calls may change with system states. Due to the prohibitively high complexity for computing the thresholds in a system with large state-space, we propose a heuristic CAC policy called Call-Rate-based Dynamic Threshold (CRDT) policy to approximate the theoretical optimal policy based on the insights we obtain from the modeling and the analytical study on the properties of the optimal policy. Subsequently, we turn to study the problem of minimizing new call blocking probabilities with hard constraints on handoff call blocking probabilities (MINBlock) in multi-service mobile wireless networks. We propose a new Distributed Multi-service Admission Control scheme (DMS-AC) to handle the MINBlock problem in multi-service mobile wireless networks. In order to satisfy the QoS requirements of different call classes in a dynamic traffic load environment, we also propose bandwidth re-allocation as a complementary mechanism for CAC in bandwidth asymmetry mobile wireless networks. Based on the proposed DMS-AC scheme, we investigate when and how to adjust bandwidth allocation on uplink and downlink in a multi-service mobile wireless network with bandwidth asymmetry under dynamic traffic load conditions. With the designed bandwidth re-allocation scheme in conjunction with the proposed CAC, the QoS requirements of different call classes can be guaranteed under dynamic traffic conditions and in the mean time the system bandwidth utilization is improved significantly. Our work in this thesis is an essential extension for resource management in the design of multi-service mobile wireless networks, especially for bandwidth asymmetry mobile wireless networks. By studying and analyzing the special features of the multi-service mobile wireless networks, we investigate main call level RM optimization problems in the new environment, and propose some efficient and effective RM schemes based on comprehensive analysis and mathematical models. We believe that our work can bring some insights to the research work in the area of RM design in multi-service mobile wireless networks. DOCTOR OF PHILOSOPHY (EEE)

Artificial neural network, or commonly referred to as ''neural network'', is a successful example of how human nature has led technology. However, traditional learning algorithms in neural network require iterative parameter tuning and often suffer from problems like local minimum and slow convergence. Extreme learning machine (ELM) is able to overcome all the problems above. Proposed as a learning algorithm for the single-hidden layer feedforward neural networks (SLFNs), ELM was later extended to the ''generalized'' SLFNs where the hidden nodes might take wide types of forms not limited to neuron type. The main feature of ELM lies in the random hidden nodes. Moreover, the universal approximation theorem of ELM has guaranteed good performance as long as the hidden layer mapping is any bounded piecewise continuous function. Researchers on ELM have been seeking for some other methods to improve the generalization performance. Standard optimization method was thus considered in the realization of ELM. Not only better performance in classification was achieved, but also a fact was revealed that ELM and SVM are actually consistent from optimization point of view. The resultant ELM classifier based on standard optimization method was found with comparable performance as SVM. What's more, the implementation of ELM is much easier since the performance is insensitive to parameters. Afterwards, ELM was further analyzed from optimization point of view and solution of kernel version was derived. So far the unified framework of ELM has been formed that includes traditional neural networks, support vector networks, and regularized networks. Since the ELM theory is only developed since very recent years, there are plenty of places ELM can be applied. In this thesis, works of ELM successfully applied in real world applications, such as face recognition system and relevance ranking for information. In real world applications, the natural data is with different characteristics. For example, situations when data is not available at once or data is of large scale often arise. In this case, online sequential learning model of a machine learning technique is generally regarded as one typical solution. In this thesis, online sequential model based on ELM framework is provided so that not only all the advantages of ELM over other machine learning techniques are pertained but also the fore mentioned problems are solved. Another situation happens quite often is that the training data is not well balanced. Any normal machine learning technique that assumes well balanced data distribution is supposed with the tendency to bias the performance. In this case, weighted version of ELM is proposed as the most straightforward and efficient method to tackle such problem. DOCTOR OF PHILOSOPHY (EEE)

Medical ultrasound images are degraded representations of the true acoustical reflectors in the imaged anatomical tissue. In theory, the qualities of medical ultrasound images rely mainly on the collected ultrasound echo signals. If ultrasound echo signals were accurate representations of biological tissues, the resolution of medical ultrasound images would be improved correspondingly. However, ultrasound echo signals do not provide a direct description of biological tissues usually because they are regarded as the reflected echoes arising from the interactions between ultrasound system and biological tissues. In order to retrieve the true properties of tissues of interest, we need to resort to accurate models of ultrasound echo signals. A reasonable and accurate model is very instructive in understanding the essential characteristics of ultrasound imaging systems. This thesis focuses on the investigations of current models and development of novel models for ultrasound imaging systems. A theoretical model based on classical ultrasound acoustics was developed in 1970s. This model with integration format has been verified and cited for decades. However, from both the signal and system point of view, it purely concerns ultrasound physics and is difficult to be applied in engineering field. Since 1990s, a conventional convolution model was fully derived based on the theoretical model and presented an appealing format for engineers. In this convolution model, ultrasound echo signals are modeled mathematically as a spatio-temporal convolution between the spacevariant ultrasonic system impulse response or point-spread function (PSF) and the biological tissues, with the addition of observation noise introduced in the image formation process. Furthermore, simple discrete convolution model directly arising from continuous conventional model was proposed and widely applied in ultrasound deconvolution filtering techniques. The first contribution of our work is the development of a new convolution model with reasonable and feasible conditions. After careful studies, we find there is a flawed approximation in the conventional convolution model which plays the key role in the original derivations such that the original convolution model is not fully theoretically sound and valid. Consequently, the deconvolution-related techniques for processing ultrasound echo signals no longer have theoretical foundation. In this thesis, a new convolution model of medical ultrasound echo process is proposed and its derivation and formulation are provided. Based on the investigations of classical acoustical equations, the new convolution model presents dominant terms under common practically feasible conditions of medical ultrasound. It provides a new theoretical foundation for the prevailing deconvolution techniques in ultrasound signal processing and new insight in exploring interactions between ultrasound pulses and body tissues in ultrasound scanning and imaging. The second contribution of our work is the proposal of two new discrete models of ultrasound echo signals. The conventional discrete model does not provide a convincing discretization from conventional continuous convolution model. The reason behind is that time dimension is ignored deliberately during the discretization without strict proof. Meanwhile, tissue information is only conceptual 2-D sequences of signals without specific indications on how to connect the so-called tissue signals with real soft tissues' anatomy. Meanwhile, the simple discrete model is quite general and symbolic such that it is not mathematically straightforward and accurate enough. Besides, we have pointed out the problematic approximation in the derivation of conventional convolution model and proposed new convolution model with reasonable conditions. Hence, starting from the original theoretical model and new convolution model proposed in Chapter 3, two new discrete models and their careful derivations are presented, respectively. In comparison with the conventional discrete model, they are of familiar format of standard discrete systems. Also, inside the models, there are clear indications on how the tissue property parameters interact with the input impulse signals exactly. Hence, provided the proposed discrete models, the tissue property parameters can be separated with ease from the echo signals, which are exactly what are of ultrasound imaging interest. In summary, this thesis is a contribution to the development of novel ultrasound echo models for medical ultrasound imaging systems. Several new models and their extensive studies for medical ultrasound imaging are provided. Our work establishes a new foundation for ultrasound deconvolution techniques in modern signal processing. DOCTOR OF PHILOSOPHY (EEE)

Image Classification is one of the key computer vision tasks. Among numerous machine learning methods, we choose the Extreme Learning Machine (ELM) for our image classification applications. This thesis contributes to four aspects of ELM netwroks. From the view of efficient input data, we have designed handcrafted feature extraction method for smile images classification. From the perspective of the distribution of random weights between the input layer and hidden layer, we have proposed and proved the effectiveness of the sparse binary ELM. Inspired by the deep architecture of deep learning, we have extended the single layer to multiple layers of ELM to achieve better performance on large image classification datasets. Finally, from the point of target coding, we have introduced and evaluated different target coding methods for image classification. Doctor of Philosophy

The raindrop adhered to a camera lens could severely degrade images it captured, because that the raindrop pixels captured by cameras will replace the background pixels correspondingly. In the outdoor environment, such problem is much common, and this problem will worsen the outdoor surveillance’s performance. Thus this paper proposed a brand new Convolution Neural Network(CNN) +Recurrent Neural Network(RNN) method to recover the background information from the degraded images, and it could recover the degraded images in common situations. In this paper, CNN is used for extract the image feature for better processing, RNN is used for the reason that in every step, the information of derained image is considered useful for the next step. Since the raindrop is considered cannot be removed in just one stage, a four stages deraining method is used here. For faster processing of images for surveillance, an Extreme Learning Machining(ELM) method is also used. It can classify these surveillance images into two parts: degraded images and non-degraded images. The proposed CNN+RNN method will be used for the non-degraded image. In addition, this paper also explored the Generative Adversarial Networks(GAN) method in deraining task. All the training data and test data used in this paper are real world data. Master of Science (Computer Control and Automation)

This thesis is written based on two main topics: Scene semantic recognition and road lane marks recognition. The thesis first reviews the studies of general scene understanding or recognition method using semantic segmentation approaches, and then focuses on a more specific topic of analyzing road scene images for detecting road lane marks. Semantic segmentation has been a popular topic in the field of computer vision research. The main purpose of semantic segmentation is to label pixels of interest in an image with corresponding categories of the objects. This thesis mainly focuses on scene recognition, a branch of semantic segmentation which takes more contextual information into consideration. This thesis presents an experimental study in which a multi-task method for scene recognition is proposed. In this method, edge information is used in enhancing recognition performance. A network which outputs both edge detection map and pixel-wise segmentation is designed. The network is based on FCN and the prediction branches of the two outputs are parallel. Each branch uses multi-scale features concatenation as the image representations. The method expects that the information from edge detection could contribute to the ability of extracting image features for pixel-wise segmentation. Modern approaches on multiple road lane marks detection are facing several problems. First, insufficient database make related solutions with machine learning technique difficult to train a robust model for application; second, current researches focus on single lane marks detection, which pays less attention to entire roads’ condition. To solve the problems, a database with proper ground truth of marks’ label set is constructed and a method is developed for detecting and classifying road lane marks of entire roads with Extreme Learning Machines (ELM). The implementation result shows promising performance and further improvement could be expected. Master of Engineering

Advanced Driving Assistant System (ADAS) was developed to reduce hazard on road, as drivers tend to get distracted from non-driving tasks. Researchers widely acknowledge that machine learning should be applied in ADAS, so that system can recognize driver’s state and adapt accordingly. Applying machine learning to physiological signals to learn psychological model is a common research topic. Yet, little work has considered the challenges in implementation, which is different from other machine learning domains. Usual approach is to collect many signals and go through tedious signal processing to output feature vectors. Machine learning plays its part only after features are available, which is costly and unlikely to be feasible in real world situation. We propose new machine learning based methods that only require simple single signal combined with manifold learning algorithms. Our methods are robust in that they require only one signal but don’t compromise the performance. The first contribution of the work is that we collected data from partly automated vehicle’s simulation, where machine learning has seldom been applied. The second contribution is that we proposed new feature extraction methods which only exploit ECG signal. Our methods do not require domain specific knowledge or tedious signal processing procedure. As long as intervals between R peaks are available (which can be measured easily with cheap commercial equipment), our manifold learning based feature extractor will provide reliable features. As this implies no pre-processing, it is more beneficial for implementation. Master of Engineering

Clustering and Semi-Supervised Classification (SSC) algorithms can make use of unlabeled training data and thus have the potential to alleviate labeling costs. For example, Extreme Learning Machine (ELM) was recently extended to semi-supervised learning and clustering with promising performance. Meanwhile, it is either costly or infeasible to obtain labeled training samples in some real-world applications. The thesis investigates clustering and SSC algorithms with application to driver distraction detection. Firstly, the thesis investigates embedding-based clustering. The desirable properties of embedding are reviewed in the literature, e.g., preserving the intrinsic data structure and maximizing the class separability. To obtain better embedding for clustering, the thesis considers both properties together and develops a novel clustering algorithm referred to as ELM for Joint Embedding and Clustering (ELM-JEC). Experimental studies on a wide range of benchmark datasets have show that ELM-JEC is competitive with the related methods. Secondly, the thesis investigates graph-based clustering. One limitation of existing graph learning methods is that they adjust the graph based on either the original data or the linearly projected data, which may not effectively reveal the underlying low- dimensional structures. To address this limitation, this thesis develops dual data representations, i.e., the original data and their nonlinear embedding obtained via an ELM- based neural network, and uses them as the basis for graph learning. The resulting algorithm is named as clustering based on ELM and Constrained Laplacian Rank (ELM- CLR). The experimental results show that ELM-CLR outperforms other adaptive graph learning methods on most benchmark datasets. Finally, the thesis applies the proposed clustering algorithms, i.e., ELM-JEC and ELM- CLR, and several SSC algorithms to driver distraction detection. The clustering algorithms are used on unlabeled data to generate preliminary labels as reference to assist human experts in the labeling process. In terms of the clustering accuracy, both proposed clustering algorithms perform better or on par with the related algorithms. The best clustering accuracy is achieved by ELM-JEC. Moreover, the research question of “which type of SSC method is more suitable for driver distraction detection?” is answered by evaluating two popular types of semi-supervised methods on a real-world dataset of drivers’ eye and head movements. The experimental results show that the graph-based methods achieve twice the improvement by the low-density-separation based method. It has also been shown that 1) the graph-based methods reduce the required amount of labeled training data, and 2) the benefits in detection accuracy increase with the size of unlabeled datasets. Overall, the thesis contributes two novel clustering algorithms by making use of ELM- based embedding and discovers that 1) better clustering performance on some datasets is expected, if the embedding preserves the intrinsic local structure and maximizes the class separability simultaneously, and 2) Both original and nonlinear embedded spaces are crucial to learning graphs with clear clusters. Moreover, the thesis contributes to the research on driver distraction detection by putting forward a semi-supervised driver distraction detection system with efficient labeling assistance and verifies it on an on- road driver distraction dataset. Doctor of Philosophy

With the explosive growth of Internet and computing technology, human beings are confronted by a great amount of unstructured text data. The need to extract useful knowledge from the data also grows. Researchers in the natural language processing community have delivered many marvelous technologies for various applications, such as information retrieval, machine translation, sentiment analysis, etc. Traditional methods usually rely on rigid language assumptions and require great efforts and time to be devoted to feature engineering. The research goal of this thesis is to develop machine learning models that can automatically learn semantic representations from texts with few or no human interventions. The models proposed in this thesis can induce effective representations for sentences or documents which are used to solve high-level language understanding tasks. The models are shown in four main chapters in this thesis according to the tasks they are addressing. The first task is document summarization which is addressed by two new approaches; after that, another two innovative algorithms are proposed for sentiment analysis and sentence modeling respectively; at last, one model is developed for human demography prediction. However, the models are never limited to these applications but can easily generalized to diverse natural language understanding tasks. The core of all the models lies in learning good semantic representations. Document summarization is aimed at generating a brief summary for a long document or a set of documents. In this thesis, the task is transformed into a regression problem which ranks sentences by saliency scores. Methods are explored to represent sentences as vectors so as to obtain scores of sentences by a regressor. The first model leverages on word embedding to represent sentences so as to avoid the intensive labor of feature engineering. A new technique, termed window-based sentence representation, is proposed and achieves satisfactory summarization performance compared with baseline methods. However, the representation power is still weak because of its simple structure. To improve the representation capability, we employ deep learning algorithms and develop an innovative variant of the convolutional neural network, namely multi-view convolutional neural network which can obtain the features of sentences and rank sentences jointly. The performance of the new model is evaluated on five benchmark datasets and demonstrates better performance than the state-of-the-art approaches. The second natural language understanding task addressed in this thesis is sentiment analysis which has been applied to recommender systems, business intelligence and automated trading, etc. A new architecture termed comprehensive attention recurrent model is developed to access comprehensive information contained in sentences. The model employs the recurrent neural network to capture the past and future context information and the convolutional neural network to access local information of words in a sentence. Empirical results on large-scale datasets demonstrate that the new architecture effectively improves the prediction performance compared with standard recurrent methods. The sentence modeling problem is at the core of many natural language processing tasks whose main objective is to learn good representations for sentences. Actually the objective of the thesis is to learn good semantic representations for texts. Therefore, this task lies at core and is the foundation of the other three tasks addressed in this thesis. One innovative model combining the bidirectional long-term short memory and convolutional structures is developed for the problem. A new pooling scheme for the convolutional neural networks, which better retains significant information than the popular max pooling method, is proposed by leveraging on attention mechanism. The model achieves state-of-the-art performance on seven benchmark datasets for text classification. At last, a simple but effective document representation approach is designed for predicting demographic attributes of web users based on their browsing history. I put this task at the last position because it is a practical application of natural language understanding. The new representation approach exploits word embedding and term frequency-inverse document frequency weighting scheme and owns both the power of word embedding capturing semantic and syntactic information and the statistical nature of term frequency-inverse document frequency. Experimental results demonstrate that the new representation method is more powerful than other feature representation methods including sophisticated deep learning models for this task. Doctor of Philosophy

Attention is constantly required in many daily life tasks. Attention-related behavior, such as driving distraction, has been reported as a major reason in traffic accidents. Therefore, the recognition of attention can enhance task performance. Electroencephalogram (EEG) is used to study attention, since it provides a direct measure of the brain activity with high temporal resolution at 1-10ms. In this thesis, we study the recognition of selective attention and sustained attention (vigilance) using machine learning. We start by proposing an experiment which aims to recognize unattended and attended conditions induced by Test of Variables of Attention (TOVA), using EEG features supported by the event-related potential (ERP) literature. However, ERP does not work for real-time applications where the external stimuli (event) required by ERP cannot be controlled. In face of the low signal-to-noise ratio (SNR) in the non-ERP approach, we propose Channel Selection with Different Features (CSDF) algorithm, which selects channels with their own different feature sets, as well as restricts features to as few channels as possible. Using CSDF, 83 out of 868 features are selected to distinguish the unattended and attended conditions. The accuracy 94.3% (±5.6%) is the best compared to other feature selection and channel selection algorithms. Based on CSDF, we find that the first and second order difference in the left parietal and temporal lobes, as well as the Higuchi fractal dimension and mean signal amplitude in the right frontal lobe, are relevant to selective attention. Unlike selective attention which has discrete conditions such as attended/unattended, the vigilance stages cannot be easily observed. Analogous to sleep stages, we want to define the vigilance stages in open eye and situation-aware state in a subject-independent and data-driven way. In the literature, there are vigilance stage models defined under closed eye. However, the EEG signals are more complex in open eye and situation-aware state. Extreme learning machine autoencoder (ELMAE) is used to learn the EEG spectral features and define the vigilance stages during simulated driving. Results show that ELMAE is an efficient alternative to restricted Boltzmann machine (RBM) in vigilance recognition: ELMAE achieves root mean square error at 0.189 (±0.049), which is better than RBM at 0.195 (±0.046); and training speed significantly faster than RBM. Based on ELMAE, we define three vigilance stages in open eye and situation-aware state. Stage I is high vigilance, where the subject is attentive. Stage II is low vigilance, which is further divided into Stage II.1: drowsiness and difficulty in attention allocation; and Stage II.2: distraction instead of falling asleep. A possible explanation for stage II.2 is that, the environment contains not enough external stimuli to keep the open eye and situation-aware state, so that the brain performs vigilance regulation to seek external stimuli, and hence leading to distraction. A major limitation of ELMAE is that it learns nonsparse hidden weights and features. Analogous to feature selection using CSDF, we want to learn sparse features so that the useful information is restricted to a few nonzero features to achieve better interpretability. To address this issue, a novel bio-inspired algorithm, joint weight-delay spike-timing dependent plasticity (joint STDP), is proposed for learning sparse hidden weights and EEG spectral features. Compared to other nonsparse and sparse algorithms, joint STDP achieves the highest level of sparseness in both learned features and hidden weights significantly. On the other hand, joint STDP has slightly larger root mean square error at 0.206 (±0.061), due to the trade-off between performance and sparseness. Extensive experiments and comparisons are carried out to evaluate all the proposed algorithms. The experimental results confirm the advantages of the proposed algorithms, hence making contribution to EEG-based attention research. This thesis is interdisciplinary and includes several fields such as machine learning, computational neuroscience, brain-computer interface and psychology. Doctor of Philosophy (IGS)

The machine learning techniques have been extensively studied in the past few decades. One of the most common approaches in machine learning is the Artificial Neural Network (ANN), or commonly referred as Neural Network (NN) that is inspirited by how the human brain functions. Many learning algorithms and paradigms have been developed for Neural Network since 1940s. However, most of the traditional neural network learning algorithms suffer from problems like local minima, slow learning rate and trivial human intervene. Extreme Learning Machine (ELM) proposed by Huang et al. in 2004 is an emergent technology that has great potential of overcoming the problems faced by traditional neural network learning algorithms. ELM is based on the structure of the ``generalized'' single hidden-layer feedforward neural networks, where the hidden node parameters are randomly generated. In the aspect of the standard optimization method, the ELM problem could be formulated as an optimization problem that is similar to the formulation of Support Vector Machine (SVM)'s optimization problem. However, SVM tends to obtain a solution that is sub-optimal to ELM's. With the finding of the relationship between ELM and SVM, we could extend ELM to many of SVM's variants. In the work presented in chapter 3, the equality constrained approach from both Least Square SVM and Proximal SVM was adopted in the optimization method based ELM. By implementing the equality constraints in its optimization equations, ELM can provide a unified solution to different practical applications (e.g. regression, binary and multiclass classifications). ELM could also provide different solutions based on the application size thus to reduce the training complexity. The kernel trick can also be used in ELM's solution. As supported in theory and by simulation results, ELM tends to have better generalization performance than SVM and its variants when the same kernel functions are used. The equality constrained optimization method based ELM has shown promising results in the benchmark datasets. It is also important to test its performance in real-world applications. In chapter 5, the kernel based ELM was implemented in credit risk evaluation for two credit datasets. Simulation results showed that the kernel based ELM was more suitable for credit risk evaluation than the popular Support Vector Machines with consideration of overall, good and bad accuracy. Compared with other machine learning techniques, ELM has greater potential of solving larger scale dataset problems due to its simple network structure. However when solving very large data problems, ELM requires a large number of hidden nodes to map the data to higher dimensional space where the data can be separated well. The large number of hidden nodes result in a large hidden layer matrix, which usually requires very large memory during computation. In chapter 6, a stacked ELMs (S-ELMs) method was proposed to solve the memory issue. Instead of using one single ELM with large hidden layer matrix, we broke it into multiple small ELMs and connected them serially. The stacked ELMs not only reduces the computational memory requirement, but also saves the training time. The generalization performance of S-ELMs can be further improved by implementing the unsupervised pretraining approach (usually an autoencoder) in each layer. The work presented in chapter 7 shows that when adding ELM based autoencoder to each layers of S-ELMs network, the testing accuracy could be significantly improved. DOCTOR OF PHILOSOPHY (EEE)

In this thesis, we present further studies of extreme learning machine and signal detection in compressed sensing. In Chapter 1, we give literature reviews of extreme learning machine (ELM) and compressed sensing (CS). In part I of the thesis (Chapters 2, 3, and 4), we consider the recent ELM for training neural networks and present several improved algorithms. We first propose a composite function wavelet neural network (WNN) learning with the recent ELM algorithm in Chapter 2. The main contributions of the proposed WNN comparing with traditional ones are using composite functions at the hidden nodes and applying ELM algorithm to WNN as a learning algorithm. To reduce the network size and optimize the hidden node parameters, we then introduce an improvement method for training the proposed WNN by incorporating the global optimization algorithm Differential Evolution into searching for the optimal network input weights and the dilation and translation values in Chapter 3. To further enhance the classification rate of the ELM, we propose an improved algorithm named voting based ELM (V-ELM) for signal classification in Chapter 4. In V-ELM, the voting method is incorporated into the ELM in classification applications. Several individual ELMs with the same network structure are trained with the same dataset and the final class label of a test sample is determined by majority voting method on all the results obtained by these independent ELMs. Numerical simulations are provided to illustrate the efficiency of our proposed methods. In part II of the thesis (Chapters 5 and 6), we study the signal detection in CS. We first consider the theoretical bound of the probability of error by detecting the signal reconstructed in CS with the Bayesian approach in Chapter 5. Utilizing the oracle estimator in CS, we provide a theoretical bound of the probability of error when the noise in CS is white Gaussian noise. We then consider the Bayesian approach to signal detection in CS using compressed measurements directly in Chapter 6. We start by revisiting the classical signal detection problem and show that with an additive Gaussian noise, the probability of error for unequal prior probabilities of the hypotheses is always smaller than the one with equal prior probability. We then consider signal detection with compressed measurements directly, assuming that the additive noise is Gaussian but with unequal variances. A general expression is obtained for the probability of error where the prior probabilities could be equal or unequal. We have also derived performance bounds for the probability of error using the restricted isometry property constant and then the computationally more feasible mutual coherence of a given sampling matrix in CS. An approximate but simpler expression of the probability of error and its approximate upper bound are also obtained. % the measurement domain which is easier to calculate than computing the Numerical simulations are given to verify the new theoretical results. In Chapter 7, conclusions and future work are provided. DOCTOR OF PHILOSOPHY (EEE)

Computational intelligence techniques have been extensively explored in wide applications in the past three decades. Out of numerous computational intelligence techniques, neural networks have been playing the dominant roles. However, it is known that neural networks usually face some challenging issues such as local minima, slow learning rate, and trivial human intervene. Extreme Learning Machines (ELMs) as an emergent technology, which overcome some challenges faced by other techniques, study a much wider type of ``generalized'' single-hidden layer feedforward networks (GSLFNs). The essence of ELMs is that (1) the hidden layer of ELMs need not be iteratively tuned so that the training time required is dramatically reduced and the human intervene is eliminated; (2) the hidden layer feature mapping need to satisfy the universal approximation condition so that the approximation capability of ELMs can be guaranteed; (3) both of the training error and the norm of output weights need to be minimized so that better generalization performance can be obtained. Although a plenty of simulation results have revealed that the performance of ELMs are relatively stable in a wide range of the number of hidden nodes, it may be hard to tell what the exact and proper range of the number of hidden nodes should be for a specific task. In addition, based on achieving the equal performance, it is apparent that the less the number of hidden nodes is, the more efficient and less computationally expensive ELMs will be, and hence the convergence rate of ELMs raises. Therefore, finding an appropriate and parsimonious network which can be best suited to the given problem, and further exploring its corresponding approximation capability, are still the important issues in the study of ELMs. The original incremental extreme learning machine (I-ELM) was proposed where the hidden nodes are randomly added one by one and the output weights of the existing hidden nodes are frozen when a new hidden node is added. Convex incremental extreme learning machine (CI-ELM) adopts another incremental method and allows to properly adjust the output weights of the existing hidden nodes when a new hidden node is added. CI-ELM can achieve faster convergence rate than I-ELM while retaining the simplicity of I-ELM. However, such convex combination of the existing hidden nodes and the newly added hidden node may not be the optimal fashion by which the residual error can be reduced largest. On the basis of this observation, an incremental extreme learning machine with optimal incremental coefficients (OCI-ELM) has been proposed in this thesis. Such OCI-ELM can not only provide an optimal tradeoff between the existing hidden nodes and the newly added hidden nodes but also work as an universal approximator. Although different architecture-adjustable ELMs by using either pruning method or constructive method are engaged in addressing the problem of network acquisition, they investigate only restricted topological subsets rather than the complete class of network architectures. Inspired by this cognition, an extreme learning machine with adaptive growth of hidden nodes (AG-ELM) where a novel approach to determining the network architecture in an adaptive manner has been introduced in this thesis. AG-ELM realizes the automatic design of networks in a search space of possible structures which are suitable to the problem. Furthermore, the obtained universal approximation theory of AG-ELM not only strengthens those existing results, but also provides a fundamental theory for studying the approximation capability of algorithms which use random mechanism for the parameter searching in neural networks. With the exception of I-ELMs, EM-ELM is another efficient algorithm that allows to add random hidden nodes one by one or group by group (with varying size) and incrementally updates the output weights during the network growth. However, in the implementation of both I-ELMs and EM-ELM, the number of the hidden nodes always monotonically increases with the learning progress and the final number of the hidden nodes is equivalent to the learning steps. Then large number of hidden nodes will be obtained eventually if there needs many iterative steps, while some of the hidden nodes may play a very minor role in the network output. On the other hand, although AG-ELM can automatically determine the network architecture, it is not so efficient as expected. Therefore, in order to choose the right size network automatically as well as improve the efficiency of ELMs, a dynamic extreme learning machine (D-ELM) performing dynamic growth of hidden nodes and incrementally updating of output weights has been proposed in this thesis. On the basis of the theory obtained in AG-ELM, it has been proved that the obtained D-ELM can approximate any Lebesgue $p-$integrable function as long as the hidden activation is Lebesgue $p-$integrable. DOCTOR OF PHILOSOPHY (EEE)