Search

Recommender systems are the backbones of a variety of critical services provided by tech-heavy applications and companies. In social media applications such as Facebook, Instagram, TikTok, and Snapchat, recommender systems of different types are leveraged to suggest the next post, image, or video to users to their satisfaction. Online shopping websites, such as Amazon, eBay, and Taobao, recommend items to users so that they can immediately find what they favor without the need for intensive querying. Due to its outstanding significance, both academia and industry put great effort into developing more powerful recommendation engines.In this dissertation, we aim at improving recommender systems via different ways of incorporating data from multiple modalities such as the graphical structure of the entity relations, the attributes of entities, and the textual reviews to items from users. We exemplify the process of incorporating multimodal data via five works completed during my Ph.D. study. In these works, we will demonstrate the incorporation of different data modalities for different recommendation scenarios. NeRank focuses on the question routing task that recommends experts to question raisers combining user expertise and structural relations of entities. InterHAt considers the polysemy of features to build an interpretable click-through rate predictor. GEAPR, specialized in point of interest recommendation, decomposes the user motivation by data modalities such as social network, attribute information, and geolocation. The framework of ASPE+APRE presents a possibility to objectively understand the preference of users through what they said rather than what they purchased, clicked, or viewed. Using the objective information, recommender systems can obtain a detailed and fine-grained picture of user interests and item properties.This framework handles the descriptive statements of reviews leaving the comparative statements unattended.Finally, we introduce SAECON that deals with comparative statements and analyzes the reviews with larger coverage.The research effort demonstrates that incorporating data from multiple modalities can hugely improve the performance of recommendations. In addition, it provides recommendation engines with interpretability to decompose the motivation behind certain user behaviors when using the service. It can be envisioned that the fusion of multimodal data will inspire the development of recommender systems in both academic research and industrial practice.

This article focuses on the investigation of finite-time dissipative state estimation for Markov jump neural networks. First, in view of the subsistent phenomenon that the state estimator cannot capture the system modes synchronously, the hidden Markov model with partly unknown probabilities is introduced in this article to describe such asynchronization constraint. For the upper limit of network bandwidth and computing resources, a novel dynamic event-triggered transmission mechanism, whose threshold parameter is constructed as an adjustable diagonal matrix, is set between the estimator and the original system to avoid data collision and save energy. Then, with the assistance of Lyapunov techniques, an event-based asynchronous state estimator is designed to ensure that the resulting system is finite-time bounded with a prescribed dissipation performance index. Ultimately, the effectiveness of the proposed estimator design approach combining with a dynamic event-triggered transmission mechanism is demonstrated by a numerical example.

In the industrial batch process monitoring domain, the conventional multivariate monitoring methods may not always function well in monitoring faults that have both Non-Linear and Non-Gaussian properties. To enhance the monitoring capability, the adversarial auto-encoder (AAE) was introduced to increase the sensitivity to Non-Gaussian anomalies by projecting non-Gaussian information into a given Gaussian distribution feature space. At the same time, low-dimensional feature space can avoid the problem of “Concentration of measure” and improve the ability to distinguish minor small abnormalities. Therefore, A novel statistic index was constructed in the feature space based on the k-nearest neighbor rule (KNN) to improve the ability of minor fault monitoring. The proposed model is compared with the traditional multivariate statistical process monitoring methods in numerical examples and penicillin fermentation platform, which proves that it has better monitoring ability for minor magnitude and non-Gaussian faults.

Understanding image content is one of the ultimate goals of computer vision, and effectively and efficiently extracting features from images is a key component of all vision research. This thesis discusses methods related to an image-patch based approach to this feature analysis. Image-patch based methods have attracted a lot of interest for the analysis of a single images in application areas such as visual object recognition, image denoising, and super-resolution computation. The basic idea is to treat a single image as a collection of independent image patches, each of which can be encoded by, for example, a sparse coding model. The global characterization of that image is attained by aggregating the patch codes, which brings some level of shift-invariance and robustness to image noise and signal degradation. In this thesis, a new scheme, \textit{scene geometry-aware image-patch modeling}, based on the concept of a \textbf{patch-cube}, is proposed to model image patches in a light field, rather than in a single image. A light field is a collection of images all acquired at the same instant, providing a set of perspectives on the scene as though observing all of the light information that passes through a windowing portal (clearly with some discretization and sampling). The scene geometric information is implicitly incorporated in our modeling process, including depth and occlusion, without explicit knowledge of 3D scene structure. These extra constraints on the scene geometry empower our learned features to be less affected by image noise, lighting conditions, etc. As demonstration, we apply our method to joint image denoising and joint spatial/angular image super-resolution tasks, where its use of the light field will be seen to permit it to outperform its image-patch based counterparts. Here, a 2D camera array with small incremental baselines is used to capture the light field data, and this analysis is the majority of what we report. Additionally, working with real data from real light-field cameras, we present novel and highly effective methods for the calibration of these camera arrays.In common with the single-image model, learning a good "dictionary" plays a very important role in our work -- selecting an appropriate set of features that can provide succinct representations of a scene. Inspired by the success of the image patch-based method \cite{NGSingle}, we show that feature extraction for image patches is closely related to the low-rank kernel matrix approximation using the Nystrom method. The dictionary in sparse coding, or cluster centers in K-means clustering, are actually landmark points which can better capture the underlying higher-dimensional (manifold) structure of the data. Based upon this observation, our contribution is two fold: 1) an efficient algorithm to perform Kernel Principle Component Analysis feature extraction using landmark points, and 2) an alternative method for finding better landmark points based on \textit{Generalized Extreme Value distribution}s, GEV-Kmeans.

Group activity recognition has received significant interest due to its widely practical applications in sports analysis, intelligent surveillance and abnormal behavior detection. In a complex multi-person scenario, only a few key actors participate in the overall group activity and others may bring irrelevant information for recognition. However, most previous approaches model all the actors’ actions in the scene equivalently. To this end, we propose a relation-guided actor attention (RGAA) module to learn reinforced feature representations for effective group activity recognition. First, a location-aware relation module (LARM) is designed to explore the relation among pairwise actors’ features in which appearance and position information are both considered. We propose to stack all the pairwise relation features and the features themselves of an actor to learn actor attention which determines the importance degree from local and global information. Extensive experiments on two publicly benchmarks demonstrate the effectiveness of our method and the state-of-the-art performance is achieved.

This paper proposes a low-power low-cost on-chip digital background calibration for a pipelined ADC. This new redundant-stage calibration algorithm reduces the effect of quantization noise and can be applied for multiple stages; hence, it improves the calibration accuracy and is easily implemented fully on-chip with low power and low hardware cost. We realize the proposed calibration technique in a prototype 12-bit 250-MS/s pipelined ADC fabricated in a 55-nm technology. The measured results show that the prototype ADC, with an active area of 1310 $\mu \text{m}\,\,\times 510\,\,\mu \text{m}$ , achieves an signal-to-noise-and-distortion ratio of 66.7 dB [effective number of bits (ENOB) = 10.8 bit] and consumes a total power of 85 mW with a sampling rate of 250 MS/s after applying our digital calibration, where the on-chip digital calibration circuit consumes only 5 mW and an active area of $360\,\,\mu \text{m}\,\,\times 510\,\,\mu \text{m}$ .

In order to ensure the safety of transmission lines, the use of unmanned aerial vehicle (UAV) images for automatic object detection has important application prospects, such as the detection of birds&rsquo, nests. The traditional bird&rsquo, s nest detection methods mainly include the study of morphological characteristics of the bird&rsquo, s nest. These methods have poor applicability and low accuracy. In this work, we propose a deep learning-based birds&rsquo, nests automatic detection framework&mdash, region of interest (ROI) mining faster region-based convolutional neural networks (RCNN). First, the prior dimensions of anchors are obtained by using k-means clustering to improve the accuracy of coordinate boxes generation. Second, in order to balance the number of foreground and background samples in the training process, the focal loss function is introduced in the region proposal network (RPN) classification stage. Finally, the ROI mining module is added to solve the class imbalance problem in the classification stage, combined with the characteristics of difficult-to-classify bird&rsquo, s nest samples in the UAV images. After parameter optimization and experimental verification, the deep learning-based bird&rsquo, s nest automatic detection framework proposed in this work achieves high detection accuracy. In addition, the mean average precision (mAP) and formula 1 (F1) score of the proposed method are higher than the original faster RCNN and cascade RCNN. Our comparative analysis verifies the effectiveness of the proposed method.

The application of ultrasound (US) imaging in orthopedic surgery has always been a research direction. However, the various problems of US imaging hinder the development of computer assisted orthopedic surgery guided by US. US bone segmentation has been an important yet challenging task for many clinical applications. We propose a new end-to-end fully convolution network called BoneNet for real-time and accurate segmentation of bone surface from US image. The BoneNet employs the squeeze-and-excitation residual to realize a robust feature learning. In order to speed up the segmentation, we reduce the convolution kernel and used depth-wise separable convolution to reduce network parameters. In addition, we assessed the impact of different normalization operations and loss functions on network performance. Finally, we realize the comparison of the segmentation performance and generalization ability of the existing real-time US bone surface segmentation network under the unified dataset. We achieved an average Dice coefficient of 93.03 % on segmentation performance test, and 91.25 % on the generalization ability test. The results show that our proposed method ensures the real-time performance and achieves significant improvements in accuracy, which substantially outperformed the state-of-the-art.

With the development of China's economy, the number of private cars has increased significantly, leading to heavy traffic pressure on the road network and a serious decline in traffic efficiency, which has become a shackle hindering urban development. This paper takes traffic data as the research object, establishes a DBN network model with a top-level predictor, and extracts three influencing factors affecting traffic flow, namely lane number, time ratio of red and green signal lights, and whether to turn left, through learning the past traffic data. On this basis, SVR algorithm is combined with the above traffic impact factors to make short-term prediction of the future traffic flow of the road section. Finally, the prediction results are used to optimize the road efficiency to improve the road network efficiency. In this paper, a simulation experiment is carried out on the real road network data of jinan city, and the experimental analysis proves that the method proposed in this paper can effectively improve traffic congestion and improve traffic efficiency. This method has important theoretical significance and practical value for intelligent road construction.

The wastewater treatment process (WWTP) is a complex biochemical reaction process that features highly nonlinear, non-Gaussian and time correlation. As a new monitoring method, deep recurrent neural net-work (DRNN) has an effective performance in dealing with the nonlinear and time correlation of the data, but it is insufficient in dealing with the non-Gaussian characteristics. In this study, an effective deep recurrent network with high-order statistic information (HSI-DRN) is proposed for solving the insufficiency in dealing with the non-Gaussian characteristics. The proposed method extracts the high-order statistics characteristics by the over-complete independent component analysis (OICA) method. After that, weights in DRNN can be trained based on the obtained high-order statistics information and their corresponding fault labels. Because of the architecture of the network, the ability of extracting the non-Gaussian characteristics by HSI-DRN can be improved by the high-order statistics characteristics. Finally, HSI-DRN can generate visual monitoring results by discretizing the output data, which leads to more intuitively reflection of faults. Simulation studies on the BSM1 model has been performed to verify the performance of the method. For the different faults, the proposed method have higher fault monitoring ability with average false alarm rate (FAR) and missed detection rate (MAR) for respective 0.0215% and 0.586% when compared to other state-of-the-art fault monitoring methods.

Thin-walled cold-formed steel (TWCFS) structures have a great application prospect and are widely adopted in low-rise residential constructions due to great mechanical and economical properties. Since TWCFS member has low flexural rigidity and is hard to weld, it is rarely used in bridge engineering. In Poyang Lake Wetland of Jiangxi Province, China, a TWCFS footbridge was built. In this paper, Serial tests including static loading test, natural frequency test and pedestrian comfort test were conducted. At the same time, finite element analysis (FEA) models were built with Midas/Civil software. The data shows that although natural frequency is far greater than the range of pedestrian walking frequency and will not lead to the resonance of bridge, the peak acceleration under crowd load reflects poor pedestrian comfort. The structure was modified for improving pedestrian comfort and the results after modification verify that the comfort can be ensured.

When doubly-fed induction generator (DFIG)-based wind turbines operate in the islanded mode, nonlinear loads may cause voltage distortion at stator side. A control method that can improve the quality of stator voltage is proposed in this paper. First of all, the mathematics model of DFIG is established in the fundamental frequency synchronous reference frame. Then, the proportion integration resonance (PIR) controller is designed for rotor side converter (RSC) to suppress the 5th and 7th harmonic components of stator voltage. Finally, the simulation results validate the effectiveness and advantages of the proposed control strategy.

Mobile service and wearable devices have become a new hotspot in recent year. People also pay more attention to the problem of e-health. Cardiovascular disease is a kind of serious disease that jeopardizes the health and life of human beings which has been widespread concern by academic and practice researchers. But the automated monitoring and prevention of heart disease, cardiovascular disease and other issues have not been effectively resolved. In this paper, we combine Cloud Computing, Internet of Things, large data analysis with ECG Monitor technologies, by using mobile medical as a platform to design Internet of Things based Cloud ECG intelligent monitoring system. The development of the system not only in time to capture and send abnormal ECG signal, but also improve heart disease and cardiovascular disease in patients with the treatment of timeliness and success rate by using the big data processing technology. Our experiments show that, it can effectively monitor and protect the lives of patients with heart disease and cardiovascular disease.

With the rapid development of social security service, people have higher demands for medical service. Therefore, it is the development direction of mobile medical construction to build humanized and personalized service concepts. Mobile Health is of great significance to the society and health. Because of the slow running and unsupported distribution, the traditional computer can't solve the current issue of big data processing in healthcare industry. Mobile medical based on mobile cloud computing environment can be a good solution to these problems. The research of k Nearest Neighbor (kNN) query in mobile cloud computing environments has become a popular topic. Supporting scalable and distributed spatial data indexes has a great impact on the efficiency of KNN queries. The existing query methods are not suitable for parallelization or lead to redundancy of content. In this paper, firstly, we introduce an inverted Voronoi based kNN query processing with MapReduce. Such a query tends to be applied in various fields including m-health treatment where medical services can be provided by hospitals, clinics and other medical institutions to meet more people's medical requirements in time. Secondly, we introduce a mobile-health call system based on kNN query. Thirdly, we build a novel efficient spatial data index: Inverted Voronoi Index, which contains both inverted index and Voronoi graph. Finally, we present the outcomes of extensive experiment that are gained by both real and simulated data sets which indicate efficiency and scalability of the proposed approach.

Preoperative path planning plays a critical role in vascular access surgery. Vascular access surgery has superior difficulties and requires long training periods as well as precise operation. Yet doctors are on different leves, thus bulky size of blood vessels is usually chosen to undergo surgery and other possible optimal path is not considered. Moreover, patients and surgeons will suffer from X-ray radiation during the surgical procedure. The study proposed an improved ant colony algorithm to plan a vascular optimal three-dimensional path with overall consideration of factors such as catheter diameter, vascular length, diameter as well as the curvature and torsion. To protect the doctor and patient from exposing to X-ray long-term, the paper adopted augmented reality technology to register the reconstructed vascular model and physical model meanwhile, locate catheter by the electromagnetic tracking system and used Head Mounted Display to show the planning path in real time and monitor catheter push procedure. The experiment manifests reasonableness of preoperative path planning and proves the reliability of the algorithm. The augmented reality experiment real time and accurately displays the vascular phantom model, planning path and the catheter trajectory and proves the feasibility of this method. The paper presented a useful and feasible surgical scheme which was based on the improved ant colony algorithm to plan vascular three-dimensional path in augmented reality. The study possessed practical guiding significance in preoperative path planning, intraoperative catheter guiding and surgical training, which provided a theoretical method of path planning for vascular access surgery. It was a safe and reliable path planning approach and possessed practical reference value.

In this paper, a hierarchical structure combining a linear classifier based on the Mahalanobis distance and pairwise coupling (PWC) is proposed to effectively tackle multi-class classification problem. By taking the advantage of the distribution information of the dataset, both the recognition rate and speed are all improved. At the same time, the proposed method can estimate the posterior probabilities of a testing pattern more accurately than the classical PWC in multi-class cases. Experimental results on handwritten digit recognition demonstrate the effectiveness and efficiency of our method.