Search

Reinforcement Learning (RL) is an area of machine learning figuring out how agents take actions in an unknown environment to maximize its rewards. Unlike classical Markov Decision Process (MDP) in which agent has full knowledge of its state, rewards, and transitional probability, reinforcement learning utilizes exploration and exploitation for the model uncertainty. Under the condition that the model usually has a large state space, a neural network (NN) can be used to correlate its input state to its output actions to maximize the agent's rewards. However, building and training an efficient neural network is challenging. Inspired by Double Q-learning and Asynchronous Advantage Actor-Critic (A3C) algorithm, we will propose and implement an improved version of Double A3C algorithm which utilizing the strength of both algorithms to play OpenAI Gym Atari 2600 games to beat its benchmarks for our project.

Inspired by the recent success of application of dense data approach by using ORB-SLAM and RGB-D SLAM, we propose a better pipeline of real-time SLAM in dynamics environment. Different from previous SLAM which can only handle static scenes, we are presenting a solution which use RGB-D SLAM as well as YOLO real-time object detection to segment and remove dynamic scene and then construct static scene 3D. We gathered a dataset which allows us to jointly consider semantics, geometry, and physics and thus enables us to reconstruct the static scene while filtering out all dynamic objects.

Point clouds are rich geometric data structures, where their three dimensional structure offers an excellent domain for understanding the representation learning and generative modeling in 3D space. In this work, we aim to improve the performance of point cloud latent-space generative models by experimenting with transformer encoders, latent-space flow models, and autoregressive decoders. We analyze and compare both generation and reconstruction performance of these models on various object types.

Graph neural networks (GNN) extends deep learning to graph-structure dataset. Similar to Convolutional Neural Networks (CNN) using on image prediction, convolutional and pooling layers are the foundation to success for GNN on graph prediction tasks. In the initial PAN paper, it uses a path integral based graph neural networks for graph prediction. Specifically, it uses a convolution operation that involves every path linking the message sender and receiver with learnable weights depending on the path length, which corresponds to the maximal entropy random walk. It further generalizes such convolution operation to a new transition matrix called maximal entropy transition (MET). Because the diagonal entries of the MET matrix is directly related to the subgraph centrality, it provide a trial mechanism for pooling based on centrality score. While the initial PAN paper only considers node features. We further extends its capability to handle complex heterogeneous graph including both node and edge features.

Automatic music transcription (AMT) aims to convert raw audio to symbolic music representation. As a fundamental problem of music information retrieval (MIR), AMT is considered a difficult task even for trained human experts due to overlap of multiple harmonics in the acoustic signal. On the other hand, speech recognition, as one of the most popular tasks in natural language processing, aims to translate human spoken language to texts. Based on the similar nature of AMT and speech recognition (as they both deal with tasks of translating audio signal to symbolic encoding), this paper investigated whether a generic neural network architecture could possibly work on both tasks. In this paper, we introduced our new neural network architecture built on top of the current state-of-the-art Onsets and Frames, and compared the performances of its multiple variations on AMT task. We also tested our architecture with the task of speech recognition. For AMT, our models were able to produce better results compared to the model trained using the state-of-art architecture; however, although similar architecture was able to be trained on the speech recognition task, it did not generate very ideal result compared to other task-specific models.

We propose DeepSim, a reinforcement learning environment build toolkit for ROS and Gazebo. It allows machine learning or reinforcement learning researchers to access the robotics domain and create complex and challenging custom tasks in ROS and Gazebo simulation environments. This toolkit provides building blocks of advanced features such as collision detection, behaviour control, domain randomization, spawner, and many more. DeepSim is designed to reduce the boundary between robotics and machine learning communities by providing Python interface. In this paper, we discuss the components and design decisions of DeepSim Toolkit.

We propose Unified Distributed Environment (UDE), an environment virtualization toolkit for reinforcement learning research. UDE is designed to integrate environments built on any simulation platform such as Gazebo, Unity, Unreal, and OpenAI Gym. Through environment virtualization, UDE enables offloading the environment for execution on a remote machine while still maintaining a unified interface. The UDE interface is designed to support multi-agent by default. With environment virtualization and its interface design, the agent policies can be trained in multiple machines for a multi-agent environment. Furthermore, UDE supports integration with existing major RL toolkits for researchers to leverage the benefits. This paper discusses the components of UDE and its design decisions.

By investigating Yanjiao and Beijing sub-centre, on both sides of the Chaobai River, it reveals that today more than 40% of the population in Yanjiao travels to and from Beijing for work. Because of the inequities of political and administrative systems, as well as massive and uncontrolled urbanization, Yanjiao is losing its identity gradually. Comparing Yanjiao with Beijing Sub-centre on the urban configuration, building typology, topography, and hydrology, the conclusion is that there are: 1) traditional small-scale allays and buildings, 2) significant unidirectional commuter flow to Beijing, and 3) the Chaobai River is not only the administrative boundary but also the development boundary between the two cities. To optimize this situation, three different levels of planning and design have been made:1.Regional planning: To improve the sustainable development situation of Yanjiao through inter-regional synergistic development, while creating more equal urban spaces with a water-based landscape structure. 2.Urban Design: By extracting the traditional urban morphology, local lifestyles, three primary interventions have been created for the design part. Through these three interfaces, the site links the surrounding urban space with a good spatial transition.3.Detailed design: Intervention and feeling of spatial qualities are shown through the rendering of the three interfaces.

By investigating Yanjiao and Beijing sub-centre, on both sides of the Chaobai River, it reveals that today more than 40% of the population in Yanjiao travels to and from Beijing for work. Because of the inequities of political and administrative systems, as well as massive and uncontrolled urbanization, Yanjiao is losing its identity gradually. Comparing Yanjiao with Beijing Sub-centre on the urban configuration, building typology, topography, and hydrology, the conclusion is that there are: 1) traditional small-scale allays and buildings, 2) significant unidirectional commuter flow to Beijing, and 3) the Chaobai River is not only the administrative boundary but also the development boundary between the two cities. To optimize this situation, three different levels of planning and design have been made:1.Regional planning: To improve the sustainable development situation of Yanjiao through inter-regional synergistic development, while creating more equal urban spaces with a water-based landscape structure. 2.Urban Design: By extracting the traditional urban morphology, local lifestyles, three primary interventions have been created for the design part. Through these three interfaces, the site links the surrounding urban space with a good spatial transition.3.Detailed design: Intervention and feeling of spatial qualities are shown through the rendering of the three interfaces.

This thesis reviews the theoretical background of urban renaissance represented by Rogers’ research and report, was well as the theories and models of preservation and reuse of industrial heritages, to discuss the trends and principles of urban renewal process, and the concept and measures of protecting-utilization of industrial sites and buildings, and several successful examples are studied here to learn the application of these theories into real practice. On trying to apply the theories into the project of the Textile City Core Area, the background and the current situation of this area is analyzed, and according to the Master plan of the whole Textile City made by the Urban Planning and Design Research Institute of Xi'an University of Architecture and Technology, the master plan of the core area is made and presented here to create a synthetic district and revitalize the whole Textile City is presented. And a further design proposal is made focusing on the old industrial site and textile factories going to be reused as the Creative Factory, the Textile Museum and the Shopping Mall, the abandoned railway and the linear green field along with it having great potential to be transformed into public space, and a former urban village area that can be remodified into Street-shopping District. Key words: urban renaissance, industrial heritage, revitalization