Search

Programming robot behaviour in a complex world faces challenges on multiple levels, from dextrous low-level skills to high-level planning and reasoning. Recent pre-trained Large Language Models (LLMs) have shown remarkable reasoning ability in zero-shot robotic planning. However, it remains challenging to ground LLMs in multimodal sensory input and continuous action output, while enabling a robot to interact with its environment and acquire novel information as its policies unfold. We develop a robot interaction scenario with a partially observable state, which necessitates a robot to decide on a range of epistemic actions in order to sample sensory information among multiple modalities, before being able to execute the task correctly. An interactive perception framework is therefore proposed with an LLM as its backbone, whose ability is exploited to instruct epistemic actions and to reason over the resulting multimodal sensations (vision, sound, haptics, proprioception), as well as to plan an entire task execution based on the interactively acquired information. Our study demonstrates that LLMs can provide high-level planning and reasoning skills and control interactive robot behaviour in a multimodal environment, while multimodal modules with the context of the environmental state help ground the LLMs and extend their processing ability., See website at https://xf-zhao.github.io/projects/Matcha/

We study a class of reinforcement learning problems where the reward signals for policy learning are generated by a discriminator that is dependent on and jointly optimized with the policy. This interdependence between the policy and the discriminator leads to an unstable learning process because reward signals from an immature discriminator are noisy and impede policy learning, and conversely, an under-optimized policy impedes discriminator learning. We call this learning setting \textit{Internally Rewarded Reinforcement Learning} (IRRL) as the reward is not provided directly by the environment but \textit{internally} by the discriminator. In this paper, we formally formulate IRRL and present a class of problems that belong to IRRL. We theoretically derive and empirically analyze the effect of the reward function in IRRL and based on these analyses propose the clipped linear reward function. Experimental results show that the proposed reward function can consistently stabilize the training process by reducing the impact of reward noise, which leads to faster convergence and higher performance compared with baselines in diverse tasks., Comment: Accepted at ICML 2023. Project webpage at https://ir-rl.github.io

Explaining the behavior of intelligent agents such as robots to humans is challenging due to their incomprehensible proprioceptive states, variational intermediate goals, and resultant unpredictability. Moreover, one-step explanations for reinforcement learning agents can be ambiguous as they fail to account for the agent's future behavior at each transition, adding to the complexity of explaining robot actions. By leveraging abstracted actions that map to task-specific primitives, we avoid explanations on the movement level. Our proposed framework combines reward decomposition (RD) with abstracted action spaces into an explainable learning framework, allowing for non-ambiguous and high-level explanations based on object properties in the task. We demonstrate the effectiveness of our framework through quantitative and qualitative analysis of two robot scenarios, showcasing visual and textual explanations, from output artifacts of RD explanation, that are easy for humans to comprehend. Additionally, we demonstrate the versatility of integrating these artifacts with large language models for reasoning and interactive querying., Comment: https://lukaswill.github.io

Sound is one of the most informative and abundant modalities in the real world while being robust to sense without contacts by small and cheap sensors that can be placed on mobile devices. Although deep learning is capable of extracting information from multiple sensory inputs, there has been little use of sound for the control and learning of robotic actions. For unsupervised reinforcement learning, an agent is expected to actively collect experiences and jointly learn representations and policies in a self-supervised way. We build realistic robotic manipulation scenarios with physics-based sound simulation and propose the Intrinsic Sound Curiosity Module (ISCM). The ISCM provides feedback to a reinforcement learner to learn robust representations and to reward a more efficient exploration behavior. We perform experiments with sound enabled during pre-training and disabled during adaptation, and show that representations learned by ISCM outperform the ones by vision-only baselines and pre-trained policies can accelerate the learning process when applied to downstream tasks., Accepted at IROS 2022

Direct laser fabrication (DLF) has been widely used as an additive manufacturing technology. However, the effect of shielding gas flow rate and interaction among other parameters such as laser power, scanning speed, and powder feed rate in fabrication have not been studied. In this study, four factors were evaluated by two levels of full factorial experiment and response surface methodology (RSM). Thus, the interactions between processing parameters affecting two responses (shape factor and dilution rate) were determined, and regression models for two responses related to shielding gas flow rate were statistically evaluated for the first time. Interactive optimization was performed using contour and surface plots. Hence, the optimum combinations of four parameters were verified by metallurgical and mechanical analyses. More uniform surface profiles and finer microstructures can be obtained by adjusting the shielding gas flow rate. The results show that shielding gas flow rate and its interaction with other parameters for two responses play an important role in fabricating parts of AISI 316 L stainless steel. This fundamental study provides the significance of this parameter and better understanding of interaction between multiple processing parameters.

In this investigation, in order to improve bonding strength between superabrasive/metal matrix/grinding substrate, life span of grinding wheel, and grinding using small grits in continuous grinding simultaneously to fit for high speed and high precision machining in industry, coaxial powder feeding laser cladding method with CAD/CAM technology is introduced to manufacture textured CBN/CuSnTi–grinding wheels. The morphology of CBN grit on laser-cladding layer under optimized laser-cladding parameters and a pit created by fallen-off CBN on laser-cladding layer with lower laser–cladding energy density are observed by scanning electron microscope (SEM). The element distributions of interfaces of CBN/CuSnTi/AISI 1045 are analyzed by SEM and energy disperse spectroscope (EDS). The morphology and elements distribution of residual resultants on the surface of CBN grits etched by nitric acid are analyzed by SEM and EDS. Comparative-grinding process between laser cladding–grinding wheel (LCGW) and customized electroplated grinding wheel (EGW) is analyzed with grinding forces and temperature aspects respectively. The wear morphology of CBN grits on LCGW after grinding is observed by SEM. The results show that CBN grit with integrate cutting edges can protrude 50% height of its diameter on laser-cladding layer under optimized laser–cladding parameters. Fe, N, Ti, and B segregates attached to the interfaces of CBN/CuSnTi/AISI 1045 with Cu and Sn distributed uniformly in the laser-cladding layer. Residual resultants on CBN can be divided into two parts based on the distances from the surface of CBN grits. The grinding forces (Fz and Fy) and grinding temperature from LCGW are lower than those from EGW. The wear conditions of CBN on laser cladding are three parts: microfracture, cleavage plane, and wear-out.

Aiming at the serious problem of gap leakage between the stator and rotor of the supercritical carbon dioxide compressor, the dry gas seal scheme for centrifugal compressor of small S-CO2 power system is designed in this paper. The sealing characteristics of the single-stage spiral groove dry gas seal in the S-CO2 operating environment are studied by the large-scale commercial CFD numerical analysis software ANSYS CFX. The leakage characteristics of the S-CO2 dry gas seal at different film thicknesses are obtained. The results show that the S-CO2 dry gas seal leakage flow rate increases linearly with the increase of the film thickness. The dry gas seal leakage flow rate under different film thickness is 0.04%∼0.45% of the S-CO2 compressor flow rate, which can meet the leakage requirements of the centrifugal compressor shaft end seal in the small S-CO2 power system.

In order to study the effect of surface microarc oxidation modification on the comprehensive properties of titanium alloys in seawater environment, the metallographic structure of titanium alloy Microarc oxidation modified surface layer and the basic physical parameters such as hardness, thickness and roughness were measured, and the PTFE, filled with 15% glass fiber +5% were filled with 25% fiberglass. The PTFE of graphite and the PTFE filled with 60% tin bronze were tested for friction by friction pair materials, and the wear amount and roughness of titanium alloy and Microarc oxidation modified surface layer were test in simulated sea water as 3.5% Nacl solution, and the results showed that the surface modification of titanium alloy by Microarc oxidation could significantly improve its hardness. The surface roughness can reach 3.2μm, and the friction coefficient of different matching pairs is small, while the surface wear of microarc oxidation modification is less than that of substrate wear.

Zhao, X.; Yu, T.; Zhao, Y.; Zhang, Y.; Sun, X.; Ma, Z.; Li, M., and Zhang, T., 2018. The investigation of the morphology and failure types of pre-placed CBN on Cu-Ni-Sn-Ti bonded to steel by laser process. In: Liu, Z.L. and Mi, C. (eds.), Advances in Sustainable Port and Ocean Engineering. Journal of Coastal Research, Special Issue No. 83, pp. 486–489 . Coconut Creek (Florida), ISSN 0749-0208. Pre-placed CBN on Cu-Ni-Sn-Ti was bonded to AISI 1045 steel with various process parameters by laser in the present paper. The wettability and the oxidization condition of the CBN in laser bead were analyzed by scanning electron microscopy (SEM), energy dispersive X-ray spectroscopy (EDS) respectively. The failure types of the CBN in laser bead were observed by optical microscopy (OM). The results show that the wettability of the CBN and Cu-Ni-Sn-Ti mixed powder is pronounced because of active element Ti regardless the CBN size and laser processing parameters. Oxidization is serious after laser process even...

A new type of water mist fire extinguisher was proposed to be used in the manned spacecraft. The goal of prototype is that it can extinguish class 8B fires on the ground and its mass is less than 2.5 kg. The prototype employed only water as suppression agents. In the extinguisher, a piston separated the water from nitrogen artificially, which dispels the influence of the gravity on discharging. It possesses some advantages, such as high reliability, easy refilling of water and nitrogen, long life span, and convenient maintenance. A pressure-swirl nozzle was employed to produce water mist. There was no part to keep the pressure constant at the nozzle’s entrance; the service pressure of nozzle was determined in the range from 6 to 15 MPa. In that scope, the variation of droplet size with pressure did not have a significant impact on suppression performance, and the suppression time increment due to velocity reduction was in the acceptable level. A series of tests were conducted to verify its performance. The vibration test presented that there was no nitrogen leakage and no obvious resonance happened, which meant it possessed high reliability. The fire suppression tests indicated that it can suppress different fire types of a certain scale. The simulation results showed that the spray performance is improved in the microgravity. Conclusion can be drawn that the prototype is feasible in the microgravity and it reached the requirements of the design.

This paper compares the models of replacement first, replacement interval, replacement last and replacement overtime to decide which policy would be better before or after a specified time T in the viewpoint of cost saving. Minimal repair, which can return the unit to operation again after failure when the repair is completed, is modeled for respective policies. In order to save the total cost of repairs, preventive replacement policies are done at a planned time T and a number N of failures. It will be shown that the comparisons can be concluded according to the relative size of given T and obtained T∗ and T̃. © 2016, International Society of Science and Applied Technologies. All rights reserved. This work is supported by Ministry of Science and Technology in Taiwan (No. MOST 104-2221-E-030-010), National Natural Science Foundation of China (No. 71371097), and Qatar National Research Fund (No. PDRA1-0116-14107). Scopus

The main purpose of this paper is to propose several new maintenance policies that can be performed at suitable middle times for trivariate preventive maintenance scenarios. Firstly, characteristics and limitations for the approaches of whichever occurs first and whichever occurs last are analyzed, and their respective maintenance models, which are called first maintenance and last maintenance, are given directly. Secondly, a combined approach of first and last is employed and middle maintenance is proposed. Thirdly, the aforementioned policy is modified to be performed at discrete working cycles, using the method of overtime maintenance. Finally, a new approach of whichever occurs middle is proposed for modelings. We give the analyses of how to formulate the middle maintenances and obtain their expected maintenance cost rates, using the technique of cumulative damage models. Numerical examples are given to compare the optimal policies among first, last, and middle maintenances. Finally, a potential application of middle maintenance is provided for future studies. Scopus

In this article, the mechanical characteristics of the squeeze of the F7 fault of the Wushaoling Tunnel are analyzed. The measurements and techniques are proposed to resist the deformation. The result indicates that the method of construction to control the further squeeze deflection is appropriate.

This dissertation proposes several maintenance policies for extended cumulative damage models in reliability theory and their applications to garbage collection policies for a generational garbage collector in computer science. Using the tech-niques of cumulative processes, the expected costs per unit of time, i.e., expected cost rate models, are obtained, and optimal policies which minimize them are discussed analytically and computed numerically. An initial chapter gives introduction which is constructed by review of literatures and organization of dissertation. Extended cumulative damage models in theory and their optimizations are proposed in the following chapters: Chapter 2 proposes two basic preventive maintenance policies for a used system with an initial variable damage level. Chapter 3 considers three replacement policies that are com-bined additive with independent damages. Chapter 4 takes up three maintenance policies for an operating system which works at random times for jobs. Chapter 5 proposes a standard cumulative damage model in which the notion of “whichever occurs last” is applied, which is called maintenance last. As applications, two stochastic models based on the working schemes of a generational garbage collector are proposed in Chapter 6. In the end of dissertation, the results are summarized and future problems are given. The models proposed in Chapter 2-5 are derived from practical systems as introduced in every chapter and could be applied to them by suitable modifications and extensions. The theoretical methods proposed in Chapter 6 could provide some useful information to computer programmers to design more efficient collectors.