Search

Large Language Models (LLMs) have gained popularity in task planning for long-horizon manipulation tasks. To enhance the validity of LLM-generated plans, visual demonstrations and online videos have been widely employed to guide the planning process. However, for manipulation tasks involving subtle movements but rich contact interactions, visual perception alone may be insufficient for the LLM to fully interpret the demonstration. Additionally, visual data provides limited information on force-related parameters and conditions, which are crucial for effective execution on real robots. In this paper, we introduce an in-context learning framework that incorporates tactile and force-torque information from human demonstrations to enhance LLMs' ability to generate plans for new task scenarios. We propose a bootstrapped reasoning pipeline that sequentially integrates each modality into a comprehensive task plan. This task plan is then used as a reference for planning in new task configurations. Real-world experiments on two different sequential manipulation tasks demonstrate the effectiveness of our framework in improving LLMs' understanding of multi-modal demonstrations and enhancing the overall planning performance.

Virtual Private Cloud (VPC) is the main network abstraction technology used in public cloud systems. VPCs are composed of a set of network services that permit the definition of complex network reachability properties among internal and external cloud entities such as tenants' VMs or some generic internet nodes. Although hiding the underlying complexity through a comprehensible abstraction layer, manually enforcing particular reachability intents in VPC networks is still notably error-prone and complex. In this paper, we propose AutoNet, a new model for assisting cloud tenants in managing reachability-based policies in VPC networks. AutoNet is capable of safely generating incremental VPC configurations while satisfying some metric-based high-level intent defined by the tenants. To achieve this goal, we leverage a MaxSAT-based encoding of the network configuration combined with several optimizations to scale to topologies with thousands of nodes. Our results show that the developed system is capable of achieving a sub-second response time for production VPC deployments while still providing fine-grained control over the generated configurations.

In the field of Learning from Demonstration (LfD), Dynamical Systems (DSs) have gained significant attention due to their ability to generate real-time motions and reach predefined targets. However, the conventional convergence-centric behavior exhibited by DSs may fall short in safety-critical tasks, specifically, those requiring precise replication of demonstrated trajectories or strict adherence to constrained regions even in the presence of perturbations or human intervention. Moreover, existing DS research often assumes demonstrations solely in Euclidean space, overlooking the crucial aspect of orientation in various applications. To alleviate these shortcomings, we present an innovative approach geared toward ensuring the safe execution of learned orientation skills within constrained regions surrounding a reference trajectory. This involves learning a stable DS on SO(3), extracting time-varying conic constraints from the variability observed in expert demonstrations, and bounding the evolution of the DS with Conic Control Barrier Function (CCBF) to fulfill the constraints. We validated our approach through extensive evaluation in simulation and showcased its effectiveness for a cutting skill in the context of assisted teleoperation.

Automatic license plate recognition plays a crucial role in modern transportation systems such as for traffic monitoring and vehicle violation detection. In real-world scenarios, license plate recognition still faces many challenges and is impaired by unpredictable interference such as weather or lighting conditions. Many machine learning based ALPR solutions have been proposed to solve such challenges in recent years. However, most are not convincing, either because their results are evaluated on small or simple datasets that lack diverse surroundings, or because they require powerful hardware to achieve a reasonable frames-per-second in real-world applications. In this paper, we propose a novel segmentation-free framework for license plate recognition and introduce NP-ALPR, a diverse and challenging dataset which resembles real-world scenarios. The proposed network model consists of the latest deep learning methods and state-of-the-art ideas, and benefits from a novel network architecture. It achieves higher accuracy with lower computational requirements than previous works. We evaluate the effectiveness of the proposed method on three different datasets and show a recognition accuracy of over 99% and over 70 fps, demonstrating that our method is not only robust but also computationally efficient.