Search

An evolving solution to address hallucination and enhance accuracy in large language models (LLMs) is Retrieval-Augmented Generation (RAG), which involves augmenting LLMs with information retrieved from an external knowledge source, such as the web. This paper profiles several RAG execution pipelines and demystifies the complex interplay between their retrieval and generation phases. We demonstrate that while exact retrieval schemes are expensive, they can reduce inference time compared to approximate retrieval variants because an exact retrieval model can send a smaller but more accurate list of documents to the generative model while maintaining the same end-to-end accuracy. This observation motivates the acceleration of the exact nearest neighbor search for RAG. In this work, we design Intelligent Knowledge Store (IKS), a type-2 CXL device that implements a scale-out near-memory acceleration architecture with a novel cache-coherent interface between the host CPU and near-memory accelerators. IKS offers 13.4-27.9x faster exact nearest neighbor search over a 512GB vector database compared with executing the search on Intel Sapphire Rapids CPUs. This higher search performance translates to 1.7-26.3x lower end-to-end inference time for representative RAG applications. IKS is inherently a memory expander; its internal DRAM can be disaggregated and used for other applications running on the server to prevent DRAM, which is the most expensive component in today's servers, from being stranded.

Large language models (LLMs) have emerged due to their capability to generate high-quality content across diverse contexts. To reduce their explosively increasing demands for computing resources, a mixture of experts (MoE) has emerged. The MoE layer enables exploiting a huge number of parameters with less computation. Applying state-of-the-art continuous batching increases throughput; however, it leads to frequent DRAM access in the MoE and attention layers. We observe that conventional computing devices have limitations when processing the MoE and attention layers, which dominate the total execution time and exhibit low arithmetic intensity (Op/B). Processing MoE layers only with devices targeting low-Op/B such as processing-in-memory (PIM) architectures is challenging due to the fluctuating Op/B in the MoE layer caused by continuous batching. To address these challenges, we propose Duplex, which comprises xPU tailored for high-Op/B and Logic-PIM to effectively perform low-Op/B operation within a single device. Duplex selects the most suitable processor based on the Op/B of each layer within LLMs. As the Op/B of the MoE layer is at least 1 and that of the attention layer has a value of 4-8 for grouped query attention, prior PIM architectures are not efficient, which place processing units inside DRAM dies and only target extremely low-Op/B (under one) operations. Based on recent trends, Logic-PIM adds more through-silicon vias (TSVs) to enable high-bandwidth communication between the DRAM die and the logic die and place powerful processing units on the logic die, which is best suited for handling low-Op/B operations ranging from few to a few dozens. To maximally utilize the xPU and Logic-PIM, we propose expert and attention co-processing., Comment: 15 pages, 16 figures, accepted at MICRO 2024

The 3D Bin Packing Problem (3D-BPP) is one of the most demanded yet challenging problems in industry, where an agent must pack variable size items delivered in sequence into a finite bin with the aim to maximize the space utilization. It represents a strongly NP-Hard optimization problem such that no solution has been offered to date with high performance in space utilization. In this paper, we present a new reinforcement learning (RL) framework for a 3D-BPP solution for improving performance. First, a buffer is introduced to allow multi-item action selection. By increasing the degree of freedom in action selection, a more complex policy that results in better packing performance can be derived. Second, we propose an agnostic data augmentation strategy that exploits both bin item symmetries for improving sample efficiency. Third, we implement a model-based RL method adapted from the popular algorithm AlphaGo, which has shown superhuman performance in zero-sum games. Our adaptation is capable of working in single-player and score based environments. In spite of the fact that AlphaGo versions are known to be computationally heavy, we manage to train the proposed framework with a single thread and GPU, while obtaining a solution that outperforms the state-of-the-art results in space utilization., Comment: Accepted in IROS 2022

In this paper, we present a novel audio synthesizer, CAESynth, based on a conditional autoencoder. CAESynth synthesizes timbre in real-time by interpolating the reference sounds in their shared latent feature space, while controlling a pitch independently. We show that training a conditional autoencoder based on accuracy in timbre classification together with adversarial regularization of pitch content allows timbre distribution in latent space to be more effective and stable for timbre interpolation and pitch conditioning. The proposed method is applicable not only to creation of musical cues but also to exploration of audio affordance in mixed reality based on novel timbre mixtures with environmental sounds. We demonstrate by experiments that CAESynth achieves smooth and high-fidelity audio synthesis in real-time through timbre interpolation and independent yet accurate pitch control for musical cues as well as for audio affordance with environmental sound. A Python implementation along with some generated samples are shared online., Comment: MLSP 2021

An assembly planning system that operates based on a recursive decomposition of assembly into subassemblies, and analyzes assembly cost in terms of stability, directionality, and manipulability to guide the generation of preferred assembly plans is presented. The planning in this system incorporates the special processes, such as cleaning, testing, labeling, etc. that must occur during the assembly, and handles nonreversible as well as reversible assembly tasks through backward assembly planning. In order to increase the planning efficiency, the system avoids the analysis of decompositions that do not correspond to feasible assembly tasks. This is achieved by grouping and merging those parts that can not be decomposable at the current stage of backward assembly planning due to the requirement of special processes and the constraint of interconnection feasibility. The invention includes methods of evaluating assembly cost in terms of the number of fixtures (or holding devices) and reorientations required for assembly, through the analysis of stability, directionality, and manipulability. All these factors are used in defining cost and heuristic functions for an AO* search for an optimal plan.

Optoelectronic proximity sensor comprises seven identical units functioning together to provide partially redundant data on spatial relationship between sensor and nearby diffusely reflecting surface. Provides data on distance, relative orientation, and curvature of surface. Sensor called "Hexeye" because its units exhibit hexagonal symmetry when viewed along optical axes and mounted together in hexagonal close packing. Because all six arrays of photodetectors in unit measure essentially identical distributions of light in operating mode, each provides redundant information averaged to increase accuracy or used to verify operation of others. Malfunctioning photodector identified, and output ignored or replaced by interpolated value. Developed for use in such robotic applications as noncontact probing or scanning of surfaces, localization and recognition of objects, and guiding "smart" robot hands in grasping objects.

Proposed teleoperator control system incorporates several advanced features to improve coordination between remote robotic manipulator (slave arm) and human operator at control manipulator (master arm). Provides superior performance in terms of error profiles and task-completion time in presence of human control errors varying over wide range and maintains stability under large signal-propagation delays. System highly robust in presence of nonlinearities and incorrect human commands in both position and rate modes of control.

Proposed two-arm robotic manipulator capable of changing its mechanical structure to fit given task. No reassembly or replacement in conventional sense: instead, arms commanded during operation to assume any of number of alternative configurations, described briefly in "Dynamic Coordination of a Two-Arm Robotic Manipulator" (NPO-18816), NASA Tech Briefs.

Report presents study of dynamical and kinematical considerations guiding selection of configuration of self-reconfigurable, two-arm robotic manipulator. Two multiple-link arms cooperate in manipulating single object, reconfiguring their mutual, cooperative structure according to changing task requirements.

System of algorithms developed to automate planning of sequences of manufacturing or field operations in which components are put together into assemblies. Results of DFA analysis also used to modify designs to optimize them with respect to assembly considerations.

A theoretical framework is presented for a 'sensing-knowledge command-fusion' paradigm of interactive and cooperative sensing and control in advanced teleoperators, which takes advantage of both current and projected robotic dexterousness and sensor-based autonomy capabilities. Attention is given to (1) a method for the achievement of a sensing-knowledge-command computational mechanism that implements the intended cooperative/interactive system, and (2) the system architecture and man/machine-interface protocols entailed by this implementation.

Teleoperation experiments at JPL have shown that advanced features in a telerobotic system are a necessary condition for good results, but that they are not sufficient to assure consistently good performance by the operators. Two or three operators are normally used during training and experiments to maintain the desired performance. An alternative to this multi-operator control station is a man-machine interface embedding computer programs that can perform some of the operator's functions. In this paper we present our first experiments with these concepts, in which we focused on the areas of real-time task monitoring and interactive path planning. In the first case, when performing a known task, the operator has an automatic aid for setting control parameters and camera views. In the second case, an interactive path planner will rank different path alternatives so that the operator will make the correct control decision. The monitoring function has been implemented with a neural network doing the real-time task segmentation. The interactive path planner was implemented for redundant manipulators to specify arm configurations across the desired path and satisfy geometric, task, and performance constraints.

This paper presents the paradigm of interactive and cooperative sensing and control as a fundamental mechanism of integrating and fusing the strengths of man and machine for advanced teleoperation. The interactive and cooperative sensing and control is considered as an extended and generalized form of traded and shared control. The emphasis of interactive and cooperative sensing and control is given to the distribution of mutually nonexclusive subtasks to man and machine, the interactive invocation of subtasks under the man/machine symbiotic relationship, and the fusion of information and decisionmaking between man and machine according to their confidence measures. The proposed interactive and cooperative sensing and control system is composed of such major functional blocks as the logical sensor system, the sensor-based local autonomy, the virtual environment formation, and the cooperative decision-making between man and machine. The Sensing-Knowledge-Command (SKC) fusion network is proposed as a fundamental architecture for implementing cooperative and interactive sensing and control. Simulation results are shown.

An assembly planning system that operates based on a recursive decomposition of assembly into subassemblies is presented. The planning system analyzes assembly cost in terms of stability, directionality, and manipulability to guide the generation of preferred assembly plans. The planning in this system incorporates the special processes, such as cleaning, testing, labeling, etc., that must occur during the assembly. Additionally, the planning handles nonreversible, as well as reversible, assembly tasks through backward assembly planning. In order to decrease the planning efficiency, the system avoids the analysis of decompositions that do not correspond to feasible assembly tasks. This is achieved by grouping and merging those parts that can not be decomposable at the current stage of backward assembly planning due to the requirement of special processes and the constraint of interconnection feasibility. The invention includes methods of evaluating assembly cost in terms of the number of fixtures (or holding devices) and reorientations required for assembly, through the analysis of stability, directionality, and manipulability. All these factors are used in defining cost and heuristic functions for an AO* search for an optimal plan.