Search

First-order conic optimization solvers are sensitive to problem conditioning and typically perform poorly in the face of ill-conditioned problem data. To mitigate this, we propose an approach to preconditioning for a class of quadratic cone programs (QCPs), i.e., conic optimization problems with a quadratic objective function, wherein the objective function is strongly convex and possesses a certain structure. This approach lends itself to factorization-free, customizable, first-order conic optimization for online applications wherein the solver is called repeatedly to solve problems of the same size/structure, but with changing problem data. One of the steps in the proposed preconditioning procedure is to scale the objective function: in addition to deriving an analytical expression for the optimal objective function scaling factor, we establish the relationship between the objective function scaling factor and the primal-dual step-size ratio for a first-order method, the proportional-integral projected gradient method (PIPG), which applies to the general class of QCPs, including quadratic programs (QPs), second-order cone programs (SOCPs), and semidefinite programs (SDPs). We demonstrate the efficacy of our approach on a numerical nonconvex trajectory optimization example, using sequential conic optimization (SeCO).

As foundation AI models continue to increase in size, an important question arises - is massive scale the only path forward? This survey of about 160 papers presents a family of Small Language Models (SLMs) in the 1 to 8 billion parameter range that demonstrate smaller models can perform as well, or even outperform large models. We explore task agnostic, general purpose SLMs, task-specific SLMs and techniques to create SLMs that can guide the community to build models while balancing performance, efficiency, scalability and cost. Furthermore we define and characterize SLMs' effective sizes, representing increased capability with respect to LLMs.

This paper presents auto-tuned primal-dual successive convexification (Auto-SCvx), an algorithm designed to reliably achieve dynamically-feasible trajectory solutions for constrained hypersonic reentry optimal control problems across a large mission parameter space. In Auto-SCvx, we solve a sequence of convex subproblems until convergence to a solution of the original nonconvex problem. This method iteratively optimizes dual variables in closed-form in order to update the penalty hyperparameters used in the primal variable updates. A benefit of this method is that it is auto-tuning, and requires no hand-tuning by the user with respect to the constraint penalty weights. Several example hypersonic reentry problems are posed and solved using this method, and comparative studies are conducted against current methods. In these numerical studies, our algorithm demonstrates equal and often improved performance while not requiring hand-tuning of penalty hyperparameters., Comment: 38 pages, 27 figures; submitted to the AIAA Journal of Guidance, Control, and Dynamics (JGCD)

This paper presents an architecture for Ultrasound Beamforming using Synthetic Transmit Aperture with Low Complexity and High SNR for medical imaging. Synthetic Transmit Aperture is a novel approach in ultrasound imaging system by which frame rate and image quality is increased significantly on less data-transfer and computational requirements. The real-time beam-forming performance of Phased Array(PA) method is limited by high computation and cost. Thus STA method(data-transfer rate-8MB/frame) advances over the Phased Array Method(data-transfer rate-95MB/frame) with comparitively much higher frame rate and Signal to Noise Ratio(SNR. In this paper, we have implemented receive beamforming using Synthetic Transmit Aperture (STA) method for eight channels and have obtained the sample data for reconstruction of image. The experimental results are compared with the conventional phased array and linear array beamforming, where it can be observed that the reduction in memory requirement and high SNR, Comment: 7 Pages, 5 figures

Training a deep neural network (DNN) requires substantial computational and memory requirements. It is common to use multiple devices to train a DNN to reduce the overall training time. There are several choices to parallelize each layer in a DNN. Exhaustively searching this list to find an optimal parallelization strategy is prohibitively time consuming and impractical. The standard practice is to use data parallelism because of its simplicity. However, data parallelism is often sub-optimal, and suffers from poor performance and high memory requirement. Expert-designed strategies have been proposed on a case-by-case basis using domain specific knowledge. These expert-designed strategies do not generalize well to DNNs other than the ones for which they were designed, and are not always necessarily the best choice. In this paper, we propose an approach to automatically find efficient parallelization strategies for DNNs from their computation graphs. We present an efficient algorithm to compute these strategies within a reasonable time in practice. We evaluate the effectiveness of our approach on various DNNs. We also compare the performance of the strategies identified by our approach against data parallelism, expert-designed strategies, and the state-of-the-art approaches. Our results show that the strategies found using our approach outperform the baseline data parallelism strategy in all the cases. In addition, our strategies achieve better performance than the expert-designed strategies and the state-of-the-art approaches., Comment: Published as conference paper at IPDPS 2021

This paper introduces a generalized mean-based C^1-smooth robustness measure over discrete-time signals (D-GMSR) for signal temporal logic (STL) specifications. In conjunction with its C1-smoothness, D-GMSR is proven to be both sound and complete. Furthermore, it demonstrates favorable gradient properties and addresses locality and masking problems, which are critical for numerical optimization. The C^1-smoothness of the proposed formulations enables the implementation of robust and efficient numerical optimization algorithms to solve problems with STL specifications while preserving their theoretical guarantees. The practical utility of the proposed robustness measure is demonstrated on two real-world trajectory optimization problems: i) quadrotor flight, and ii) autonomous rocket landing. A sequential convex programming (SCP) framework, incorporating a convergence-guaranteed optimization algorithm (the prox-linear method) is used to solve inherently non-convex trajectory optimization problems with STL specifications. The implementation is available at https://github.com/UW-ACL/D-GMSR

We introduce a GPU-accelerated Monte Carlo framework for nonconvex, free-final-time trajectory optimization problems. This framework makes use of the prox-linear method, which belongs to the larger family of sequential convex programming (SCP) algorithms, in conjunction with a constraint reformulation that guarantees inter-sample constraint satisfaction. Key features of this framework are: (1) continuous-time constraint satisfaction; (2) a matrix-inverse-free solution method; (3) the use of the proportional-integral projected gradient (PIPG) method, a first-order convex optimization solver, customized to the convex subproblem at hand; and, (4) an end-to-end, library-free implementation of the algorithm. We demonstrate this GPU-based framework on the 6-DoF powered-descent guidance problem, and show that it is faster than an equivalent serial CPU implementation for Monte Carlo simulations with over 1000 runs. To the best of our knowledge, this is the first GPU-based implementation of a general-purpose nonconvex trajectory optimization solver., Comment: AIAA SciTech 2024 Forum

We propose a nonlinear model predictive control (NMPC) framework based on a direct optimal control method that ensures continuous-time constraint satisfaction and accurate evaluation of the running cost, without compromising computational efficiency. We leverage the recently proposed successive convexification framework for trajectory optimization, where: (1) the path constraints and running cost are equivalently reformulated by augmenting the system dynamics, (2) multiple shooting is used for exact discretization, and (3) a convergence-guaranteed sequential convex programming (SCP) algorithm, the prox-linear method, is used to solve the discretized receding-horizon optimal control problems. The resulting NMPC framework is computationally efficient, owing to its support for warm-starting and premature termination of SCP, and its reliance on first-order information only. We demonstrate the effectiveness of the proposed NMPC framework by means of a numerical example with reference-tracking and obstacle avoidance. The implementation is available at https://github.com/UW-ACL/nmpc-ctcs

We present successive convexification, a real-time-capable solution method for nonconvex trajectory optimization, with continuous-time constraint satisfaction and guaranteed convergence, that only requires first-order information. The proposed framework combines several key methods to solve a large class of nonlinear optimal control problems: (i) exterior penalty-based reformulation of the path constraints; (ii) generalized time-dilation; (iii) multiple-shooting discretization; (iv) $\ell_1$ exact penalization of the nonconvex constraints; and (v) the prox-linear method, a sequential convex programming (SCP) algorithm for convex-composite minimization. The reformulation of the path constraints enables continuous-time constraint satisfaction even on sparse discretization grids and obviates the need for mesh refinement heuristics. Through the prox-linear method, we guarantee convergence of the solution method to stationary points of the penalized problem and guarantee that the converged solutions that are feasible with respect to the discretized and control-parameterized optimal control problem are also Karush-Kuhn-Tucker (KKT) points. Furthermore, we highlight the specialization of this property to global minimizers of convex optimal control problems, wherein the reformulated path constraints cannot be represented by canonical cones, i.e., in the form required by existing convex optimization solvers. In addition to theoretical analysis, we demonstrate the effectiveness and real-time capability of the proposed framework with numerical examples based on popular optimal control applications: dynamic obstacle avoidance and rocket landing.

The purpose of this note is to highlight and address inaccuracies in the convergence guarantees of SCvx, a nonconvex trajectory optimization algorithm proposed by Mao et al. (arXiv:1804.06539), and make connections to relevant prior work. Specifically, we identify errors in the convergence proof within Mao et al. (arXiv:1804.06539) and reestablish the proof of convergence by employing a new method under stricter assumptions.

Narrow bit-width data formats are key to reducing the computational and storage costs of modern deep learning applications. This paper evaluates Microscaling (MX) data formats that combine a per-block scaling factor with narrow floating-point and integer types for individual elements. MX formats balance the competing needs of hardware efficiency, model accuracy, and user friction. Empirical results on over two dozen benchmarks demonstrate practicality of MX data formats as a drop-in replacement for baseline FP32 for AI inference and training with low user friction. We also show the first instance of training generative language models at sub-8-bit weights, activations, and gradients with minimal accuracy loss and no modifications to the training recipe.