Search

We propose CHRT (Control Hidden Representation Transformation) - a controlled language generation framework that steers large language models to generate text pertaining to certain attributes (such as toxicity). CHRT gains attribute control by modifying the hidden representation of the base model through learned transformations. We employ a contrastive-learning framework to learn these transformations that can be combined to gain multi-attribute control. The effectiveness of CHRT is experimentally shown by comparing it with seven baselines over three attributes. CHRT outperforms all the baselines in the task of detoxification, positive sentiment steering, and text simplification while minimizing the loss in linguistic qualities. Further, our approach has the lowest inference latency of only 0.01 seconds more than the base model, making it the most suitable for high-performance production environments. We open-source our code and release two novel datasets to further propel controlled language generation research., Comment: Accepted at ACL 2023 as a long paper (Findings)

Intrinsic Frequency (IF) has recently been introduced as an ample signal processing method for analyzing carotid and aortic pulse pressure tracings. The IF method has also been introduced as an effective approach for the analysis of cardiovascular system dynamics. The physiological significance, convergence and accuracy of the IF algorithm has been established in prior works. In this paper, we show that the IF method could be derived by appropriate mathematical approximations from the Navier-Stokes and elasticity equations. We further introduce a fast algorithm for the IF method based on the mathematical analysis of this method. In particular, we demonstrate that the IF algorithm can be made faster, by a factor or more than 100 times, using a proper set of initial guesses based on the topology of the problem, fast analytical solution at each point iteration, and substituting the brute force algorithm with a pattern search method. Statistically, we observe that the algorithm presented in this article complies well with its brute-force counterpart. Furthermore, we will show that on a real dataset, the fast IF method can draw correlations between the extracted intrinsic frequency features and the infusion of certain drugs. In general, this paper aims at a mathematical analysis of the IF method to show its possible origins and also to present faster algorithms.