Search

Traditional per-title encoding schemes aim to optimize encoding resolutions to deliver the highest perceptual quality for each representation. XPSNR is observed to correlate better with the subjective quality of VVC-coded bitstreams. Towards this realization, we predict the average XPSNR of VVC-coded bitstreams using spatiotemporal complexity features of the video and the target encoding configuration using an XGBoost-based model. Based on the predicted XPSNR scores, we introduce a Quality-A ware Dynamic Resolution Adaptation (QADRA) framework for adaptive video streaming applications, where we determine the convex-hull online. Furthermore, keeping the encoding and decoding times within an acceptable threshold is mandatory for smooth and energy-efficient streaming. Hence, QADRA determines the encoding resolution and quantization parameter (QP) for each target bitrate by maximizing XPSNR while constraining the maximum encoding and/ or decoding time below a threshold. QADRA implements a JND-based representation elimination algorithm to remove perceptually redundant representations from the bitrate ladder. QADRA is an open-source Python-based framework published under the GNU GPLv3 license. Github: https://github.com/PhoenixVideo/QADRA Online documentation: https://phoenixvideo.github.io/QADRA/, Comment: ACM MMSys '24 | Open-Source Software and Dataset. arXiv admin note: substantial text overlap with arXiv:2401.15346

Current model-based reinforcement learning (MBRL) agents struggle with long-term dependencies. This limits their ability to effectively solve tasks involving extended time gaps between actions and outcomes, or tasks demanding the recalling of distant observations to inform current actions. To improve temporal coherence, we integrate a new family of state space models (SSMs) in world models of MBRL agents to present a new method, Recall to Imagine (R2I). This integration aims to enhance both long-term memory and long-horizon credit assignment. Through a diverse set of illustrative tasks, we systematically demonstrate that R2I not only establishes a new state-of-the-art for challenging memory and credit assignment RL tasks, such as BSuite and POPGym, but also showcases superhuman performance in the complex memory domain of Memory Maze. At the same time, it upholds comparable performance in classic RL tasks, such as Atari and DMC, suggesting the generality of our method. We also show that R2I is faster than the state-of-the-art MBRL method, DreamerV3, resulting in faster wall-time convergence., Comment: Published as a conference paper at The International Conference on Learning Representations 2024

Recent works have argued that high-level semantic concepts are encoded "linearly" in the representation space of large language models. In this work, we study the origins of such linear representations. To that end, we introduce a simple latent variable model to abstract and formalize the concept dynamics of the next token prediction. We use this formalism to show that the next token prediction objective (softmax with cross-entropy) and the implicit bias of gradient descent together promote the linear representation of concepts. Experiments show that linear representations emerge when learning from data matching the latent variable model, confirming that this simple structure already suffices to yield linear representations. We additionally confirm some predictions of the theory using the LLaMA-2 large language model, giving evidence that the simplified model yields generalizable insights.

Short form content has permeated into the video creator space over the past few years, led by industry leading products such as TikTok, YouTube Shorts and Instagram Reels. YouTube in particular was previously synonymous with being the main hub for long form video content consumption. The monetization of long form videos was easier as it allowed multiple advertisement placements during the course of the video. This model also facilitated thematic brand partnerships. However, since the introduction of short form content, creators have found it more difficult to generate revenue as advertisement placements have decreased. This leads to a unique situation where people are spending more time watching shorter videos, and yet they generate less revenue for the creators. In this paper, we perform a study of 250 creators with significant audiences to see if the introduction of short form content has affected the view counts and engagement of long form content. Our findings reveal a noteworthy trend: since the advent of short-form content, there has been a significant decrease in both view counts and engagement in long-form videos on these channels.

The tentative identification of approximately ten relativistic anti-helium cosmic-ray events at AMS-02 would, if confirmed, challenge our understanding of the astrophysical synthesis of heavy anti-nuclei. We propose a novel scenario for the enhanced production of such anti-nuclei that is triggered by isolated, catastrophic injections of large quantities of energetic Standard Model (SM) anti-quarks in our galaxy by physics beyond the Standard Model (BSM). We demonstrate that SM anti-nucleosynthetic processes that occur in the resulting rapidly expanding, thermalized fireballs of SM plasma can, for a reasonable range of parameters, produce the reported tentative $\sim 2:1$ ratio of anti-helium-3 to anti-helium-4 events at AMS-02, as well as their relativistic boosts. Moreover, we show that this can be achieved without violating anti-deuterium or anti-proton flux constraints for the appropriate anti-helium fluxes. A plausible BSM paradigm for the catastrophic injections is the collision of macroscopic composite dark-matter objects carrying large net anti-baryon number. Such a scenario would require these objects to be cosmologically stable, but to destabilize upon collision, promptly releasing a fraction of their mass energy into SM anti-particles within a tiny volume. We show that, in principle, the injection rate needed to attain the necessary anti-helium fluxes and the energetic conditions required to seed the fireballs appear possible to obtain in such a paradigm. We leave open the question of constructing a BSM particle physics model to realize this, but we suggest two concrete scenarios as promising targets for further investigation., Comment: 42 pages, 10 figures, journal version

Machine learning has shown great promise in addressing several critical hardware security problems. In particular, researchers have developed novel graph neural network (GNN)-based techniques for detecting intellectual property (IP) piracy, detecting hardware Trojans (HTs), and reverse engineering circuits, to name a few. These techniques have demonstrated outstanding accuracy and have received much attention in the community. However, since these techniques are used for security applications, it is imperative to evaluate them thoroughly and ensure they are robust and do not compromise the security of integrated circuits. In this work, we propose AttackGNN, the first red-team attack on GNN-based techniques in hardware security. To this end, we devise a novel reinforcement learning (RL) agent that generates adversarial examples, i.e., circuits, against the GNN-based techniques. We overcome three challenges related to effectiveness, scalability, and generality to devise a potent RL agent. We target five GNN-based techniques for four crucial classes of problems in hardware security: IP piracy, detecting/localizing HTs, reverse engineering, and hardware obfuscation. Through our approach, we craft circuits that fool all GNNs considered in this work. For instance, to evade IP piracy detection, we generate adversarial pirated circuits that fool the GNN-based defense into classifying our crafted circuits as not pirated. For attacking HT localization GNN, our attack generates HT-infested circuits that fool the defense on all tested circuits. We obtain a similar 100% success rate against GNNs for all classes of problems., Comment: To appear in USENIX Security Symposium, 2024

Elliptic curve cryptography (ECC) is widely used in security applications such as public key cryptography (PKC) and zero-knowledge proofs (ZKP). ECC is composed of modular arithmetic, where modular multiplication takes most of the processing time. Computational complexity and memory constraints of ECC limit the performance. Therefore, hardware acceleration on ECC is an active field of research. Processing-in-memory (PIM) is a promising approach to tackle this problem. In this work, we design ModSRAM, the first 8T SRAM PIM architecture to compute large-number modular multiplication efficiently. In addition, we propose R4CSA-LUT, a new algorithm that reduces the cycles for an interleaved algorithm and eliminates carry propagation for addition based on look-up tables (LUT). ModSRAM is co-designed with R4CSA-LUT to support modular multiplication and data reuse in memory with 52% cycle reduction compared to prior works with only 32% area overhead., Comment: DAC 2024

To build intelligent machine learning systems, there are two broad approaches. One approach is to build inherently interpretable models, as endeavored by the growing field of causal representation learning. The other approach is to build highly-performant foundation models and then invest efforts into understanding how they work. In this work, we relate these two approaches and study how to learn human-interpretable concepts from data. Weaving together ideas from both fields, we formally define a notion of concepts and show that they can be provably recovered from diverse data. Experiments on synthetic data and large language models show the utility of our unified approach., Comment: 36 pages

Spiking Neural Networks (SNNs) have been recently integrated into Transformer architectures due to their potential to reduce computational demands and to improve power efficiency. Yet, the implementation of the attention mechanism using spiking signals on general-purpose computing platforms remains inefficient. In this paper, we propose a novel framework leveraging stochastic computing (SC) to effectively execute the dot-product attention for SNN-based Transformers. We demonstrate that our approach can achieve high classification accuracy ($83.53\%$) on CIFAR-10 within 10 time steps, which is comparable to the performance of a baseline artificial neural network implementation ($83.66\%$). We estimate that the proposed SC approach can lead to over $6.3\times$ reduction in computing energy and $1.7\times$ reduction in memory access costs for a digital CMOS-based ASIC design. We experimentally validate our stochastic attention block design through an FPGA implementation, which is shown to achieve $48\times$ lower latency as compared to a GPU implementation, while consuming $15\times$ less power.

Timing vulnerabilities in processors have emerged as a potent threat. As processors are the foundation of any computing system, identifying these flaws is imperative. Recently fuzzing techniques, traditionally used for detecting software vulnerabilities, have shown promising results for uncovering vulnerabilities in large-scale hardware designs, such as processors. Researchers have adapted black-box or grey-box fuzzing to detect timing vulnerabilities in processors. However, they cannot identify the locations or root causes of these timing vulnerabilities, nor do they provide coverage feedback to enable the designer's confidence in the processor's security. To address the deficiencies of the existing fuzzers, we present WhisperFuzz--the first white-box fuzzer with static analysis--aiming to detect and locate timing vulnerabilities in processors and evaluate the coverage of microarchitectural timing behaviors. WhisperFuzz uses the fundamental nature of processors' timing behaviors, microarchitectural state transitions, to localize timing vulnerabilities. WhisperFuzz automatically extracts microarchitectural state transitions from a processor design at the register-transfer level (RTL) and instruments the design to monitor the state transitions as coverage. Moreover, WhisperFuzz measures the time a design-under-test (DUT) takes to process tests, identifying any minor, abnormal variations that may hint at a timing vulnerability. WhisperFuzz detects 12 new timing vulnerabilities across advanced open-sourced RISC-V processors: BOOM, Rocket Core, and CVA6. Eight of these violate the zero latency requirements of the Zkt extension and are considered serious security vulnerabilities. Moreover, WhisperFuzz also pinpoints the locations of the new and the existing vulnerabilities., Comment: Accepted to USENIX Sec'24

Conventional per-title encoding schemes strive to optimize encoding resolutions to deliver the utmost perceptual quality for each bitrate ladder representation. Nevertheless, maintaining encoding time within an acceptable threshold is equally imperative in online streaming applications. Furthermore, modern client devices are equipped with the capability for fast deep-learning-based video super-resolution (VSR) techniques, enhancing the perceptual quality of the decoded bitstream. This suggests that opting for lower resolutions in representations during the encoding process can curtail the overall energy consumption without substantially compromising perceptual quality. In this context, this paper introduces a video super-resolution-based latency-aware optimized bitrate encoding scheme (ViSOR) designed for online adaptive streaming applications. ViSOR determines the encoding resolution for each target bitrate, ensuring the highest achievable perceptual quality after VSR within the bound of a maximum acceptable latency. Random forest-based prediction models are trained to predict the perceptual quality after VSR and the encoding time for each resolution using the spatiotemporal features extracted for each video segment. Experimental results show that ViSOR targeting fast super-resolution convolutional neural network (FSRCNN) achieves an overall average bitrate reduction of 24.65 % and 32.70 % to maintain the same PSNR and VMAF, compared to the HTTP Live Streaming (HLS) bitrate ladder encoding of 4 s segments using the x265 encoder, when the maximum acceptable latency for each representation is set as two seconds. Considering a just noticeable difference (JND) of six VMAF points, the average cumulative storage consumption and encoding energy for each segment is reduced by 79.32 % and 68.21 %, respectively, contributing towards greener streaming., Comment: 2024 Picture Coding Symposium (PCS)

Existing large language models (LLMs) for register transfer level code generation face challenges like compilation failures and suboptimal power, performance, and area (PPA) efficiency. This is due to the lack of PPA awareness in conventional transformer decoding algorithms. In response, we present an automated transformer decoding algorithm that integrates Monte Carlo tree-search for lookahead, guiding the transformer to produce compilable, functionally correct, and PPA-optimized code. Empirical evaluation with a fine-tuned language model on RTL codesets shows that our proposed technique consistently generates functionally correct code compared to prompting-only methods and effectively addresses the PPA-unawareness drawback of naive large language models. For the largest design generated by the state-of-the-art LLM (16-bit adder), our technique can achieve a 31.8% improvement in the area-delay product.

Bayesian neural networks offer better estimates of model uncertainty compared to frequentist networks. However, inference involving Bayesian models requires multiple instantiations or sampling of the network parameters, requiring significant computational resources. Compared to traditional deep learning networks, spiking neural networks (SNNs) have the potential to reduce computational area and power, thanks to their event-driven and spike-based computational framework. Most works in literature either address frequentist SNN models or non-spiking Bayesian neural networks. In this work, we demonstrate an optimization framework for developing and implementing efficient Bayesian SNNs in hardware by additionally restricting network weights to be binary-valued to further decrease power and area consumption. We demonstrate accuracies comparable to Bayesian binary networks with full-precision Bernoulli parameters, while requiring up to $25\times$ less spikes than equivalent binary SNN implementations. We show the feasibility of the design by mapping it onto Zynq-7000, a lightweight SoC, and achieve a $6.5 \times$ improvement in GOPS/DSP while utilizing up to 30 times less power compared to the state-of-the-art., Comment: Submitted and Accepted in ISCAS 2024

Traditional per-title encoding schemes aim to optimize encoding resolutions to deliver the highest perceptual quality for each representation. However, keeping the encoding time within an acceptable threshold for a smooth user experience is important to reduce the carbon footprint and energy consumption on encoding servers in video streaming applications. Toward this realization, we introduce an encoding latency-a ware dynamic resolution encoding scheme (LADRE) for adaptive video streaming applications. LADRE determines the encoding resolution for each target bitrate by utilizing a random forest-based prediction model for every video segment based on spatiotemporal features and the acceptable target latency. Experimental results show that LADRE achieves an overall average quality improvement of 0.58 dB PSNR and 0.43 dB XPSNR while maintaining the same bitrate, compared to the HTTP Live Streaming (HLS) bitrate ladder encoding of 200 s segments using the VVenC encoder, when the encoding latency for each representation is set to remain below the 200 s threshold. This is accompanied by an 84.17 % reduction in overall encoding energy consumption., Comment: 2024 Mile High Video (MHV)

In HTTP adaptive live streaming applications, video segments are encoded at a fixed set of bitrate-resolution pairs known as bitrate ladder. Live encoders use the fastest available encoding configuration, referred to as preset, to ensure the minimum possible latency in video encoding. However, an optimized preset and optimized number of CPU threads for each encoding instance may result in (i) increased quality and (ii) efficient CPU utilization while encoding. For low latency live encoders, the encoding speed is expected to be more than or equal to the video framerate. To this light, this paper introduces a Just Noticeable Difference (JND)-Aware Low latency Encoding Scheme (JALE), which uses random forest-based models to jointly determine the optimized encoder preset and thread count for each representation, based on video complexity features, the target encoding speed, the total number of available CPU threads, and the target encoder. Experimental results show that, on average, JALE yield a quality improvement of 1.32 dB PSNR and 5.38 VMAF points with the same bitrate, compared to the fastest preset encoding of the HTTP Live Streaming (HLS) bitrate ladder using x265 HEVC open-source encoder with eight CPU threads used for each representation. These enhancements are achieved while maintaining the desired encoding speed. Furthermore, on average, JALE results in an overall storage reduction of 72.70 %, a reduction in the total number of CPU threads used by 63.83 %, and a 37.87 % reduction in the overall encoding time, considering a JND of six VMAF points., Comment: 2024 Mile High Video (MHV)

This paper aims to present an innovative and cost-effective design for Acoustic Soft Tactile (AST) Skin, with the primary goal of significantly enhancing the accuracy of 2-D tactile feature estimation. The existing challenge lies in achieving precise tactile feature estimation, especially concerning contact geometry characteristics, using cost-effective solutions. We hypothesise that by harnessing acoustic energy through dedicated acoustic channels in 2 layers beneath the sensing surface and analysing amplitude modulation, we can effectively decode interactions on the sensory surface, thereby improving tactile feature estimation. Our approach involves the distinct separation of hardware components responsible for emitting and receiving acoustic signals, resulting in a modular and highly customizable skin design. Practical tests demonstrate the effectiveness of this novel design, achieving remarkable precision in estimating contact normal forces (MAE < 0.8 N), 2D contact localisation (MAE < 0.7 mm), and contact surface diameter (MAE < 0.3 mm). In conclusion, the AST skin, with its innovative design and modular architecture, successfully addresses the challenge of tactile feature estimation. The presented results showcase its ability to precisely estimate various tactile features, making it a practical and cost-effective solution for robotic applications., Comment: IEEE Robosoft conference 2024 (accepted)

Rydberg excitons (analogues of Rydberg atoms in condensed matter systems) are highly excited bound electron-hole states with large Bohr radii. The interaction between them as well as exciton coupling to light may lead to strong optical nonlinearity, with applications in sensing and quantum information processing. Here, we achieve strong effective photon-photon interactions (Kerr-like optical nonlinearity) via the Rydberg blockade phenomenon and the hybridisation of excitons and photons forming polaritons in a Cu$_2$O-filled microresonators. Under pulsed resonant excitation polariton resonance frequencies are renormalised due to the reduction of the photon-exciton coupling with increasing exciton density. Theoretical analysis shows that the Rydberg blockade plays a major role in the experimentally observed scaling of the polariton nonlinearity coefficient as $\propto n^{4.4 \pm 1.8}$ for principal quantum numbers up to n = 7. Such high principal quantum numbers studied in a polariton system for the first time are essential for realisation of high Rydberg optical nonlinearities, which paves the way towards quantum optical applications and fundamental studies of strongly-correlated photonic (polaritonic) states in a solid state system.

Nanorobotics offers an emerging frontier in biomedicine, holding the potential to revolutionize diagnostic and therapeutic applications through its unique capabilities in manipulating biological systems at the nanoscale. Following PRISMA guidelines, a comprehensive literature search was conducted using IEEE Xplore and PubMed databases, resulting in the identification and analysis of a total of 414 papers. The studies were filtered to include only those that addressed both nanorobotics and direct medical applications. Our analysis traces the technology's evolution, highlighting its growing prominence in medicine as evidenced by the increasing number of publications over time. Applications ranged from targeted drug delivery and single-cell manipulation to minimally invasive surgery and biosensing. Despite the promise, limitations such as biocompatibility, precise control, and ethical concerns were also identified. This review aims to offer a thorough overview of the state of nanorobotics in medicine, drawing attention to current challenges and opportunities, and providing directions for future research in this rapidly advancing field.