Search

Deciding if there exists a path from one vertex to another in a graph is known as the s-t connectivity or the reachability problem. Reachability can be solved using graph traversal algorithms like Depth First Search(DFS) or Breadth First Search(BFS) in linear time but these algorithms also take linear space. On the other hand, Savitch's algorithm solves the same problem using O(log^2 n) space but takes quasipolynomial time. A path decomposition P of a directed graph G is a collection of simple directed paths such that every edge of G lies on exactly one path in P. A minimal path decomposition of G is a path decomposition of G having the smallest number of paths possible and the number of paths in a minimal path decomposition of G is called the path number of G. We show that if a path decomposition P of a directed graph G consisting of k directed paths is provided, then reachability in G can be decided simultaneously in O(klog n) space and polynomial time. In fact, our result holds even when a walk decomposition is provided (instead of a path decomposition) where the graph is decomposed into k directed walks (instead of paths) and the walks are not necessarily edge-disjoint. We further show that a minimal path decomposition can be computed in logspace for directed acyclic graphs. This leads to the conclusion that reachability in directed acyclic graphs having bounded path number is logspace computable.

Radiance fields are powerful and, hence, popular models for representing the appearance of complex scenes. Yet, constructing them based on image observations gives rise to ambiguities and uncertainties. We propose a versatile approach for learning Gaussian radiance fields with explicit and fine-grained uncertainty estimates that impose only little additional cost compared to uncertainty-agnostic training. Our key observation is that uncertainties can be modeled as a low-dimensional manifold in the space of radiance field parameters that is highly amenable to Monte Carlo sampling. Importantly, our uncertainties are differentiable and, thus, allow for gradient-based optimization of subsequent captures that optimally reduce ambiguities. We demonstrate state-of-the-art performance on next-best-view planning tasks, including high-dimensional illumination planning for optimal radiance field relighting quality., Comment: Siggraph Asia 2024 conference

We investigate the effects of attractive Hubbard interaction on two-component fermionic atoms in narrow two-dimensional (2D) energy bands that exhibit Rashba spin-orbit coupling (SOC) in the presence of an applied Zeeman field. This narrow-band 2D spin-orbit coupled attractive Fermi-Hubbard model can potentially be realized in cold atom systems in optical lattices with artificially engineered Rashba SOC and Zeeman field. Employing a self-consistent mean field approximation for the pairing potential, we uncover a complex phase diagram featuring various topological superfluid (TS) phases, dependent on the chemical potential and the Zeeman field. We focus on the pairing potential and the corresponding quasiparticle gap characterizing the TS phases, which are notably small for a wide-band model with quadratic dispersion near the $\Gamma$-point, as found in earlier work, and we identify the parameter regimes that maximize the gap. We find that, while generally the value of the pairing potential increases with the reduction of the fermionic bandwidth, as expected for narrow- or flat-band systems, the magnitude of the topological gap characterizing the TS phases reaches a maximum of about $10-12.5\%$ of the interaction strength at finite values of the hopping amplitude, Rashba coupling, and Zeeman field., Comment: 10 pages, 10 figures

A catalytic Turing machine is a variant of a Turing machine in which there exists an auxiliary tape in addition to the input tape and the work tape. This auxiliary tape is initially filled with arbitrary content. The machine can read and write on the auxiliary tape, but it is constrained to restore its initial content when it halts. Studying such a model and finding its powers and limitations has practical applications. In this paper, we study catalytic Turing machines with O(log n)-sized work tape and polynomial-sized auxiliary tape that are allowed to lose at most constant many bits of the auxiliary tape when they halt. We show that such catalytic Turing machines can only decide the same set of languages as standard catalytic Turing machines with the same size work and auxiliary tape.

We investigate the problem of learning operators between function spaces, focusing on the linear layer of the Fourier Neural Operator. First, we identify three main errors that occur during the learning process: statistical error due to finite sample size, truncation error from finite rank approximation of the operator, and discretization error from handling functional data on a finite grid of domain points. Finally, we analyze a Discrete Fourier Transform (DFT) based least squares estimator, establishing both upper and lower bounds on the aforementioned errors., Comment: 30 pages

This paper focuses on studying the configuration spaces of graphs realised in $\mathbb C^2$, such that the configuration space is, after normalisation, one dimensional. If this is the case, then the configuration space is, generically, a smooth complex curve, and can be seen as a Riemann surface. The property of interest in this paper is the genus of this curve. Using tropical geometry, we give an algorithm to compute this genus. We provide an implementation in Python and give various examples.

Neural rendering methods can achieve near-photorealistic image synthesis of scenes from posed input images. However, when the images are imperfect, e.g., captured in very low-light conditions, state-of-the-art methods fail to reconstruct high-quality 3D scenes. Recent approaches have tried to address this limitation by modeling various degradation processes in the image formation model; however, this limits them to specific image degradations. In this paper, we propose a generalizable neural rendering method that can perform high-fidelity novel view synthesis under several degradations. Our method, GAURA, is learning-based and does not require any test-time scene-specific optimization. It is trained on a synthetic dataset that includes several degradation types. GAURA outperforms state-of-the-art methods on several benchmarks for low-light enhancement, dehazing, deraining, and on-par for motion deblurring. Further, our model can be efficiently fine-tuned to any new incoming degradation using minimal data. We thus demonstrate adaptation results on two unseen degradations, desnowing and removing defocus blur. Code and video results are available at vinayak-vg.github.io/GAURA., Comment: European Conference on Computer Vision(ECCV) 2024

We give an elementary approach utilizing only the divided difference formalism for obtaining expansions of Schubert polynomials that are manifestly nonnegative, by studying solutions to the equation $\sum Y_i\partial_i=\mathrm{id}$ on polynomials with no constant term. This in particular recovers the pipe dream and slide polynomial expansions. We also show that slide polynomials satisfy an analogue of the divided difference formalisms for Schubert polynomials and forest polynomials, which gives a simple method for extracting the coefficients of slide polynomials in the slide polynomial decomposition of an arbitrary polynomial., Comment: 29 pages, 7 figures

In decentralized multiagent trajectory planners, agents need to communicate and exchange their positions to generate collision-free trajectories. However, due to localization errors/uncertainties, trajectory deconfliction can fail even if trajectories are perfectly shared between agents. To address this issue, we first present PARM and PARM*, perception-aware, decentralized, asynchronous multiagent trajectory planners that enable a team of agents to navigate uncertain environments while deconflicting trajectories and avoiding obstacles using perception information. PARM* differs from PARM as it is less conservative, using more computation to find closer-to-optimal solutions. While these methods achieve state-of-the-art performance, they suffer from high computational costs as they need to solve large optimization problems onboard, making it difficult for agents to replan at high rates. To overcome this challenge, we present our second key contribution, PRIMER, a learning-based planner trained with imitation learning (IL) using PARM* as the expert demonstrator. PRIMER leverages the low computational requirements at deployment of neural networks and achieves a computation speed up to 5500 times faster than optimization-based approaches., Comment: 6 pages, 3 figures

Software failures inform engineering work, standards, regulations. For example, the Log4J vulnerability brought government and industry attention to evaluating and securing software supply chains. Accessing private engineering records is difficult, so failure analyses tend to use information reported by the news media. However, prior works in this direction have relied on manual analysis. That has limited the scale of their analyses. The community lacks automated support to enable such analyses to consider a wide range of news sources and incidents. In this paper, we propose the Failure Analysis Investigation with LLMs (FAIL) system to fill this gap. FAIL collects, analyzes, and summarizes software failures as reported in the news. FAIL groups articles that describe the same incidents. It then analyzes incidents using existing taxonomies for postmortems, faults, and system characteristics. To tune and evaluate FAIL, we followed the methods of prior works by manually analyzing 31 software failures. FAIL achieved an F1 score of 90% for collecting news about software failures, a V-measure of 0.98 for merging articles reporting on the same incident, and extracted 90% of the facts about failures. We then applied FAIL to a total of 137,427 news articles from 11 providers published between 2010 and 2022. FAIL identified and analyzed 2457 distinct failures reported across 4,184 articles. Our findings include: (1) current generation of large language models are capable of identifying news articles that describe failures, and analyzing them according to structured taxonomies; (2) high recurrences of similar failures within organizations and across organizations; and (3) severity of the consequences of software failures have increased over the past decade. The full FAIL database is available so that researchers, engineers, and policymakers can learn from a diversity of software failures., Comment: Accapted at the 9th IEEE/ACM International Conference on Automated Software Engineering (ASE 2024)

We develop a quasisymmetric analogue of the combinatorial theory of Schubert polynomials and the associated divided difference operators. Our counterparts are "forest polynomials", and a new family of linear operators, whose theory of compositions is governed by forests and the "Thompson monoid". Our approach extends naturally to $m$-colored quasisymmetric functions. We then give several applications of our theory to fundamental quasisymmetric functions, the study of quasisymmetric coinvariant rings and their associated harmonics, and positivity results for various expansions. In particular we resolve a conjecture of Aval-Bergeron-Li regarding quasisymmetric harmonics., Comment: 54 pages, 17 figures

Sequential decision-making domains such as recommender systems, healthcare and education often have unobserved heterogeneity in the population that can be modeled using latent bandits $-$ a framework where an unobserved latent state determines the model for a trajectory. While the latent bandit framework is compelling, the extent of its generality is unclear. We first address this by establishing a de Finetti theorem for decision processes, and show that $\textit{every}$ exchangeable and coherent stateless decision process is a latent bandit. The latent bandit framework lends itself particularly well to online learning with offline datasets, a problem of growing interest in sequential decision-making. One can leverage offline latent bandit data to learn a complex model for each latent state, so that an agent can simply learn the latent state online to act optimally. We focus on a linear model for a latent bandit with $d_A$-dimensional actions, where the latent states lie in an unknown $d_K$-dimensional subspace for $d_K \ll d_A$. We present SOLD, a novel principled method to learn this subspace from short offline trajectories with guarantees. We then provide two methods to leverage this subspace online: LOCAL-UCB and ProBALL-UCB. We demonstrate that LOCAL-UCB enjoys $\tilde O(\min(d_A\sqrt{T}, d_K\sqrt{T}(1+\sqrt{d_AT/d_KN})))$ regret guarantees, where the effective dimension is lower when the size $N$ of the offline dataset is larger. ProBALL-UCB enjoys a slightly weaker guarantee, but is more practical and computationally efficient. Finally, we establish the efficacy of our methods using experiments on both synthetic data and real-life movie recommendation data from MovieLens., Comment: 40 pages. 14 pages for main paper, 26 pages for references + appendix

End-to-end engineering design pipelines, in which designs are evaluated using concurrently defined optimal controllers, are becoming increasingly common in practice. To discover designs that perform well even under the misspecification of system dynamics, such end-to-end pipelines have now begun evaluating designs with a robust control objective in place of the nominal optimal control setup. Current approaches of specifying such robust control subproblems, however, rely on hand specification of perturbations anticipated to be present upon deployment or margin methods that ignore problem structure, resulting in a lack of theoretical guarantees and overly conservative empirical performance. We, instead, propose a novel methodology for LQR systems that leverages conformal prediction to specify such uncertainty regions in a data-driven fashion. Such regions have distribution-free coverage guarantees on the true system dynamics, in turn allowing for a probabilistic characterization of the regret of the resulting robust controller. We then demonstrate that such a controller can be efficiently produced via a novel policy gradient method that has convergence guarantees. We finally demonstrate the superior empirical performance of our method over alternate robust control specifications in a collection of engineering control systems, specifically for airfoils and a load-positioning system.

Uncertainty quantification for multi-view learning is motivated by the increasing use of multi-view data in scientific problems. A common variant of multi-view learning is late fusion: train separate predictors on individual views and combine them after single-view predictions are available. Existing methods for uncertainty quantification for late fusion often rely on undesirable distributional assumptions for validity. Conformal prediction is one approach that avoids such distributional assumptions. However, naively applying conformal prediction to late-stage fusion pipelines often produces overly conservative and uninformative prediction regions, limiting its downstream utility. We propose a novel methodology, Multi-View Conformal Prediction (MVCP), where conformal prediction is instead performed separately on the single-view predictors and only fused subsequently. Our framework extends the standard scalar formulation of a score function to a multivariate score that produces more efficient downstream prediction regions in both classification and regression settings. We then demonstrate that such improvements can be realized in methods built atop conformalized regressors, specifically in robust predict-then-optimize pipelines.

We study online classification under smoothed adversaries. In this setting, at each time point, the adversary draws an example from a distribution that has a bounded density with respect to a fixed base measure, which is known apriori to the learner. For binary classification and scalar-valued regression, previous works \citep{haghtalab2020smoothed, block2022smoothed} have shown that smoothed online learning is as easy as learning in the iid batch setting under PAC model. However, we show that smoothed online classification can be harder than the iid batch classification when the label space is unbounded. In particular, we construct a hypothesis class that is learnable in the iid batch setting under the PAC model but is not learnable under the smoothed online model. Finally, we identify a condition that ensures that the PAC learnability of a hypothesis class is sufficient for its smoothed online learnability., Comment: 18 pages

We resolve the open problem of designing a computationally efficient algorithm for infinite-horizon average-reward linear Markov Decision Processes (MDPs) with $\widetilde{O}(\sqrt{T})$ regret. Previous approaches with $\widetilde{O}(\sqrt{T})$ regret either suffer from computational inefficiency or require strong assumptions on dynamics, such as ergodicity. In this paper, we approximate the average-reward setting by the discounted setting and show that running an optimistic value iteration-based algorithm for learning the discounted setting achieves $\widetilde{O}(\sqrt{T})$ regret when the discounting factor $\gamma$ is tuned appropriately. The challenge in the approximation approach is to get a regret bound with a sharp dependency on the effective horizon $1 / (1 - \gamma)$. We use a computationally efficient clipping operator that constrains the span of the optimistic state value function estimate to achieve a sharp regret bound in terms of the effective horizon, which leads to $\widetilde{O}(\sqrt{T})$ regret.

We study online classification when the learner has access to predictions about future examples. We design an online learner whose expected regret is never worse than the worst-case regret, gracefully improves with the quality of the predictions, and can be significantly better than the worst-case regret when the predictions of future examples are accurate. As a corollary, we show that if the learner is always guaranteed to observe data where future examples are easily predictable, then online learning can be as easy as transductive online learning. Our results complement recent work in online algorithms with predictions and smoothed online classification, which go beyond a worse-case analysis by using machine-learned predictions and distributional assumptions respectively., Comment: 24 pages

Traditional optimization-based planners, while effective, suffer from high computational costs, resulting in slow trajectory generation. A successful strategy to reduce computation time involves using Imitation Learning (IL) to develop fast neural network (NN) policies from those planners, which are treated as expert demonstrators. Although the resulting NN policies are effective at quickly generating trajectories similar to those from the expert, (1) their output does not explicitly account for dynamic feasibility, and (2) the policies do not accommodate changes in the constraints different from those used during training. To overcome these limitations, we propose Constraint-Guided Diffusion (CGD), a novel IL-based approach to trajectory planning. CGD leverages a hybrid learning/online optimization scheme that combines diffusion policies with a surrogate efficient optimization problem, enabling the generation of collision-free, dynamically feasible trajectories. The key ideas of CGD include dividing the original challenging optimization problem solved by the expert into two more manageable sub-problems: (a) efficiently finding collision-free paths, and (b) determining a dynamically-feasible time-parametrization for those paths to obtain a trajectory. Compared to conventional neural network architectures, we demonstrate through numerical evaluations significant improvements in performance and dynamic feasibility under scenarios with new constraints never encountered during training., Comment: 8 pages, 3 figures

As large language models (LLMs) see increasing adoption across the globe, it is imperative for LLMs to be representative of the linguistic diversity of the world. India is a linguistically diverse country of 1.4 Billion people. To facilitate research on multilingual LLM evaluation, we release IndicGenBench - the largest benchmark for evaluating LLMs on user-facing generation tasks across a diverse set 29 of Indic languages covering 13 scripts and 4 language families. IndicGenBench is composed of diverse generation tasks like cross-lingual summarization, machine translation, and cross-lingual question answering. IndicGenBench extends existing benchmarks to many Indic languages through human curation providing multi-way parallel evaluation data for many under-represented Indic languages for the first time. We evaluate a wide range of proprietary and open-source LLMs including GPT-3.5, GPT-4, PaLM-2, mT5, Gemma, BLOOM and LLaMA on IndicGenBench in a variety of settings. The largest PaLM-2 models performs the best on most tasks, however, there is a significant performance gap in all languages compared to English showing that further research is needed for the development of more inclusive multilingual language models. IndicGenBench is released at www.github.com/google-research-datasets/indic-gen-bench, Comment: ACL 2024

This paper introduces FlowMap, an end-to-end differentiable method that solves for precise camera poses, camera intrinsics, and per-frame dense depth of a video sequence. Our method performs per-video gradient-descent minimization of a simple least-squares objective that compares the optical flow induced by depth, intrinsics, and poses against correspondences obtained via off-the-shelf optical flow and point tracking. Alongside the use of point tracks to encourage long-term geometric consistency, we introduce differentiable re-parameterizations of depth, intrinsics, and pose that are amenable to first-order optimization. We empirically show that camera parameters and dense depth recovered by our method enable photo-realistic novel view synthesis on 360-degree trajectories using Gaussian Splatting. Our method not only far outperforms prior gradient-descent based bundle adjustment methods, but surprisingly performs on par with COLMAP, the state-of-the-art SfM method, on the downstream task of 360-degree novel view synthesis (even though our method is purely gradient-descent based, fully differentiable, and presents a complete departure from conventional SfM)., Comment: Project website: https://cameronosmith.github.io/flowmap/

A contract is a type of legal document commonly used in organizations. Contract review is an integral and repetitive process to avoid business risk and liability. Contract analysis requires the identification and classification of key provisions and paragraphs within an agreement. Identification and validation of contract clauses can be a time-consuming and challenging task demanding the services of trained and expensive lawyers, paralegals or other legal assistants. Classification of legal provisions in contracts using artificial intelligence and natural language processing is complex due to the requirement of domain-specialized legal language for model training and the scarcity of sufficient labeled data in the legal domain. Using general-purpose models is not effective in this context due to the use of specialized legal vocabulary in contracts which may not be recognized by a general model. To address this problem, we propose the use of a pre-trained large language model which is subsequently calibrated on legal taxonomy. We propose LegalPro-BERT, a BERT transformer architecture model that we fine-tune to efficiently handle classification task for legal provisions. We conducted experiments to measure and compare metrics with current benchmark results. We found that LegalPro-BERT outperforms the previous benchmark used for comparison in this research., Comment: 17 pages, 4 figures

Successful supply chain optimization must mitigate imbalances between supply and demand over time. While accurate demand prediction is essential for supply planning, it alone does not suffice. The key to successful supply planning for optimal and viable execution lies in maximizing predictability for both demand and supply throughout an execution horizon. Therefore, enhancing the accuracy of supply predictions is imperative to create an attainable supply plan that matches demand without overstocking or understocking. However, in complex supply chain networks with numerous nodes and edges, accurate supply predictions are challenging due to dynamic node interactions, cascading supply delays, resource availability, production and logistic capabilities. Consequently, supply executions often deviate from their initial plans. To address this, we present the Graph-based Supply Prediction (GSP) probabilistic model. Our attention-based graph neural network (GNN) model predicts supplies, inventory, and imbalances using graph-structured historical data, demand forecasting, and original supply plan inputs. The experiments, conducted using historical data from a global consumer goods company's large-scale supply chain, demonstrate that GSP significantly improves supply and inventory prediction accuracy, potentially offering supply plan corrections to optimize executions.

We associate a quotient of superspace to any hyperplane arrangement by considering the differential closure of an ideal generated by powers of certain homogeneous linear forms. This quotient is a superspace analogue of the external zonotopal algebra, and it further contains the central zonotopal algebra in the appropriate grading. We show that an evaluation of the bivariate Tutte polynomial is the bigraded Hilbert series of this quotient. We then use this fact to construct an explicit basis for the Macaulay inverse. These results generalize those of Ardila-Postnikov and Holtz-Ron. We also discuss enumerative consequences of our results in the setting of hyperplane arrangements.

This work introduces Extent of Violation Indices (EVIs), a novel metric for quantifying how well exchange-correlation functionals adhere to local conditions. Applying EVIs to a diverse set of molecules for GGA functionals reveals widespread violations, particularly for semi-empirical functionals. We leverage EVIs to explore potential connections between these violations and errors in chemical properties. While no correlation is observed for atomization energies, a link emerges between EVIs and total energies. Similarly, the analysis of reaction energies suggests weak positive correlations for specific conditions, but definitive conclusions about error cancellation require advancements in both functional accuracy and our understanding of cancellation mechanisms. Overall, this study highlights EVIs as a powerful tool for analyzing functional behavior and adherence to local conditions, paving the way for future research to fully elucidate the impact of violations on energy errors.

Multitask Reinforcement Learning (MTRL) approaches have gained increasing attention for its wide applications in many important Reinforcement Learning (RL) tasks. However, while recent advancements in MTRL theory have focused on the improved statistical efficiency by assuming a shared structure across tasks, exploration--a crucial aspect of RL--has been largely overlooked. This paper addresses this gap by showing that when an agent is trained on a sufficiently diverse set of tasks, a generic policy-sharing algorithm with myopic exploration design like $\epsilon$-greedy that are inefficient in general can be sample-efficient for MTRL. To the best of our knowledge, this is the first theoretical demonstration of the "exploration benefits" of MTRL. It may also shed light on the enigmatic success of the wide applications of myopic exploration in practice. To validate the role of diversity, we conduct experiments on synthetic robotic control environments, where the diverse task set aligns with the task selection by automatic curriculum learning, which is empirically shown to improve sample-efficiency.

We investigate theoretically a one-dimensional semiconductor-superconductor (SM-SC) heterostructure with Rashba spin-orbit coupling and parallel Zeeman field in the presence of disorder generated by random charged impurities and identify the optimal regimes for realizing topological superconductivity and Majorana zero modes. Using a Green's function approach, we show that upon increasing the disorder strength the stable topological superconducting phase characterized by robust end-to-end Majorana correlations "migrates" toward larger values of the Zeeman field and can be stabilized by increasing the effective SM-SC coupling. Based on these findings, we propose a strategy for accessing a regime characterized by well-separated Majorana zero modes that is based on (a) enhancing the strength of the effective SM-SC coupling (e.g., through interface engineering) and (b) expanding the range of accessible Zeeman fields (e.g., by enhancing the gyromagnetic ratio or optimizing the parent superconductor, to enable the application of larger magnetic fields). While this strategy may still require some reduction of the disorder strength, this requirement is significantly less strict than the corresponding requirement in a strategy that focuses exclusively on disorder reduction.

We study a novel pure exploration problem: the $\epsilon$-Thresholding Bandit Problem (TBP) with fixed confidence in stochastic linear bandits. We prove a lower bound for the sample complexity and extend an algorithm designed for Best Arm Identification in the linear case to TBP that is asymptotically optimal., Comment: arXiv admin note: substantial text overlap with arXiv:2006.16073 by other authors

We study the problem of learning to predict the next state of a dynamical system when the underlying evolution function is unknown. Unlike previous work, we place no parametric assumptions on the dynamical system, and study the problem from a learning theory perspective. We define new combinatorial measures and dimensions and show that they quantify the optimal mistake and regret bounds in the realizable and agnostic setting respectively., Comment: 35 pages

We focus on checking the validity of the half-plane property on two prominent classes of transversal matroids, namely lattice path matroids and bicircular matroids. We show that lattice path matroids satisfy the half-plane property. Subsequently, we show an explicit example of a bicircular matroid that is not a positroid and discuss the negative correlation properties of bases of transversal matroids. We prove that sparse paving matroids do not satisfy the Rayleigh property, which helps us gain new perspectives about conjectures on negative correlation in basis elements of matroids in general., Comment: Comments are welcome!

Craters are one of the most studied planetary features used for different scientific analyses, such as estimation of surface age and surface processes. Satellite images utilized for crater detection often have low resolution (LR) due to hardware constraints and transmission time. Super-resolution (SR) is a practical and cost-effective solution; however, most SR approaches work on fixed integer scale factors, i.e., a single model can generate images of a specific resolution. In practical applications, SR on multiple scales provides various levels of detail, but training for each scale is resource-intensive. Therefore, this paper proposes a system for crater detection assisted with an arbitrary scale super-resolution approach (i.e., a single model can be used for multiple scale factors) for the lunar surface. Our work is composed of two subsystems. The first sub-system employs an arbitrary scale SR approach to generate super-resolved images of multiple resolutions. Subsequently, the second sub-system passes super-resolved images of multiple resolutions to a deep learning-based crater detection framework for identifying craters on the lunar surface. Employed arbitrary scale SR approach is based on a combination of convolution and transformer modules. For the crater detection sub-system, we utilize the Mask-RCNN framework. Using SR images of multiple resolutions, the proposed system detects 13.47% more craters from the ground truth than the craters detected using only the LR images. Further, in complex crater settings, specifically in overlapping and degraded craters, 11.84% and 15.01% more craters are detected as compared to the crater detection networks using only the LR images. The proposed system also leads to better localization performance, 3.19% IoU increment compared to the LR images, Comment: 15 pages, 8 figures, 8 Tables

We study offline reinforcement learning (RL) with linear MDPs under the infinite-horizon discounted setting which aims to learn a policy that maximizes the expected discounted cumulative reward using a pre-collected dataset. Existing algorithms for this setting either require a uniform data coverage assumptions or are computationally inefficient for finding an $\epsilon$-optimal policy with $O(\epsilon^{-2})$ sample complexity. In this paper, we propose a primal dual algorithm for offline RL with linear MDPs in the infinite-horizon discounted setting. Our algorithm is the first computationally efficient algorithm in this setting that achieves sample complexity of $O(\epsilon^{-2})$ with partial data coverage assumption. Our work is an improvement upon a recent work that requires $O(\epsilon^{-4})$ samples. Moreover, we extend our algorithm to work in the offline constrained RL setting that enforces constraints on additional reward signals.

The growing deployment of reinforcement learning from human feedback (RLHF) calls for a deeper theoretical investigation of its underlying models. The prevalent models of RLHF do not account for neuroscience-backed, partially-observed "internal states" that can affect human feedback, nor do they accommodate intermediate feedback during an interaction. Both of these can be instrumental in speeding up learning and improving alignment. To address these limitations, we model RLHF as reinforcement learning with partially observed reward-states (PORRL). We accommodate two kinds of feedback $-$ cardinal and dueling feedback. We first demonstrate that PORRL subsumes a wide class of RL problems, including traditional RL, RLHF, and reward machines. For cardinal feedback, we present two model-based methods (POR-UCRL, POR-UCBVI). We give both cardinal regret and sample complexity guarantees for the methods, showing that they improve over naive history-summarization. We then discuss the benefits of a model-free method like GOLF with naive history-summarization in settings with recursive internal states and dense intermediate feedback. For this purpose, we define a new history aware version of the Bellman-eluder dimension and give a new guarantee for GOLF in our setting, which can be exponentially sharper in illustrative examples. For dueling feedback, we show that a naive reduction to cardinal feedback fails to achieve sublinear dueling regret. We then present the first explicit reduction that converts guarantees for cardinal regret to dueling regret. In both feedback settings, we show that our models and guarantees generalize and extend existing ones., Comment: 58 pages. 12 pages formain paper, 46 pages for references + appendix