Search

The increasing use of Retrieval-Augmented Generation (RAG) systems in various applications necessitates stringent protocols to ensure RAG systems accuracy, safety, and alignment with user intentions. In this paper, we introduce VERA (Validation and Evaluation of Retrieval-Augmented Systems), a framework designed to enhance the transparency and reliability of outputs from large language models (LLMs) that utilize retrieved information. VERA improves the way we evaluate RAG systems in two important ways: (1) it introduces a cross-encoder based mechanism that encompasses a set of multidimensional metrics into a single comprehensive ranking score, addressing the challenge of prioritizing individual metrics, and (2) it employs Bootstrap statistics on LLM-based metrics across the document repository to establish confidence bounds, ensuring the repositorys topical coverage and improving the overall reliability of retrieval systems. Through several use cases, we demonstrate how VERA can strengthen decision-making processes and trust in AI applications. Our findings not only contribute to the theoretical understanding of LLM-based RAG evaluation metric but also promote the practical implementation of responsible AI systems, marking a significant advancement in the development of reliable and transparent generative AI technologies., Comment: Accepted in Workshop on Evaluation and Trustworthiness of Generative AI Models, KDD 2024

Retrieval Augmented Generation (RAG) is a technique used to augment Large Language Models (LLMs) with contextually relevant, time-critical, or domain-specific information without altering the underlying model parameters. However, constructing RAG systems that can effectively synthesize information from large and diverse set of documents remains a significant challenge. We introduce a novel data-centric RAG workflow for LLMs, transforming the traditional retrieve-then-read system into a more advanced prepare-then-rewrite-then-retrieve-then-read framework, to achieve higher domain expert-level understanding of the knowledge base. Our methodology relies on generating metadata and synthetic Questions and Answers (QA) for each document, as well as introducing the new concept of Meta Knowledge Summary (MK Summary) for metadata-based clusters of documents. The proposed innovations enable personalized user-query augmentation and in-depth information retrieval across the knowledge base. Our research makes two significant contributions: using LLMs as evaluators and employing new comparative performance metrics, we demonstrate that (1) using augmented queries with synthetic question matching significantly outperforms traditional RAG pipelines that rely on document chunking (p < 0.01), and (2) meta knowledge-augmented queries additionally significantly improve retrieval precision and recall, as well as the final answers breadth, depth, relevancy, and specificity. Our methodology is cost-effective, costing less than $20 per 2000 research papers using Claude 3 Haiku, and can be adapted with any fine-tuning of either the language or embedding models to further enhance the performance of end-to-end RAG pipelines., Comment: Accepted in Workshop on Generative AI for Recommender Systems and Personalization, KDD 2024

In the first part of the paper, we address an invertible matrix polynomial $L(z)$ and its inverse $\hat{L}(z) := -L(z)^{-1}$. We present a method for obtaining a canonical set of root functions and Jordan chains of $L(z)$ through elementary transformations of the matrix $L(z)$ alone. This method provides a new and simple approach to deriving a general solution of the system of ordinary linear differential equations $L\left(\frac{d}{dt}\right)u=0$ using only elementary transformations of the corresponding matrix polynomial $L(z)$. In the second part of the paper, given a matrix generalized Nevanlinna function $Q\in N_{\kappa }^{n \times n}$ and a canonical set of root functions of $\hat{Q}(z) := -Q(z)^{-1}$, we provide an algorithm to determine a specific Pontryagin space $(\mathcal{K}, [.,.])$, a specific self-adjoint operator $A:\mathcal{K}\rightarrow \mathcal{K}$ and an operator $\Gamma: \mathbb{C}^{n}\rightarrow \mathcal{K}$ that represent the function $Q$ in a Krein-Langer type representation. We demonstrate the main results through examples of linear systems of ODEs., Comment: 20 pages

Given Krein and Hilbert spaces $\left( \mathcal{K},[.,.] \right)$ and $\left( \mathcal{H}, \left( .,. \right) \right)$, respectively, the concept of the boundary triple $\Pi =(\mathcal{H}, \Gamma _{0}, \Gamma_{1})$ is generalized through the abstract Green's identity for the isometric relation $\Gamma$ between Krein spaces $\left( \mathcal{K}^{2}, \left[ .,.\right]_{\mathcal{K}^{2}} \right) $ and $\left(\mathcal{H}^{2}, \left[ .,.\right]_{\mathcal{H}^{2}} \right) $ without any conditions on $\dom\, \Gamma$ and $\ran\, \Gamma$. This also means that we do not assume the existence of a closed symmetric linear relation $S$ such that $\dom\, \Gamma=S^{+}$, which is a standard assumptions in all previous research of boundary triples. The main properties of such a general Green's boundary model are proven. In the process, some useful properties of the isometric relation $V$ between two Krein spaces $X$ and $Y$ are proven. Additionally, surprising properties of the unitary relation $\Gamma : \mathcal{K}^{2} \rightarrow\mathcal{H}^{2}$ and the self-adjoint main transformation $\tilde{A}$ of $\Gamma$ are discovered. Then, two statements about generalized Nevanlinna families are generalized using this Green's boundary model. Furthermore, several previously known boundary triples involving a Hilbert space $\mathcal{K}$ and reduction operator $\Gamma : \mathcal{K}^{2} \rightarrow\mathcal{H}^{2}$, such as AB-generalized, B-generalized, ordinary, isometric, unitary, quasi-boundary, and S-generalized boundary triples, have been extended to a Krein space $\mathcal{K}$ and linear relation $\Gamma$ using the Green's boundary model approach., Comment: 23 pages

We provide the necessary and sufficient conditions for a generalized Nevanlinna function $Q$ ($Q\in N_{\kappa }\left( \mathcal{H} \right)$) to be a Weyl function (also known as a Weyl-Titchmarch function). We also investigate an important subclass of $N_{\kappa }(\mathcal{H})$, the functions that have a boundedly invertible derivative at infinity $Q'\left( \infty \right):=\lim \limits_{z \to \infty}{zQ(z)}$. These functions are regular and have the operator representation $Q\left( z \right)=\tilde{\Gamma}^{+}\left( A-z \right)^{-1}\tilde{\Gamma},z\in \rho \left( A \right)$, where $A$ is a bounded self-adjoint operator in a Pontryagin space $\mathcal{K}$. We prove that every such strict function $Q$ is a Weyl function associated with the symmetric operator $S:=A_{\vert (I-P)\mathcal{K}}$, where $P$ is the orthogonal projection, $P:=\tilde{\Gamma} \left( \tilde{\Gamma}^{+} \tilde{\Gamma} \right)^{-1} \tilde{\Gamma}^{+} $. Additionally, we provide the relation matrices of the adjoint relation $S^{+}$ of $S$, and of $\hat{A}$, where $\hat{A}$ is the representing relation of $\hat{Q}:=-Q^{-1}$. We illustrate our results through examples, wherein we begin with a given function $Q\in N_{\kappa }\left( \mathcal{H} \right)$ and proceed to determine the closed symmetric linear relation $S$ and the boundary triple $\Pi$ so that $Q$ becomes the Weyl function associated with $\Pi$., Comment: 23 pages, Accepted in Sarajevo Journal of Mathematics

Necessary and sufficient conditions for reducidibility of a self-adjoint linear relation in a Krein space are given. Then a generalized Nevanlinna function $Q$, represented by a self-adjoint linear relation $A$, is decomposed by means of the reducing subspaces of $A$. The sum of two functions $Q_{i}{\in N}_{\kappa_{i}}\left( \mathcal{H} \right),\thinspace i=1,\thinspace 2$, minimally represented by the triplets $\left( \mathcal{K}_{i},A_{i},\Gamma_{i} \right)$, is also studied. For that purpose, a model $( \tilde{\mathcal{K}},\tilde{A},\tilde{\Gamma } )$ to represent $Q:=Q_{1}+Q_{2}$ in terms of $\left( \mathcal{K}_{i},A_{i},\Gamma_{i} \right)$ is created. By means of that model, necessary and sufficient conditions for $\kappa =\kappa_{1}+\kappa_{2}$ are proven in analytic terms. At the end, it is explained how degenerate Jordan chains of the representing relation $A$ affect reducing subspaces of $A$ and decomposition of the corresponding function $Q$., Comment: 27 pages; accepted in Ukr. Math. J

Let $\left(\mathcal{H},\left(.,.\right)\right)$ be a Hilbert space and let $\mathcal{L}\left(\mathcal{H}\right)$ be the linear space of bounded operators in $\mathcal{H}$. In this paper, we deal with $\mathcal{L}(\mathcal{H})$-valued function $Q$ that belongs to the generalized Nevanlinna class $\mathcal{N}_{\kappa} (\mathcal{H})$, where $\kappa$ is a non-negative integer. It is the class of functions meromorphic on $C \backslash R$, such that $Q(z)^{*}=Q(\bar{z})$ and the kernel $\mathcal{N}_{Q}\left( z,w \right):=\frac{Q\left( z \right)-{Q\left( w \right)}^{\ast }}{z-\bar{w}}$ has $\kappa$ negative squares. A focus is on the functions $Q \in \mathcal{N}_{\kappa} (\mathcal{H})$ which are holomorphic at $ \infty$. A new operator representation of the inverse function $\hat{Q}\left( z \right):=-{Q\left( z \right)}^{-1}$ is obtained under the condition that the derivative at infinity $Q^{'}\left( \infty\right):=\lim\limits_{z\to \infty}{zQ(z)}$ is boundedly invertible operator. It turns out that $\hat{Q}$ is the sum $\hat{Q}=\hat{Q}_{1}+\hat{Q}_{2},\, \, \hat{Q}_{i}\in \mathcal{N}_{\kappa_{i}}\left( \mathcal{H} \right)$ that satisfies $\kappa_{1}+\kappa_{2}=\kappa $. That decomposition enables us to study properties of both functions, $Q$ and $\hat{Q}$, by studying the simple components $\hat{Q}_{1}$ and $\hat{Q}_{2}$., Comment: 19 pages

In the first part of the article, a new interesting system of difference equations is introduced. It is developed for re-rating purposes in general insurance. A nonlinear transformation $\varphi $ of a d-dimensional $(d \ge 2)$ Euclidean space is introduced that enables us to express the system in the form $f^{t+1}:=\varphi (f^t),\, t=0,\, 1,\, 2,\, \ldots $. Under typical actuarial assumptions, existence of solutions of that system is proven by means of Brouwer's fixed point theorem in normed spaces. In addition, conditions that guarantee uniqueness of a solution are given. The second, smaller part of the article is about Leslie-Gower's system of $d \ge 2$ difference equations. We focus on the system that satisfies conditions consistent with weak inter-specific competition. We prove existence and uniqueness of the equilibrium of the model under surprisingly simple and very general conditions. Even though the two parts of this article have applications in two different sciences, they are connected with similar mathematics, in particular by our use of Brouwer's Fixed point Theorem., Comment: 16 pages; In this version, added Leslie-Gower model for competition of d>2 species.; Paper should appear in Journal of Difference Equations and Applications

For a given generalized Nevanlinna function $Q\in N_{\kappa }\left( H \right)$, we study decompositions that satisfy: $Q=Q_{1}+Q_{2}$; $Q_{i}{\in N}_{\kappa_{i}}\left( H \right)$, and $\kappa_{1}+\kappa_{2}=\kappa $, $0\le \kappa_{i}$, which we call desirable decompositions. In this paper, some sufficient conditions for such decompositions of $Q$ are given. One of the main results is a new operator representation of $\hat{Q}\left(z\right):=-{Q(z)}^{-1}$ if $Q\left( z \right):=\Gamma_{0}^{+}\left( A-z\right)^{-1}\Gamma_{0}$, where $A$ is a bounded self-adjoint operator in a Pontryagin space. The new representation is used to get an interesting desirable decomposition of $\hat{Q}$ and to obtain some information about singularities of $\hat{Q}$., Comment: 16 pages

In this paper the analytic characterization of generalized poles of operator valued generalized Nevanlinna functions (including the length of Jordan chains of the representing relation) is completed. In particular, given a Jordan chain of length $\ell$, we show that there exists a pole cancellation function of order at least $\ell$, and, moreover, this function is of surprisingly simple form.

In this paper we present a new approach to control variates for improving computational efficiency of Ensemble Monte Carlo. We present the approach using simulation of paths of a time-dependent nonlinear stochastic equation. The core idea is to extract information at one or more nominal model parameters and use this information to gain estimation efficiency at neighboring parameters. This idea is the basis of a general strategy, called DataBase Monte Carlo (DBMC), for improving efficiency of Monte Carlo. In this paper we describe how this strategy can be implemented using the variance reduction technique of Control Variates (CV). We show that, once an initial setup cost for extracting information is incurred, this approach can lead to significant gains in computational efficiency. The initial setup cost is justified in projects that require a large number of estimations or in those that are to be performed under real-time constraints., Comment: 15 pages, 2 ps figures, elsart.cls