Search

Reward learning plays a pivotal role in Reinforcement Learning from Human Feedback (RLHF), ensuring the alignment of language models. The Bradley-Terry (BT) model stands as the prevalent choice for capturing human preferences from datasets containing pairs of chosen and rejected responses. In preference modeling, the focus is not on absolute values but rather on the reward difference between chosen and rejected responses, referred to as preference strength. Thus, precise evaluation of preference strength holds paramount importance in preference modeling. However, an easily overlooked factor significantly affecting preference strength measurement is that human attitudes towards two responses may not solely indicate a preference for one over the other and ties are also a common occurrence. To address this, we propose the adoption of the generalized Bradley-Terry model -- the Bradley-Terry model with ties (BTT) -- to accommodate tied preferences, thus leveraging additional information. We prove that even with the access to the true distributions of prompt and response, disregarding ties can lead to a notable bias in preference strength measurement. Comprehensive experiments further validate the advantages of incorporating ties in preference modeling. Notably, fine-tuning with BTT significantly outperforms fine-tuning with BT on synthetic preference datasets with ties, labeled by state-of-the-art open-source LLMs.

Ovarian cancer detection has traditionally relied on a multi-step process that includes biopsy, tissue staining, and morphological analysis by experienced pathologists. While widely practiced, this conventional approach suffers from several drawbacks: it is qualitative, time-intensive, and heavily dependent on the quality of staining. Mid-infrared (MIR) hyperspectral photothermal imaging is a label-free, biochemically quantitative technology that, when combined with machine learning algorithms, can eliminate the need for staining and provide quantitative results comparable to traditional histology. However, this technology is slow. This work presents a novel approach to MIR photothermal imaging that enhances its speed by an order of magnitude. Our method significantly accelerates data collection by capturing a combination of high-resolution and interleaved, lower-resolution infrared band images and applying computational techniques for data interpolation. We effectively minimize data collection requirements by leveraging sparse data acquisition and employing curvelet-based reconstruction algorithms. This method enables the reconstruction of high-quality, high-resolution images from undersampled datasets and achieving a 10X improvement in data acquisition time. We assessed the performance of our sparse imaging methodology using a variety of quantitative metrics, including mean squared error (MSE), structural similarity index (SSIM), and tissue subtype classification accuracies, employing both random forest and convolutional neural network (CNN) models, accompanied by ROC curves. Our statistically robust analysis, based on data from 100 ovarian cancer patient samples and over 65 million data points, demonstrates the method's capability to produce superior image quality and accurately distinguish between different gynecological tissue types with segmentation accuracy exceeding 95%.

In this paper, we provide some characterizations of strong pseudoconvexity by the boundary behavior of intrinsic invariants for smoothly bounded pseudoconvex domains of finite type in $\mathbb{C}^2$. As a consequence, if such domain is biholomorphically equivalent to a quotient of the unit ball, then it is strongly pseudoconvex.

A recent GPU implementation of the Restarted Primal-Dual Hybrid Gradient Method for Linear Programming was proposed in Lu and Yang (2023). Its computational results demonstrate the significant computational advantages of the GPU-based first-order algorithm on certain large-scale problems. The average performance also achieves a level close to commercial solvers for the first time in history. However, due to limitations in experimental hardware and the disadvantage of implementing the algorithm in Julia compared to C language, neither the commercial solver nor cuPDLP reached their maximum efficiency. Therefore, in this report, we have re-implemented and optimized cuPDLP in C language. Utilizing state-of-the-art CPU and GPU hardware, we extensively compare cuPDLP with the best commercial solvers. The experiments further highlight its substantial computational advantages and potential for solving large-scale linear programming problems. We also discuss the profound impact this breakthrough may have on mathematical programming research and the entire operations research community., Comment: fix typos, update numerical results

Accurate position and posture measurements of the freely-falling test mass are crucial for the success of spaceborne gravitational wave detection missions. This paper presents a novel laboratory-developed test mass motion readout that utilizes quadrant photodetectors to measure the translation and tilt of a test mass. Departing from conventional methods like Zeeman effect or AOM frequency shift modulation, the readout system employs the phase locking of two lasers to generate the dual-frequency heterodyne source. Notably, the out-of-loop sensitivity of the phase locking reaches below 30 pm/Hz1/2 within the frequency band of 1 mHz and 10 Hz. The system comprises three measurement interferometers and one reference interferometer, featuring a symmetric design that enables measurements of up to six degrees of freedom based on polarization-multiplexing and differential wavefront sensing. Ground-simulated experimental results demonstrate that the proposed system has achieved a measurement sensitivity of 4 pm/Hz1/2 and 2 nrad/Hz1/2 at 1 Hz, a resolution of 5 nm and 0.1 urad, a range of 200 um and 600 urad, respectively. These findings showcase the system's potential as an alternative method for precisely monitoring the motion of test masses in spaceborne gravitational wave detection missions and other applications requiring accurate positioning and multi-degrees-of-freedom sensing., Comment: 7 pages, 10 figures. arXiv admin note: substantial text overlap with arXiv:2207.06420

In this paper, we prove that the $L^p(\mathbb{R}^d)$ norm of the maximal truncated Riesz transform in terms of the $L^p(\mathbb{R}^d)$ norm of Riesz transform is dimension-free for any $2\leq p<\infty$, using integration by parts formula for radial Fourier multipliers. Moreover, we show that $$\|R_j^*f\|_{L^p}\leq \left({2+\frac{1}{\sqrt{2}}}\right)^{\frac{2}{p}}\|R_jf\|_{L^p},\ \ \mbox{for}\ \ p\geq2,\ \ d\geq2.$$ As by products of our calculations, we infer the $L^p$ norm contractivity of the truncated Riesz transforms $R^t_j$ in terms of $R_j$, and their accurate $L^p$ norms. More precisely, we prove: $$\|R^t_jf\|_{L^p}\leq\|R_jf\|_{L^p}$$ and $$\|R^t_j\|_{L^p}=\|R_j\|_{L^p},$$ for all $10.$, Comment: 16 pages; we add some remarks and a new proof for a limit in the Appendix, also the other referee's suggestions are incorporated

Let $\{(p_n, \mathcal{D}_n, L_n)\}$ be a sequence of Hadamard triples on $\mathbb{R}$. Suppose that the associated Cantor-Moran measure $$ \mu_{\{p_n,\mathcal{D}_n\}}=\delta_{p_1^{-1}\mathcal{D}_1}\ast\delta_{(p_2p_1)^{-1}\mathcal{D}_2}\ast\cdots, $$ where $\sup_n\{|p_n^{-1}d|:d\in \mathcal{D}_n\}<\infty$ and $\sup\#\mathcal{D}_n<\infty$. It has been observed that the spectrality of $\mu_{\{p_n,D_n\}}$ is determined by equi-positivity. A significant problem is what kind of Moran measures can satisfy this property. In this paper, we introduce the conception of \textit{Double Points Condition Set} (\textit{DPCS}) to characterize the equi-positivity equivalently. As applications of our characterization, we show that all singularly continuous Cantor-Moran measures are spectral. For the absolutely continuous case, we study Fuglede's Conjecture on Cantor-Moran set. We show that the equi-positivity of $\mu_{\{p_n,D_n\}}$ implies the tiling of its support, and the reverse direction holds under certain conditions., Comment: 34 pages

In this paper, we propose several new stochastic second-order algorithms for policy optimization that only require gradient and Hessian-vector product in each iteration, making them computationally efficient and comparable to policy gradient methods. Specifically, we propose a dimension-reduced second-order method (DR-SOPO) which repeatedly solves a projected two-dimensional trust region subproblem. We show that DR-SOPO obtains an $\mathcal{O}(\epsilon^{-3.5})$ complexity for reaching approximate first-order stationary condition and certain subspace second-order stationary condition. In addition, we present an enhanced algorithm (DVR-SOPO) which further improves the complexity to $\mathcal{O}(\epsilon^{-3})$ based on the variance reduction technique. Preliminary experiments show that our proposed algorithms perform favorably compared with stochastic and variance-reduced policy gradient methods.

In this paper, we prove that the identity map for the smoothly bounded pseudoconvex domain of finite type in $\mathbb{C}^2$ extends to a bi-H\"{o}lder map between the Euclidean boundary and Gromov boundary. As an application, we show the bi-H\"{o}lder boundary extensions for quasi-isometries between these domains. Moreover, we get a more accurate index of the Gehring-Hayman type theorem for the bounded $m$-convex domains with Dini-smooth boundary.

SOLNP+ is a derivative-free solver for constrained nonlinear optimization. It starts from SOLNP proposed in 1989 by Ye Ye with the main idea that uses finite difference to approximate the gradient. We incorporate the techniques of implicit filtering, new restart mechanism and modern quadratic programming solver into this new version with an ANSI C implementation. The algorithm exhibits a great advantage in running time and robustness under noise compared with the last version by MATLAB. SOLNP+ is free to download at https://github.com/COPT-Public/SOLNP_plus.

Mid-infrared spectroscopic imaging (MIRSI) is an emerging class of label-free techniques being leveraged for digital histopathology. Modern histopathologic identification of ovarian cancer involves tissue staining followed by morphological pattern recognition. This process is time-consuming, subjective, and requires extensive expertise. This paper presents the first label-free, quantitative, and automated histological recognition of ovarian tissue subtypes using a new MIRSI technique. This technique, called optical photothermal infrared (O-PTIR) imaging, provides a 10X enhancement in spatial resolution relative to prior instruments. It enables sub-cellular spectroscopic investigation of tissue at biochemically important fingerprint wavelengths. We demonstrate that enhanced resolution of sub-cellular features, combined with spectroscopic information, enables reliable classification of ovarian cell subtypes achieving a classification accuracy of 0.98. Moreover, we present statistically robust validation from 74 patient samples with over 60 million data points. We show that sub-cellular resolution from five wavenumbers is sufficient to outperform state-of-the-art diffraction-limited techniques from up to 235 wavenumbers. We also propose two quantitative biomarkers based on the relative quantities of epithelium and stroma that exhibits efficacy in early cancer diagnosis. This paper demonstrates that combining deep learning with intrinsic biochemical MIRSI measurements enables quantitative evaluation of cancerous tissue, improving the rigor and reproducibility of histopathology.

We present a new characterization of Muckenhoupt $A_{\infty}$-weights whose logarithm is in $\operatorname{VMO}(\mathbb{R})$ in terms of vanishing Carleson measures on $\mathbb{R}_+^2$ and vanishing doubling weights on $\mathbb{R}$. This also gives a novel description of strongly symmetric homeomorphisms on the real line by using a geometric quantity., Comment: In the new version, we adjusted some proofs of the main theorem and deleted the appendix. This version is accepted by PAMS