Search

Novel-view synthesis of specular objects like shiny metals or glossy paints remains a significant challenge. Not only the glossy appearance but also global illumination effects, including reflections of other objects in the environment, are critical components to faithfully reproduce a scene. In this paper, we present Neural Directional Encoding (NDE), a view-dependent appearance encoding of neural radiance fields (NeRF) for rendering specular objects. NDE transfers the concept of feature-grid-based spatial encoding to the angular domain, significantly improving the ability to model high-frequency angular signals. In contrast to previous methods that use encoding functions with only angular input, we additionally cone-trace spatial features to obtain a spatially varying directional encoding, which addresses the challenging interreflection effects. Extensive experiments on both synthetic and real datasets show that a NeRF model with NDE (1) outperforms the state of the art on view synthesis of specular objects, and (2) works with small networks to allow fast (real-time) inference. The project webpage and source code are available at: \url{https://lwwu2.github.io/nde/}., Comment: Accepted to CVPR 2024

We introduce OpenIllumination, a real-world dataset containing over 108K images of 64 objects with diverse materials, captured under 72 camera views and a large number of different illuminations. For each image in the dataset, we provide accurate camera parameters, illumination ground truth, and foreground segmentation masks. Our dataset enables the quantitative evaluation of most inverse rendering and material decomposition methods for real objects. We examine several state-of-the-art inverse rendering methods on our dataset and compare their performances. The dataset and code can be found on the project page: https://oppo-us-research.github.io/OpenIllumination.

Inverse path tracing has recently been applied to joint material and lighting estimation, given geometry and multi-view HDR observations of an indoor scene. However, it has two major limitations: path tracing is expensive to compute, and ambiguities exist between reflection and emission. Our Factorized Inverse Path Tracing (FIPT) addresses these challenges by using a factored light transport formulation and finds emitters driven by rendering errors. Our algorithm enables accurate material and lighting optimization faster than previous work, and is more effective at resolving ambiguities. The exhaustive experiments on synthetic scenes show that our method (1) outperforms state-of-the-art indoor inverse rendering and relighting methods particularly in the presence of complex illumination effects; (2) speeds up inverse path tracing optimization to less than an hour. We further demonstrate robustness to noisy inputs through material and lighting estimates that allow plausible relighting in a real scene. The source code is available at: https://github.com/lwwu2/fipt, Comment: Updated experiment results; modified real-world sections

2-in-1 design (Chen et al. 2018) is becoming popular in oncology drug development, with the flexibility of using different endpoints at different decision time. Based on the observed interim data, sponsors choose either to seamlessly advance a small phase 2 trial to a full-scale confirmatory phase 3 trial with a pre-determined maximum sample size, or to remain in a phase 2 trial. This approach may increase efficiency in drug development but is rigid and requires a pre-specified fixed sample size. In this paper, we propose a flexible 2-in-1 design with sample size adaptation, while retains the advantage of allowing intermediate endpoint for interim decision. The proposed design reflects the needs of recent FDA's Project FrontRunner initiative to encourage using an earlier surrogate endpoint to potentially support accelerated approval with conversion to standard approval with long term endpoint from the same randomized study. Additionally, we identify the interim decision cut-off to allow conventional test procedure at the final analysis. Extensive simulation studies showed the proposed design require much smaller sample size and shorter timeline than the simple 2-in-1 design, while achieving similar power. A case study in multiple myeloma is used to demonstrate the benefits of the proposed design.

DIVeR builds on the key ideas of NeRF and its variants -- density models and volume rendering -- to learn 3D object models that can be rendered realistically from small numbers of images. In contrast to all previous NeRF methods, DIVeR uses deterministic rather than stochastic estimates of the volume rendering integral. DIVeR's representation is a voxel based field of features. To compute the volume rendering integral, a ray is broken into intervals, one per voxel; components of the volume rendering integral are estimated from the features for each interval using an MLP, and the components are aggregated. As a result, DIVeR can render thin translucent structures that are missed by other integrators. Furthermore, DIVeR's representation has semantics that is relatively exposed compared to other such methods -- moving feature vectors around in the voxel space results in natural edits. Extensive qualitative and quantitative comparisons to current state-of-the-art methods show that DIVeR produces models that (1) render at or above state-of-the-art quality, (2) are very small without being baked, (3) render very fast without being baked, and (4) can be edited in natural ways.

Adaptive subgroup enrichment design is an efficient design framework that allows accelerated development for investigational treatments while also having flexibility in population selection within the course of the trial. The adaptive decision at the interim analysis is commonly made based on the conditional probability of trial success. However, one of the critical challenges for such adaptive designs is immature data for interim decisions, particularly in the targeted subgroup with a limited sample size at the first stage of the trial. In this paper, we improve the interim decision making by incorporating information from surrogate endpoints when estimating conditional power at the interim analysis, by predicting the primary treatment effect based on the observed surrogate endpoint and prior knowledge or historical data about the relationship between endpoints. Modified conditional power is developed for both selecting the patient population to be enrolled after the interim analysis and sample size re-estimation. In the simulation study, our proposed design shows a higher chance to make desirable interim decisions and achieves higher overall power, while controlling the overall type I error. This performance is robust over drift of prior knowledge from the true relationship between two endpoints. We also demonstrate the application of our proposed design in two case studies in oncology and vaccine trials.

A sequential multiple assignment randomized trial (SMART) facilitates comparison of multiple adaptive treatment strategies (ATSs) simultaneously. Previous studies have established a framework to test the homogeneity of multiple ATSs by a global Wald test through inverse probability weighting. SMARTs are generally lengthier than classical clinical trials due to the sequential nature of treatment randomization in multiple stages. Thus, it would be beneficial to add interim analyses allowing for early stop if overwhelming efficacy is observed. We introduce group sequential methods to SMARTs to facilitate interim monitoring based on multivariate chi-square distribution. Simulation study demonstrates that the proposed interim monitoring in SMART (IM-SMART) maintains the desired type I error and power with reduced expected sample size compared to the classical SMART. Lastly, we illustrate our method by re-analyzing a SMART assessing the effects of cognitive behavioral and physical therapies in patients with knee osteoarthritis and comorbid subsyndromal depressive symptoms.