Search

Constructing 3D representations of object geometry is critical for many downstream manipulation tasks. These representations must be built from potentially noisy partial observations. In this work we focus on the problem of reconstructing a multi-object scene from a single RGBD image. Current deep learning approaches to this problem can be brittle to noisy real world observations and out-of-distribution objects. Other approaches that do not rely on training data cannot accurately infer the backside of objects. We propose BRRP, a reconstruction method that can leverage preexisting mesh datasets to build an informative prior during robust probabilistic reconstruction. In order to make our method more efficient, we introduce the concept of retrieval-augmented prior, where we retrieve relevant components of our prior distribution during inference. Our method produces a distribution over object shape that can be used for reconstruction or measuring uncertainty. We evaluate our method in both procedurally generated scenes and in real world scenes. We show our method is more robust than a deep learning approach while being more accurate than a method with an uninformative prior.

Coordinating the motion of robots with high degrees of freedom (DoF) to grasp objects gives rise to many challenges. In this paper, we propose a novel imitation learning approach to learn a policy that directly predicts 23 DoF grasp trajectories from a partial point cloud provided by a single, fixed camera. At the core of the approach is a second-order geometric-based model of behavioral dynamics. This Neural Geometric Fabric (NGF) policy predicts accelerations directly in joint space. We show that our policy is capable of generalizing to novel objects, and combine our policy with a geometric fabric motion planner in a loop to generate stable grasping trajectories. We evaluate our approach on a set of three different objects, compare different policy structures, and run ablation studies to understand the importance of different object encodings for policy learning., Comment: IEEE International Conference on Robotics and Automation (ICRA) Workshop on Geometric Representations 2023

We present a differentiable optimization-based framework for Task and Motion Planning (TAMP) that is massively parallelizable on GPUs, enabling thousands of sampled seeds to be optimized simultaneously. Existing sampling-based approaches inherently disconnect the parameters by generating samples for each independently and combining them through composition and rejection, while optimization-based methods struggle with highly non-convex constraints and local optima. Our method treats TAMP constraint satisfaction as optimizing a batch of particles, each representing an assignment to a plan skeleton's continuous parameters. We represent the plan skeleton's constraints using differentiable cost functions, enabling us to compute the gradient of each particle and update it toward satisfying solutions. Our use of GPU parallelism better covers the parameter space through scale, increasing the likelihood of finding the global optima by exploring multiple basins through global sampling. We demonstrate that our algorithm can effectively solve a highly constrained Tetris packing problem using a Franka arm in simulation and deploy our planner on a real robot arm. Website: https://williamshen-nz.github.io/gpu-tamp, Comment: 2-page paper presented at the CoRL 2024 Workshop on Differentiable Optimization Everywhere

Knowing the distribution between fresh and saline groundwater is imperative for sustainable and integrated management of water resources in coastal areas. The airborne electromagnetic (AEM) method is increasingly used for hydrogeological mapping over large areas via bulk electrical resistivity. However, accurately and reliably mapping the fresh/saltwater interface (FSI) requires accurate knowledge about the transition zone. The objective is to quantify the uncertainty in using AEM data to inform on the depth of the FSI. The study mimics a dual-moment time-domain SkyTEM sounding recorded in the Belgian coastal plain based on borehole data. It quantifies uncertainty using a differential evolution adaptive Metropolis algorithm to sample the posterior distribution. The results indicate the importance of reliable altitude, pitch and roll logging. Gathering prior knowledge about the transition zone, for example, through borehole logs, significantly improves the estimation of the FSI. The Resolve frequency-domain system, especially in context with very shallow to shallow FSIs, is more suitable for salinity mapping than the time-domain SkyTEM used in the field survey. The depth of the FSI may be defined via various threshold values. The uncertainty of three different thresholds is studied. The FSI based on the middle of the transition zone is the most reliable, while the FSI based on the 1500 mg/L total dissolved solids threshold is the least robust.

Advancements in light modulator technology have been driving discoveries and progress across various fields. The problem of large-scale coherent optical control of atomic quantum systems-including cold atoms, ions, and solid-state color centers-presents among the most stringent requirements. This motivates a new generation of high-speed large-scale modulator technology with the following requirements: (R1) operation at a design wavelength of choice in the visible (VIS) to near-infrared (NIR) spectrum, (R2) a scalable technology with a high channel density (> 100mm-2 ), (R3) a high modulation speed (> 100MHz), and (R4) a high extinction ratio (> 20 dB). To fulfill these requirements, we introduce a modulator technology based on piezoelectrically actuated silicon nitride resonant waveguide gratings fabricated on 200mm diameter silicon wafers with CMOS compatible processes. We present a proof-of-concept device with 4 x 4 individually addressable 50 {mu}m x 50 {mu}m pixels or channels, each containing a resonant waveguide grating with a ~ 780 nm design wavelength, supporting > 100MHz modulation speeds, and a spectral response with > 20 dB extinction., Comment: 9 pages + 2 pages supplementary

Recent work showed that large diffusion models can be reused as highly precise monocular depth estimators by casting depth estimation as an image-conditional image generation task. While the proposed model achieved state-of-the-art results, high computational demands due to multi-step inference limited its use in many scenarios. In this paper, we show that the perceived inefficiency was caused by a flaw in the inference pipeline that has so far gone unnoticed. The fixed model performs comparably to the best previously reported configuration while being more than 200$\times$ faster. To optimize for downstream task performance, we perform end-to-end fine-tuning on top of the single-step model with task-specific losses and get a deterministic model that outperforms all other diffusion-based depth and normal estimation models on common zero-shot benchmarks. We surprisingly find that this fine-tuning protocol also works directly on Stable Diffusion and achieves comparable performance to current state-of-the-art diffusion-based depth and normal estimation models, calling into question some of the conclusions drawn from prior works., Comment: Project page: https://vision.rwth-aachen.de/diffusion-e2e-ft

We present Points2Plans, a framework for composable planning with a relational dynamics model that enables robots to solve long-horizon manipulation tasks from partial-view point clouds. Given a language instruction and a point cloud of the scene, our framework initiates a hierarchical planning procedure, whereby a language model generates a high-level plan and a sampling-based planner produces constraint-satisfying continuous parameters for manipulation primitives sequenced according to the high-level plan. Key to our approach is the use of a relational dynamics model as a unifying interface between the continuous and symbolic representations of states and actions, thus facilitating language-driven planning from high-dimensional perceptual input such as point clouds. Whereas previous relational dynamics models require training on datasets of multi-step manipulation scenarios that align with the intended test scenarios, Points2Plans uses only single-step simulated training data while generalizing zero-shot to a variable number of steps during real-world evaluations. We evaluate our approach on tasks involving geometric reasoning, multi-object interactions, and occluded object reasoning in both simulated and real-world settings. Results demonstrate that Points2Plans offers strong generalization to unseen long-horizon tasks in the real world, where it solves over 85% of evaluated tasks while the next best baseline solves only 50%. Qualitative demonstrations of our approach operating on a mobile manipulator platform are made available at sites.google.com/stanford.edu/points2plans., Comment: Under review

Soft interfaces are ubiquitous in nature, governing quintessential hydrodynamics functions, like lubrication, stability and cargo transport. It is shown here how a magnetic force field at a magnetic-nonmagnetic fluid interface results in an ultra-soft interface with nonlinear elasticity and tunable viscous shear properties. The balance between magnetic pressure, viscous stress and Laplace pressure results in a deformed and stable liquid-in-liquid tube with apparent elasticity in the range 2 kPa -10 kPa, possibly extended by a proper choice of liquid properties. Such highly deformable liquid-liquid interfaces of arbitrary shape with vanishing viscous shear open doors to unique microfluidic phenomena, biomaterial flows and complex biosystems mimicking., Comment: 7 pages , 5 figures. arXiv admin note: text overlap with arXiv:2310.14280

Adding game elements to higher education is an increasingly common practice. As a result, many recent empirical studies focus on studying the effectiveness of gamified or game-based educational experiences. The findings of these studies are very diverse, showing both positive and negative effects, and thus calling for comparative meta-studies. In this paper we review and analyze different studies, aiming to summarise and evaluate controlled experiments conducted within different scientific disciplines. We focus on the clarity of non-experimental conditions' descriptions and show that in most cases a. educational methods used in control groups' activities are poorly described, b. educational materials used in control groups' activities are often unclear, and c. the starting conditions are unclear. We also noticed that studies in the fields of computer science and engineering, in general, report results more clearly than in other fields. Based on the above finding, we conclude with a few recommendations for the execution of future empirical studies of games in education for the sake of allowing a more structured comparison., Comment: draft

Prior research has explored potential applications of video games in programming education to elicit computational thinking skills. However, existing approaches are often either too general, not taking into account the diversity of genres and mechanisms between video games, or too narrow, selecting tools that were specifically designed for educational purposes. In this paper we propose a more fundamental approach, defining beneficial connections between individual design patterns present in video games and computational thinking skills. We argue that video games have the capacity to elicit these skills and even to potentially train them. This could be an effective method to solidify a conceptual base which would make programming education more effective., Comment: draft

A pivotal challenge in robotics is achieving fast, safe, and robust dexterous grasping across a diverse range of objects, an important goal within industrial applications. However, existing methods often have very limited speed, dexterity, and generality, along with limited or no hardware safety guarantees. In this work, we introduce DextrAH-G, a depth-based dexterous grasping policy trained entirely in simulation that combines reinforcement learning, geometric fabrics, and teacher-student distillation. We address key challenges in joint arm-hand policy learning, such as high-dimensional observation and action spaces, the sim2real gap, collision avoidance, and hardware constraints. DextrAH-G enables a 23 motor arm-hand robot to safely and continuously grasp and transport a large variety of objects at high speed using multi-modal inputs including depth images, allowing generalization across object geometry. Videos at https://sites.google.com/view/dextrah-g.

We prove an analog of the Deligne conjecture for prestacks. We show that given a prestack $\mathbb A$, its Gerstenhaber--Schack complex $\mathbf{C}_{\mathsf{GS}}(\mathbb A)$ is naturally an $E_2$-algebra. This structure generalises both the known $\mathsf{L}_\infty$-algebra structure on $\mathbf{C}_{\mathsf{GS}}(\mathbb A)$, as well as the Gerstenhaber algebra structure on its cohomology $\mathbf{H}_{\mathsf{GS}}(\mathbb A)$. The main ingredient is the proof of a conjecture of Hawkins \cite{hawkins}, stating that the homology of the dg operad $\mathsf{Quilt}$ has vanishing homology in positive degrees. As a corollary, $\Quilt$ is quasi-isomorphic to the operad $\mathsf{Brace}$ encoding brace algebras. In addition, we improve the $L_\infty$-structure on $\Quilt$ be showing that it originates from a $\mathsf{PreLie}_\infty$-structure lifting the $\mathsf{PreLie}$-structure on $\mathsf{Brace}$ in homology., Comment: Comments are welcome

Autonomous vehicles require a precise understanding of their environment to navigate safely. Reliable identification of unknown objects, especially those that are absent during training, such as wild animals, is critical due to their potential to cause serious accidents. Significant progress in semantic segmentation of anomalies has been driven by the availability of out-of-distribution (OOD) benchmarks. However, a comprehensive understanding of scene dynamics requires the segmentation of individual objects, and thus the segmentation of instances is essential. Development in this area has been lagging, largely due to the lack of dedicated benchmarks. To address this gap, we have extended the most commonly used anomaly segmentation benchmarks to include the instance segmentation task. Our evaluation of anomaly instance segmentation methods shows that this challenge remains an unsolved problem. The benchmark website and the competition page can be found at: https://vision.rwth-aachen.de/oodis ., Comment: Accepted at the VAND 2.0 Workshop at CVPR 2024. Project page: https://vision.rwth-aachen.de/oodis

PLATO (PLAnetary Transits and Oscillations of stars) is ESA's M3 mission designed to detect and characterise extrasolar planets and perform asteroseismic monitoring of a large number of stars. PLATO will detect small planets (down to <2 R_(Earth)) around bright stars (<11 mag), including terrestrial planets in the habitable zone of solar-like stars. With the complement of radial velocity observations from the ground, planets will be characterised for their radius, mass, and age with high accuracy (5 %, 10 %, 10 % for an Earth-Sun combination respectively). PLATO will provide us with a large-scale catalogue of well-characterised small planets up to intermediate orbital periods, relevant for a meaningful comparison to planet formation theories and to better understand planet evolution. It will make possible comparative exoplanetology to place our Solar System planets in a broader context. In parallel, PLATO will study (host) stars using asteroseismology, allowing us to determine the stellar properties with high accuracy, substantially enhancing our knowledge of stellar structure and evolution. The payload instrument consists of 26 cameras with 12cm aperture each. For at least four years, the mission will perform high-precision photometric measurements. Here we review the science objectives, present PLATO's target samples and fields, provide an overview of expected core science performance as well as a description of the instrument and the mission profile at the beginning of the serial production of the flight cameras. PLATO is scheduled for a launch date end 2026. This overview therefore provides a summary of the mission to the community in preparation of the upcoming operational phases.

The RSNA Abdominal Traumatic Injury CT (RATIC) dataset is the largest publicly available collection of adult abdominal CT studies annotated for traumatic injuries. This dataset includes 4,274 studies from 23 institutions across 14 countries. The dataset is freely available for non-commercial use via Kaggle at https://www.kaggle.com/competitions/rsna-2023-abdominal-trauma-detection. Created for the RSNA 2023 Abdominal Trauma Detection competition, the dataset encourages the development of advanced machine learning models for detecting abdominal injuries on CT scans. The dataset encompasses detection and classification of traumatic injuries across multiple organs, including the liver, spleen, kidneys, bowel, and mesentery. Annotations were created by expert radiologists from the American Society of Emergency Radiology (ASER) and Society of Abdominal Radiology (SAR). The dataset is annotated at multiple levels, including the presence of injuries in three solid organs with injury grading, image-level annotations for active extravasations and bowel injury, and voxelwise segmentations of each of the potentially injured organs. With the release of this dataset, we hope to facilitate research and development in machine learning and abdominal trauma that can lead to improved patient care and outcomes., Comment: 40 pages, 2 figures, 3 tables

In this paper, we propose composable part-based manipulation (CPM), a novel approach that leverages object-part decomposition and part-part correspondences to improve learning and generalization of robotic manipulation skills. By considering the functional correspondences between object parts, we conceptualize functional actions, such as pouring and constrained placing, as combinations of different correspondence constraints. CPM comprises a collection of composable diffusion models, where each model captures a different inter-object correspondence. These diffusion models can generate parameters for manipulation skills based on the specific object parts. Leveraging part-based correspondences coupled with the task decomposition into distinct constraints enables strong generalization to novel objects and object categories. We validate our approach in both simulated and real-world scenarios, demonstrating its effectiveness in achieving robust and generalized manipulation capabilities., Comment: Presented at CoRL 2023. For videos and additional results, see our website: https://cpmcorl2023.github.io/

Visual control policies can encounter significant performance degradation when visual conditions like lighting or camera position differ from those seen during training -- often exhibiting sharp declines in capability even for minor differences. In this work, we examine robustness to a suite of these types of visual changes for RGB-D and point cloud based visual control policies. To perform these experiments on both model-free and model-based reinforcement learners, we introduce a novel Point Cloud World Model (PCWM) and point cloud based control policies. Our experiments show that policies that explicitly encode point clouds are significantly more robust than their RGB-D counterparts. Further, we find our proposed PCWM significantly outperforms prior works in terms of sample efficiency during training. Taken together, these results suggest reasoning about the 3D scene through point clouds can improve performance, reduce learning time, and increase robustness for robotic learners. Project Webpage: https://pvskand.github.io/projects/PCWM, Comment: Accepted at International Conference on Robotics and Automation, 2024

The ability to construct concise scene representations from sensor input is central to the field of robotics. This paper addresses the problem of robustly creating a 3D representation of a tabletop scene from a segmented RGB-D image. These representations are then critical for a range of downstream manipulation tasks. Many previous attempts to tackle this problem do not capture accurate uncertainty, which is required to subsequently produce safe motion plans. In this paper, we cast the representation of 3D tabletop scenes as a multi-class classification problem. To tackle this, we introduce V-PRISM, a framework and method for robustly creating probabilistic 3D segmentation maps of tabletop scenes. Our maps contain both occupancy estimates, segmentation information, and principled uncertainty measures. We evaluate the robustness of our method in (1) procedurally generated scenes using open-source object datasets, and (2) real-world tabletop data collected from a depth camera. Our experiments show that our approach outperforms alternative continuous reconstruction approaches that do not explicitly reason about objects in a multi-class formulation.

The development of superconducting qubit technology has shown great potential for the construction of practical quantum computers. As the complexity of quantum processors continues to grow, the need for stringent fabrication tolerances becomes increasingly critical. Utilizing advanced industrial fabrication processes could facilitate the necessary level of fabrication control to support the continued scaling of quantum processors. However, these industrial processes are currently not optimized to produce high coherence devices, nor are they a priori compatible with the commonly used approaches to make superconducting qubits. In this work, we demonstrate for the first time superconducting transmon qubits manufactured in a 300 mm CMOS pilot line, using industrial fabrication methods, with resulting relaxation and coherence times already exceeding 100 microseconds. We show across-wafer, large-scale statistics studies of coherence, yield, variability, and aging that confirm the validity of our approach. The presented industry-scale fabrication process, using exclusively optical lithography and reactive ion etching, shows performance and yield similar to the conventional laboratory-style techniques utilizing metal lift-off, angled evaporation, and electron-beam writing. Moreover, it offers potential for further upscaling by including three-dimensional integration and additional process optimization using advanced metrology and judicious choice of processing parameters and splits. This result marks the advent of more reliable, large-scale, truly CMOS-compatible fabrication of superconducting quantum computing processors., Comment: main text: 7 pages, 4 figures. bibliography: 55 references. supplement: 8 sections, 8 figures, 2 tables

Flows are omnipresent and govern the dynamics of plasma. Solar tornadoes are a class of apparently rotating prominences, that might be formed by thermal instability. In spectroscopic studies on thermal instability background flow is commonly neglected. We here determine the effect of background flow on thermal instability in cylindrical magnetic field configurations as the influence of various parameters on the MHD spectrum. We investigate discrete thermal modes. In an analytical study, we extend upon the literature by including a generic background flow in a cylindrical coordinate system. The non-adiabatic MHD equations are linearised, Fourier-analysed, and are examined to understand how a background flow changes the continua. An approximate expression for discrete thermal modes is derived using a WKB analysis. The analytical results are then verified for a benchmark equilibrium using the eigenvalue code Legolas. The eigenfunctions of discrete thermal modes are visualised in 2D and 3D. The thermal continuum is Doppler-shifted due to the background flow, just like the slow and Alfv\'en continua. Discrete modes are altered because the governing equations contain flow-related terms. An approximate expression to predict the appearance of discrete thermal modes based on the equilibrium parameters is derived. All analytical expressions match the numerical results. The distribution of the density perturbations of the discrete thermal modes is not a singular condensation. 3D visualisation of the total velocity field shows that the helical field is heavily influenced by the radial velocity perturbation. We derived analytic expressions for non-adiabatic MHD modes of a cylindrical equilibrium with background flow and verified them using a coronal equilibrium. However, the equations are valid for and can be applied in other astrophysical environments., Comment: 19 pages, 22 figures, accepted for publication by A&A

Ferrofluids kept in place by permanent magnet quadrupoles can act as liquid walls to surround a second non-magnetic inside, resulting in a liquid fluidic channel with diameter size ranging from mm down to less than 10 micrometer. Micro particle tracking velocimetry (micro PTV) experiments and modeling show that near ideal plug flow is possible in such liquid-in-liquid channels due to the reduced friction at the walls. The measured fluids velocity profiles agree with the predictions of a hydrodynamic model of cylindrical symmetry with a minimal set of hypotheses. By introducing symmetry breaking elements in the system, we show how unique velocity and flow properties can be obtained. Our liquid-in-liquid confinement opens new possibilities for < 10 micrometer-sized microfluidics with low pressures and low shear, with flow characteristics not attainable in comparable solid-wall devices., Comment: 12 pages 7 figures

Single photon emitters with internal spin are leading contenders for developing quantum repeater networks, enabling long-range entanglement distribution for transformational technologies in communications and sensing. However, scaling beyond current few-node networks will require radical improvements to quantum link efficiencies and fidelities. Solid-state emitters are particularly promising due to their crystalline environment, enabling nanophotonic integration and providing spins for memory and processing. However, inherent spatial and temporal variations in host crystals give rise to static shifts and dynamic fluctuations in optical transition frequencies, posing formidable challenges in establishing large-scale, multipartite entanglement. Here, we introduce a scalable approach to quantum networking that utilizes frequency erasing photon detection in conjunction with adaptive, real-time quantum control. This enables frequency multiplexed entanglement distribution that is also insensitive to deleterious optical frequency fluctuations. Single rare-earth ions are an ideal platform for implementing this protocol due to their long spin coherence, narrow optical inhomogeneous distributions, and long photon lifetimes. Using two 171Yb:YVO4 ions in remote nanophotonic cavities we herald bipartite entanglement and probabilistically teleport quantum states. Then, we extend this protocol to include a third ion and prepare a tripartite W state: a useful input for advanced quantum networking applications. Our results provide a practical route to overcoming universal limitations imposed by non-uniformity and instability in solid-state emitters, whilst also showcasing single rare-earth ions as a scalable platform for the future quantum internet.

This study investigates the application of ordinal regression methods for categorizing disease severity in chest radiographs. We propose a framework that divides the ordinal regression problem into three parts: a model, a target function, and a classification function. Different encoding methods, including one-hot, Gaussian, progress-bar, and our soft-progress-bar, are applied using ResNet50 and ViT-B-16 deep learning models. We show that the choice of encoding has a strong impact on performance and that the best encoding depends on the chosen weighting of Cohen's kappa and also on the model architecture used. We make our code publicly available on GitHub., Comment: 17 pages, 3 figures, the code is available at: https://github.com/paddyOnGithub/ordinal_regression

Robotic pick and place stands at the heart of autonomous manipulation. When conducted in cluttered or complex environments robots must jointly reason about the selected grasp and desired placement locations to ensure success. While several works have examined this joint pick-and-place problem, none have fully leveraged recent learning-based approaches for multi-fingered grasp planning. We present a modular algorithm for joint pick and place planning that can make use of state of the art grasp classifiers for planning multi-fingered grasps for novel objects from partial view point clouds. We demonstrate our joint pick and place formulation with several costs associated with different placement tasks. Experiments on pick and place tasks with cluttered scenes using a physical robot show that our joint inference method is more successful than a sequential pick then place approach, while also achieving better placement configurations., Comment: 8 pages, 14 figures, IEEE RA-L

Background: Cell segmentation in bright-field histological slides is a crucial topic in medical image analysis. Having access to accurate segmentation allows researchers to examine the relationship between cellular morphology and clinical observations. Unfortunately, most segmentation methods known today are limited to nuclei and cannot segmentate the cytoplasm. Material & Methods: We present a new network architecture Cyto R-CNN that is able to accurately segment whole cells (with both the nucleus and the cytoplasm) in bright-field images. We also present a new dataset CytoNuke, consisting of multiple thousand manual annotations of head and neck squamous cell carcinoma cells. Utilizing this dataset, we compared the performance of Cyto R-CNN to other popular cell segmentation algorithms, including QuPath's built-in algorithm, StarDist and Cellpose. To evaluate segmentation performance, we calculated AP50, AP75 and measured 17 morphological and staining-related features for all detected cells. We compared these measurements to the gold standard of manual segmentation using the Kolmogorov-Smirnov test. Results: Cyto R-CNN achieved an AP50 of 58.65% and an AP75 of 11.56% in whole-cell segmentation, outperforming all other methods (QuPath $19.46/0.91\%$; StarDist $45.33/2.32\%$; Cellpose $31.85/5.61\%$). Cell features derived from Cyto R-CNN showed the best agreement to the gold standard ($\bar{D} = 0.15$) outperforming QuPath ($\bar{D} = 0.22$), StarDist ($\bar{D} = 0.25$) and Cellpose ($\bar{D} = 0.23$). Conclusion: Our newly proposed Cyto R-CNN architecture outperforms current algorithms in whole-cell segmentation while providing more reliable cell measurements than any other model. This could improve digital pathology workflows, potentially leading to improved diagnosis. Moreover, our published dataset can be used to develop further models in the future.

Prestacks are algebro-geometric objects whose defining relations are far from quadratic. Indeed, they are cubic and quartic, and moreover inhomogeneous. We show that box operads, a rectangular type of operads introduced in arXiv:2305.20036, constitute the correct framework to encode them and resolve their relations up to homotopy. Our first main result is a Koszul duality theory for box operads, extending the duality for (nonsymmetric) operads. In this new theory, the classical restriction of being quadratic is replaced by the notion of being $\textit{thin-quadratic}$, a condition referring to a particular class of ``thin'' operations. Our main case of interest is the box operad $Lax$ encoding lax prestacks as algebras. We show that $Lax$ is not Koszul by explicitly computing its Koszul complex. We then go on to remedy the situation by suitably restricting the Koszul dual box cooperad $Lax^{\antishriek}$ to obtain our second main result: we establish a minimal (in particular cofibrant) model $Lax_{\infty}$ for the box operad $Lax$ encoding lax prestacks. This sheds new light on Markl's question on the existence of a cofibrant model for the operad encoding presheaves of algebras from arXiv:math/0103052. Indeed, we answer the parallel question with presheaves viewed as prestacks, and prestacks considered as algebras over a box operad, in the positive., Comment: 45 pages, lots of pictures; All comments are welcome!

The computational resources required to solve the full 3D inversion of time-domain electromagnetic data are immense. To overcome the time-consuming 3D simulations, we construct a surrogate model, more precisely, a data-driven statistical model to replace the 3D simulations. It is trained on 3D data and predicts the approximate output much faster. We construct a surrogate model that predicts the discrepancy between a 1D subsurface model and a deviation of the 1D assumption. The latter response is fastly computable with a semi-analytical 1D forward model. We exemplify the approach on a two-layered case. The results are encouraging even with few training samples. Given the computational cost related to the 3D simulations, there are limitations in the number of training samples that can be generated. In addition, certain applications require a wide range of parameters to be sampled, such as the electrical conductivity parameters in a saltwater intrusion case. The challenge of this work is achieving the best possible accuracy with only a few thousand samples. We propose to view the performance in terms of learning gain, representing the gain from the surrogate model whilst still acknowledging a residual discrepancy. Our works open new avenues for effectively simulating 3D TDEM data.

Laser beam shaping offers remarkable possibilities to control and optimise process stability and tailor material properties and structure in laser-based welding and additive manufacturing. However, little is known about the influence of laser beam shaping on the complex melt-pool behaviour, solidified melt-track bead profile and microstructural grain morphology in laser material processing. A simulation-based approach is utilised in the present work to study the effects of laser beam intensity profile and angle of incidence on the melt-pool behaviour in conduction-mode laser melting of stainless steel 316L plates. The present high-fidelity physics-based computational model accounts for crucial physical phenomena in laser material processing such as complex laser-matter interaction, solidification and melting, heat and fluid flow dynamics, and free-surface oscillations. Experiments were carried out using different laser beam shapes and the validity of the numerical predictions is demonstrated. The results indicate that for identical processing parameters, reshaping the laser beam leads to notable changes in the thermal and fluid flow fields in the melt pool, affecting the melt-track bead profile and solidification microstructure. The columnar-to-equiaxed transition is discussed for different laser-intensity profiles., Comment: 27 pages, 10 figures

The Rayleigh-Plateau instability (RPI) is a classical hydrodynamics phenomenon that prevents a jet of liquid to flow indefinitely within air or another liquid. Here, we show how adding a magnetic force field makes possible its suppression. Enclosing the jet in a ferrofluid held by magnetic forces allows flow focusing without sheath flow, which completely avoids dripping failure at small flow rates and provides conditional stability for a continuous fluid jet. Highly deformable liquid interfaces withstanding spatial and time varying flow conditions within a large parameter space can be realized., Comment: 14 pages , 4 figures

Robots need to have a memory of previously observed, but currently occluded objects to work reliably in realistic environments. We investigate the problem of encoding object-oriented memory into a multi-object manipulation reasoning and planning framework. We propose DOOM and LOOM, which leverage transformer relational dynamics to encode the history of trajectories given partial-view point clouds and an object discovery and tracking engine. Our approaches can perform multiple challenging tasks including reasoning with occluded objects, novel objects appearance, and object reappearance. Throughout our extensive simulation and real-world experiments, we find that our approaches perform well in terms of different numbers of objects and different numbers of distractor actions. Furthermore, we show our approaches outperform an implicit memory baseline., Comment: Presented at IEEE Conference on Robotics and Automation (ICRA) 2024. Website: https://sites.google.com/view/rdmemory

Shape servoing, a robotic task dedicated to controlling objects to desired goal shapes, is a promising approach to deformable object manipulation. An issue arises, however, with the reliance on the specification of a goal shape. This goal has been obtained either by a laborious domain knowledge engineering process or by manually manipulating the object into the desired shape and capturing the goal shape at that specific moment, both of which are impractical in various robotic applications. In this paper, we solve this problem by developing a novel neural network DefGoalNet, which learns deformable object goal shapes directly from a small number of human demonstrations. We demonstrate our method's effectiveness on various robotic tasks, both in simulation and on a physical robot. Notably, in the surgical retraction task, even when trained with as few as 10 demonstrations, our method achieves a median success percentage of nearly 90%. These results mark a substantial advancement in enabling shape servoing methods to bring deformable object manipulation closer to practical, real-world applications., Comment: Submitted to IEEE Conference on Robotics and Automation (ICRA) 2024. 8 pages, 11 figures

The quality of software produced by students is often poor. How to teach students to develop good quality software has long been a topic in computer science education and research. We must conclude that we still do not have a good answer to this question. Specifications are necessary to determine the correctness of software, to develop error-free software and to write complete tests. Several attempts have been made to teach students to write specifications before writing code. So far, that has not proven to be very successful: Students do not like to write a specification and do not see the benefits of writing specifications. In this paper we focus on the use of informal specifications. Instead of teaching students how to write specifications, we teach them how to use informal specifications to develop correct software. The results were surprising: the number of errors in software and the completeness of tests both improved considerably and, most importantly, students really appreciate the specifications. We think that if students appreciate specification, we have a key to teach them how to specify and to appreciate its value., Comment: 14 pages

A single unexpected object on the road can cause an accident or may lead to injuries. To prevent this, we need a reliable mechanism for finding anomalous objects on the road. This task, called anomaly segmentation, can be a stepping stone to safe and reliable autonomous driving. Current approaches tackle anomaly segmentation by assigning an anomaly score to each pixel and by grouping anomalous regions using simple heuristics. However, pixel grouping is a limiting factor when it comes to evaluating the segmentation performance of individual anomalous objects. To address the issue of grouping multiple anomaly instances into one, we propose an approach that produces accurate anomaly instance masks. Our approach centers on an out-of-distribution segmentation model for identifying uncertain regions and a strong generalist segmentation model for anomaly instances segmentation. We investigate ways to use uncertain regions to guide such a segmentation model to perform segmentation of anomalous instances. By incorporating strong object priors from a generalist model we additionally improve the per-pixel anomaly segmentation performance. Our approach outperforms current pixel-level anomaly segmentation methods, achieving an AP of 80.08% and 88.98% on the Fishyscapes Lost and Found and the RoadAnomaly validation sets respectively. Project page: https://vision.rwth-aachen.de/ugains, Comment: Accepted for publication at GCPR 2023; Project page at https://vision.rwth-aachen.de/ugains

The $\textrm{HRI}_\textrm{EUV}$ telescope was calibrated on ground at the Physikalisch-Technische Bundesanstalt (PTB), Germany's national metrology institute, using the Metrology Light Source (MLS) synchrotron in April 2017 during the calibration campaign of the Extreme Ultraviolet Imager (EUI) instrument onboard the Solar Orbiter mission. We use the pre-flight end-to-end calibration and component-level (mirror multilayer coatings, filters, detector) characterization results to establish the beginning-of-life performance of the $\textrm{HRI}_\textrm{EUV}$ telescope which shall serve as a reference for radiometric analysis and monitoring of the telescope in-flight degradation. Calibration activities at component level and end-to-end calibration of the instrument were performed at PTB/MLS synchrotron light source (Berlin, Germany) and the SOLEIL synchrotron. Each component optical property is measured and compared to its semi-empirical model. This pre-flight characterization is used to estimate the parameters of the semi-empirical models. The end-to-end response is measured and validated by comparison with calibration measurements, as well as with its main design performance requirements. The telescope spectral response semi-empirical model is validated by the pre-flight end-to-end ground calibration of the instrument. It is found that $\textrm{HRI}_\textrm{EUV}$ is a highly efficient solar EUV telescope with a peak efficiency superior to 1 e$^-$.ph$^{-1}$), low detector noise ($\approx$ 3 e- rms), low dark current at operating temperature, and a pixel saturation above 120 ke- in low-gain or combined image mode. The ground calibration also confirms a well-modeled spectral selectivity and rejection, and low stray light. The EUI instrument achieves state-of-the-art performance in terms of signal-to-noise and image spatial resolution.

One cannot make truly fair decisions using integer linear programs unless one controls the selection probabilities of the (possibly many) optimal solutions. For this purpose, we propose a unified framework when binary decision variables represent agents with dichotomous preferences, who only care about whether they are selected in the final solution. We develop several general-purpose algorithms to fairly select optimal solutions, for example, by maximizing the Nash product or the minimum selection probability, or by using a random ordering of the agents as a selection criterion (Random Serial Dictatorship). We also discuss in detail how to extend the proposed methods when agents have cardinal preferences. As such, we embed the black-box procedure of solving an integer linear program into a framework that is explainable from start to finish. Lastly, we evaluate the proposed methods on two specific applications, namely kidney exchange (dichotomous preferences), and the scheduling problem of minimizing total tardiness on a single machine (cardinal preferences). We find that while the methods maximizing the Nash product or the minimum selection probability outperform the other methods on the evaluated welfare criteria, methods such as Random Serial Dictatorship perform reasonably well in computation times that are similar to those of finding a single optimal solution.

We introduce a symmetric operad $\square p$ ("box-op") which describes a certain calculus of rectangular labeled ``boxes''. Algebras over $\square p$, which we call box operads, have appeared under the name of fc multicategories in work by Leinster \cite{LeinsterFcmulticategories1999}. In our main result, we endow a suitable (graded, zero differential) totalisation $\square p_{\mathrm{td}}$ with a morphism $L_{\infty} \rightarrow \square p_{\mathrm{td}}$. We show that $\square p$ acts on an $\mathbb{N}^3$-graded enlargement of the $\mathbb{N}^2$-graded Gerstenhaber-Schack object $\mathbf{C}_{GS}(\mathbb{A})$ of a quiver $\mathbb{A}$ on a small category from \cite{DinhVanLowen2018}. This action restricts to an $L_{\infty}$-structure on $\mathbf{C}_{GS}(\mathbb{A})$ (with zero differential). For an element $\alpha = (m,f,c) \in \mathbf{C}_{GS}^2(\mathbb{A})$, the Maurer-Cartan equation holds precisely when $(\mathbb{A}, m, f, c)$ is a lax prestack with multiplications $m$, restrictions $f$, and twists $c$. As a consequence, the $\alpha$-twisted $L_{\infty}$-structure on $\mathbf{C}_{GS}(\mathbb{A})$ controls the deformation theory of $(\mathbb{A}, \alpha)$ as a lax prestack., Comment: 22 pages + 8 appendix. Minor changes, references updated

Most observations of the solar corona beyond 2 Rs consist of broadband visible light imagery from coronagraphs. The associated diagnostics mainly consist of kinematics and derivations of the electron number density. While the measurement of the properties of emission lines can provide crucial additional diagnostics of the coronal plasma (temperatures, velocities, abundances, etc.), these observations are comparatively rare. In visible wavelengths, observations at these heights are limited to total eclipses. In the VUV range, very few additional observations have been achieved since the pioneering results of UVCS. One of the objectives of the Full Sun Imager (FSI) channel of the EUI telescope on board the Solar Orbiter mission has been to provide very wide field-of-view EUV diagnostics of the morphology and dynamics of the solar atmosphere in temperature regimes that are typical of the lower transition region and of the corona. FSI carries out observations in two narrowbands of the EUV spectrum centered on 17.4 nm and 30.4 nm that are dominated, respectively, by lines of Fe IX/X (formed in the corona around 1 MK) and by the resonance line of He II (formed around 80 kK in the lower transition region). Unlike previous EUV imagers, FSI includes a moveable occulting disk that can be inserted in the optical path to reduce the amount of instrumental stray light to a minimum. FSI detects signals at 17.4 nm up to the edge of its FOV (7~Rs), which is about twice further than was previously possible. Comparisons with observations by the LASCO and Metis coronagraphs confirm the presence of morphological similarities and differences between the broadband visible light and EUV emissions, as documented on the basis of prior eclipse and space-based observations. The very-wide-field observations of FSI are paving the way for future dedicated instruments.

Objects rarely sit in isolation in everyday human environments. If we want robots to operate and perform tasks in our human environments, they must understand how the objects they manipulate will interact with structural elements of the environment for all but the simplest of tasks. As such, we'd like our robots to reason about how multiple objects and environmental elements relate to one another and how those relations may change as the robot interacts with the world. We examine the problem of predicting inter-object and object-environment relations between previously unseen objects and novel environments purely from partial-view point clouds. Our approach enables robots to plan and execute sequences to complete multi-object manipulation tasks defined from logical relations. This removes the burden of providing explicit, continuous object states as goals to the robot. We explore several different neural network architectures for this task. We find the best performing model to be a novel transformer-based neural network that both predicts object-environment relations and learns a latent-space dynamics function. We achieve reliable sim-to-real transfer without any fine-tuning. Our experiments show that our model understands how changes in observed environmental geometry relate to semantic relations between objects. We show more videos on our website: https://sites.google.com/view/erelationaldynamics., Comment: Accepted at IEEE Transactions on Robotics (T-RO). arXiv admin note: text overlap with arXiv:2209.11943

Applications in fields ranging from home care to warehouse fulfillment to surgical assistance require robots to reliably manipulate the shape of 3D deformable objects. Analytic models of elastic, 3D deformable objects require numerous parameters to describe the potentially infinite degrees of freedom present in determining the object's shape. Previous attempts at performing 3D shape control rely on hand-crafted features to represent the object shape and require training of object-specific control models. We overcome these issues through the use of our novel DeformerNet neural network architecture, which operates on a partial-view point cloud of the manipulated object and a point cloud of the goal shape to learn a low-dimensional representation of the object shape. This shape embedding enables the robot to learn a visual servo controller that computes the desired robot end-effector action to iteratively deform the object toward the target shape. We demonstrate both in simulation and on a physical robot that DeformerNet reliably generalizes to object shapes and material stiffness not seen during training, including ex vivo chicken muscle tissue. Crucially, using DeformerNet, the robot successfully accomplishes three surgical sub-tasks: retraction (moving tissue aside to access a site underneath it), tissue wrapping (a sub-task in procedures like aortic stent placements), and connecting two tubular pieces of tissue (a sub-task in anastomosis)., Comment: Submitted to IEEE Transactions on Robotics (T-RO). 20 pages, 27 figures. arXiv admin note: text overlap with arXiv:2110.04685

Most state-of-the-art instance segmentation methods rely on large amounts of pixel-precise ground-truth annotations for training, which are expensive to create. Interactive segmentation networks help generate such annotations based on an image and the corresponding user interactions such as clicks. Existing methods for this task can only process a single instance at a time and each user interaction requires a full forward pass through the entire deep network. We introduce a more efficient approach, called DynaMITe, in which we represent user interactions as spatio-temporal queries to a Transformer decoder with a potential to segment multiple object instances in a single iteration. Our architecture also alleviates any need to re-compute image features during refinement, and requires fewer interactions for segmenting multiple instances in a single image when compared to other methods. DynaMITe achieves state-of-the-art results on multiple existing interactive segmentation benchmarks, and also on the new multi-instance benchmark that we propose in this paper., Comment: Accepted to ICCV 2023

Recently, the self-supervised learning framework data2vec has shown inspiring performance for various modalities using a masked student-teacher approach. However, it remains open whether such a framework generalizes to the unique challenges of 3D point clouds. To answer this question, we extend data2vec to the point cloud domain and report encouraging results on several downstream tasks. In an in-depth analysis, we discover that the leakage of positional information reveals the overall object shape to the student even under heavy masking and thus hampers data2vec to learn strong representations for point clouds. We address this 3D-specific shortcoming by proposing point2vec, which unleashes the full potential of data2vec-like pre-training on point clouds. Our experiments show that point2vec outperforms other self-supervised methods on shape classification and few-shot learning on ModelNet40 and ScanObjectNN, while achieving competitive results on part segmentation on ShapeNetParts. These results suggest that the learned representations are strong and transferable, highlighting point2vec as a promising direction for self-supervised learning of point cloud representations., Comment: Accepted at GCPR 2023. Project page at https://vision.rwth-aachen.de/point2vec

Robotic grasping of 3D deformable objects is critical for real-world applications such as food handling and robotic surgery. Unlike rigid and articulated objects, 3D deformable objects have infinite degrees of freedom. Fully defining their state requires 3D deformation and stress fields, which are exceptionally difficult to analytically compute or experimentally measure. Thus, evaluating grasp candidates for grasp planning typically requires accurate, but slow 3D finite element method (FEM) simulation. Sampling-based grasp planning is often impractical, as it requires evaluation of a large number of grasp candidates. Gradient-based grasp planning can be more efficient, but requires a differentiable model to synthesize optimal grasps from initial candidates. Differentiable FEM simulators may fill this role, but are typically no faster than standard FEM. In this work, we propose learning a predictive graph neural network (GNN), DefGraspNets, to act as our differentiable model. We train DefGraspNets to predict 3D stress and deformation fields based on FEM-based grasp simulations. DefGraspNets not only runs up to 1500 times faster than the FEM simulator, but also enables fast gradient-based grasp optimization over 3D stress and deformation metrics. We design DefGraspNets to align with real-world grasp planning practices and demonstrate generalization across multiple test sets, including real-world experiments., Comment: To be published in the IEEE Conference on Robotics and Automation (ICRA), 2023

The general domain of video segmentation is currently fragmented into different tasks spanning multiple benchmarks. Despite rapid progress in the state-of-the-art, current methods are overwhelmingly task-specific and cannot conceptually generalize to other tasks. Inspired by recent approaches with multi-task capability, we propose TarViS: a novel, unified network architecture that can be applied to any task that requires segmenting a set of arbitrarily defined 'targets' in video. Our approach is flexible with respect to how tasks define these targets, since it models the latter as abstract 'queries' which are then used to predict pixel-precise target masks. A single TarViS model can be trained jointly on a collection of datasets spanning different tasks, and can hot-swap between tasks during inference without any task-specific retraining. To demonstrate its effectiveness, we apply TarViS to four different tasks, namely Video Instance Segmentation (VIS), Video Panoptic Segmentation (VPS), Video Object Segmentation (VOS) and Point Exemplar-guided Tracking (PET). Our unified, jointly trained model achieves state-of-the-art performance on 5/7 benchmarks spanning these four tasks, and competitive performance on the remaining two. Code and model weights are available at: https://github.com/Ali2500/TarViS, Comment: Accepted to CVPR'23 (Highlight). Code is available at: https://github.com/Ali2500/TarViS

Deep learning-based 3D human pose estimation performs best when trained on large amounts of labeled data, making combined learning from many datasets an important research direction. One obstacle to this endeavor are the different skeleton formats provided by different datasets, i.e., they do not label the same set of anatomical landmarks. There is little prior research on how to best supervise one model with such discrepant labels. We show that simply using separate output heads for different skeletons results in inconsistent depth estimates and insufficient information sharing across skeletons. As a remedy, we propose a novel affine-combining autoencoder (ACAE) method to perform dimensionality reduction on the number of landmarks. The discovered latent 3D points capture the redundancy among skeletons, enabling enhanced information sharing when used for consistency regularization. Our approach scales to an extreme multi-dataset regime, where we use 28 3D human pose datasets to supervise one model, which outperforms prior work on a range of benchmarks, including the challenging 3D Poses in the Wild (3DPW) dataset. Our code and models are available for research purposes., Comment: Accepted at the 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV'23)

Reliably planning fingertip grasps for multi-fingered hands lies as a key challenge for many tasks including tool use, insertion, and dexterous in-hand manipulation. This task becomes even more difficult when the robot lacks an accurate model of the object to be grasped. Tactile sensing offers a promising approach to account for uncertainties in object shape. However, current robotic hands tend to lack full tactile coverage. As such, a problem arises of how to plan and execute grasps for multi-fingered hands such that contact is made with the area covered by the tactile sensors. To address this issue, we propose an approach to grasp planning that explicitly reasons about where the fingertips should contact the estimated object surface while maximizing the probability of grasp success. Key to our method's success is the use of visual surface estimation for initial planning to encode the contact constraint. The robot then executes this plan using a tactile-feedback controller that enables the robot to adapt to online estimates of the object's surface to correct for errors in the initial plan. Importantly, the robot never explicitly integrates object pose or surface estimates between visual and tactile sensing, instead it uses the two modalities in complementary ways. Vision guides the robots motion prior to contact; touch updates the plan when contact occurs differently than predicted from vision. We show that our method successfully synthesises and executes precision grasps for previously unseen objects using surface estimates from a single camera view. Further, our approach outperforms a state of the art multi-fingered grasp planner, while also beating several baselines we propose.

Residential demand response programs aim to activate demand flexibility at the household level. In recent years, reinforcement learning (RL) has gained significant attention for these type of applications. A major challenge of RL algorithms is data efficiency. New RL algorithms, such as proximal policy optimisation (PPO), have tried to increase data efficiency. Additionally, combining RL with transfer learning has been proposed in an effort to mitigate this challenge. In this work, we further improve upon state-of-the-art transfer learning performance by incorporating demand response domain knowledge into the learning pipeline. We evaluate our approach on a demand response use case where peak shaving and self-consumption is incentivised by means of a capacity tariff. We show our adapted version of PPO, combined with transfer learning, reduces cost by 14.51% compared to a regular hysteresis controller and by 6.68% compared to traditional PPO., Comment: Submitted to Elsevier Energy and AI

Robots operating in human environments must be able to rearrange objects into semantically-meaningful configurations, even if these objects are previously unseen. In this work, we focus on the problem of building physically-valid structures without step-by-step instructions. We propose StructDiffusion, which combines a diffusion model and an object-centric transformer to construct structures given partial-view point clouds and high-level language goals, such as "set the table". Our method can perform multiple challenging language-conditioned multi-step 3D planning tasks using one model. StructDiffusion even improves the success rate of assembling physically-valid structures out of unseen objects by on average 16% over an existing multi-modal transformer model trained on specific structures. We show experiments on held-out objects in both simulation and on real-world rearrangement tasks. Importantly, we show how integrating both a diffusion model and a collision-discriminator model allows for improved generalization over other methods when rearranging previously-unseen objects. For videos and additional results, see our website: https://structdiffusion.github.io/., Comment: Accepted to Robotics: Science and Systems (RSS) 2023. The previous version appeared in CoRL Workshop on Language and Robot Learning 2022

Full 3D inversion of time-domain Airborne ElectroMagnetic (AEM) data requires specialists' expertise and a tremendous amount of computational resources, not readily available to everyone. Consequently, quasi-2D/3D inversion methods are prevailing, using a much faster but approximate (1D) forward model. We propose an appraisal tool that indicates zones in the inversion model that are not in agreement with the multidimensional data and therefore, should not be interpreted quantitatively. The image appraisal relies on multidimensional forward modelling to compute a so-called normalized gradient. Large values in that gradient indicate model parameters that do not fit the true multidimensionality of the observed data well and should not be interpreted quantitatively. An alternative approach is proposed to account for imperfect forward modelling, such that the appraisal tool is computationally inexpensive. The method is demonstrated on an AEM survey in a salinization context, revealing possible problematic zones in the estimated fresh-saltwater interface.

Modern 3D semantic instance segmentation approaches predominantly rely on specialized voting mechanisms followed by carefully designed geometric clustering techniques. Building on the successes of recent Transformer-based methods for object detection and image segmentation, we propose the first Transformer-based approach for 3D semantic instance segmentation. We show that we can leverage generic Transformer building blocks to directly predict instance masks from 3D point clouds. In our model called Mask3D each object instance is represented as an instance query. Using Transformer decoders, the instance queries are learned by iteratively attending to point cloud features at multiple scales. Combined with point features, the instance queries directly yield all instance masks in parallel. Mask3D has several advantages over current state-of-the-art approaches, since it neither relies on (1) voting schemes which require hand-selected geometric properties (such as centers) nor (2) geometric grouping mechanisms requiring manually-tuned hyper-parameters (e.g. radii) and (3) enables a loss that directly optimizes instance masks. Mask3D sets a new state-of-the-art on ScanNet test (+6.2 mAP), S3DIS 6-fold (+10.1 mAP), STPLS3D (+11.2 mAP) and ScanNet200 test (+12.4 mAP)., Comment: ICRA 2023 camera-ready version

Advances in laser technology have driven discoveries in atomic, molecular, and optical (AMO) physics and emerging applications, from quantum computers with cold atoms or ions, to quantum networks with solid-state color centers. This progress is motivating the development of a new generation of "programmable optical control" systems, characterized by criteria (C1) visible (VIS) and near-infrared (IR) wavelength operation, (C2) large channel counts extensible beyond 1000s of individually addressable atoms, (C3) high intensity modulation extinction and (C4) repeatability compatible with low gate errors, and (C5) fast switching times. Here, we address these challenges by introducing an atom control architecture based on VIS-IR photonic integrated circuit (PIC) technology. Based on a complementary metal-oxide-semiconductor (CMOS) fabrication process, this Atom-control PIC (APIC) technology meets the system requirements (C1)-(C5). As a proof of concept, we demonstrate a 16-channel silicon nitride based APIC with (5.8$\pm$0.4) ns response times and -30 dB extinction ratio at a wavelength of 780 nm. This work demonstrates the suitability of PIC technology for quantum control, opening a path towards scalable quantum information processing based on optically-programmable atomic systems.