Search

Egocentric vision aims to capture and analyse the world from the first-person perspective. We explore the possibilities for egocentric wearable devices to improve and enhance industrial use cases w.r.t. data collection, annotation, labelling and downstream applications. This would contribute to easier data collection and allow users to provide additional context. We envision that this approach could serve as a supplement to the traditional industrial Machine Vision workflow. Code, Dataset and related resources will be available at: https://github.com/Vivek9Chavan/EgoVis24, Comment: To be presented at the First Joint Egocentric Vision (EgoVis) Workshop, held in conjunction with CVPR 2024

Vision-language models, while effective in general domains and showing strong performance in diverse multi-modal applications like visual question-answering (VQA), struggle to maintain the same level of effectiveness in more specialized domains, e.g., medical. We propose a medical vision-language model that integrates large vision and language models adapted for the medical domain. This model goes through three stages of parameter-efficient training using three separate biomedical and radiology multi-modal visual and text datasets. The proposed model achieves state-of-the-art performance on the SLAKE 1.0 medical VQA (MedVQA) dataset with an overall accuracy of 87.5% and demonstrates strong performance on another MedVQA dataset, VQA-RAD, achieving an overall accuracy of 73.2%., Comment: Clinical NLP @ NAACL 2024

We present an iterative optimal control method of quantum systems, aimed at an implementation of a desired operation with optimal fidelity. The update step of the method is based on the linear response of the fidelity to the control operators, and its projection onto the mode functions of the corresponding operator. Our method extends methods such as gradient ascent pulse engineering and variational quantum algorithms, by determining the fidelity gradient in a hyperparameter-free manner, and using it for a multi-parameter update, capitalizing on the multi-mode overlap of the perturbation and the mode functions. This directly reduces the number of dynamical trajectories that need to be evaluated in order to update a set of parameters. We demonstrate this approach, and compare it to the standard GRAPE algorithm, for the example of a quantum gate on two qubits, demonstrating a clear improvement in convergence and optimal fidelity of the generated protocol., Comment: 11 pages, 10 figures

The last decade has shown the great potential that X-ray Free Electron Lasers have to study High Energy Density matter. Experiments at FELs have made significant breakthroughs in Shock Physics and Dynamic Diffraction, Dense Plasma Physics and Warm Dense Matter Science, using techniques such as isochoric heating, inelastic scattering, small angle scattering and diffraction. In addition, and complementary to these techniques, the coherent properties of the beam can be used to image HED samples with high precision. We present new imaging diagnostics and techniques developed at the MEC instrument at LCLS over the last few years. We briefly show result of a previously used Phase Contrast Imaging setup, where the X-ray beam propagates from the target to a camera some distance away revealing its phase, as well as from a new direct imaging approach where the target is re-imaged on the camera with 300nm resolution. Last, we show a new Talbot Phase Imaging method that allows for both measuring the phase and intensity changes of the x-rays introduced by a target with sub-micron resolution.

The $k$-Opt algorithm is a local search algorithm for the Traveling Salesman Problem. Starting with an initial tour, it iteratively replaces at most $k$ edges in the tour with the same number of edges to obtain a better tour. Krentel (FOCS 1989) showed that the Traveling Salesman Problem with the $k$-Opt neighborhood is complete for the class PLS (polynomial time local search) and that the $k$-Opt algorithm can have exponential running time for any pivot rule. However, his proof requires $k \gg 1000$ and has a substantial gap. We show the two properties above for a much smaller value of $k$, addressing an open question by Monien, Dumrauf, and Tscheuschner (ICALP 2010). In particular, we prove the PLS-completeness for $k \geq 17$ and the exponential running time for $k \geq 5$., Comment: Appeared in ICALP 2024

While graph neural networks (GNNs) are widely used for node and graph representation learning tasks, the reliability of GNN uncertainty estimates under distribution shifts remains relatively under-explored. Indeed, while post-hoc calibration strategies can be used to improve in-distribution calibration, they need not also improve calibration under distribution shift. However, techniques which produce GNNs with better intrinsic uncertainty estimates are particularly valuable, as they can always be combined with post-hoc strategies later. Therefore, in this work, we propose G-$\Delta$UQ, a novel training framework designed to improve intrinsic GNN uncertainty estimates. Our framework adapts the principle of stochastic data centering to graph data through novel graph anchoring strategies, and is able to support partially stochastic GNNs. While, the prevalent wisdom is that fully stochastic networks are necessary to obtain reliable estimates, we find that the functional diversity induced by our anchoring strategies when sampling hypotheses renders this unnecessary and allows us to support G-$\Delta$UQ on pretrained models. Indeed, through extensive evaluation under covariate, concept and graph size shifts, we show that G-$\Delta$UQ leads to better calibrated GNNs for node and graph classification. Further, it also improves performance on the uncertainty-based tasks of out-of-distribution detection and generalization gap estimation. Overall, our work provides insights into uncertainty estimation for GNNs, and demonstrates the utility of G-$\Delta$UQ in obtaining reliable estimates., Comment: 33 pages; 10 Figures. arXiv admin note: text overlap with arXiv:2309.10976

In [Heimann, Lehrenfeld, Preu{\ss}, SIAM J. Sci. Comp. 45(2), 2023, B139 - B165] new geometrically unfitted space-time Finite Element methods for partial differential equations posed on moving domains of higher-order accuracy in space and time have been introduced. For geometrically higher-order accuracy a parametric mapping on a background space-time tensor-product mesh has been used. In this paper, we concentrate on the geometrical accuracy of the approximation and derive rigorous bounds for the distance between the realized and an ideal mapping in different norms and derive results for the space-time regularity of the parametric mapping. These results are important and lay the ground for the error analysis of corresponding unfitted space-time finite element methods.

Safe deployment of graph neural networks (GNNs) under distribution shift requires models to provide accurate confidence indicators (CI). However, while it is well-known in computer vision that CI quality diminishes under distribution shift, this behavior remains understudied for GNNs. Hence, we begin with a case study on CI calibration under controlled structural and feature distribution shifts and demonstrate that increased expressivity or model size do not always lead to improved CI performance. Consequently, we instead advocate for the use of epistemic uncertainty quantification (UQ) methods to modulate CIs. To this end, we propose G-$\Delta$UQ, a new single model UQ method that extends the recently proposed stochastic centering framework to support structured data and partial stochasticity. Evaluated across covariate, concept, and graph size shifts, G-$\Delta$UQ not only outperforms several popular UQ methods in obtaining calibrated CIs, but also outperforms alternatives when CIs are used for generalization gap prediction or OOD detection. Overall, our work not only introduces a new, flexible GNN UQ method, but also provides novel insights into GNN CIs on safety-critical tasks., Comment: 22 pages, 11 figures

X-ray laser-induced structural changes in silicon undergoing femtosecond melting have been investigated by using an x-ray pump-x-ray probe technique. The experimental results for different initial sample temperatures reveal that the onset time and the speed of the atomic disordering are independent of the initial temperature, suggesting that equilibrium atomic motion in the initial state does not play a pivotal role in the x-ray-induced ultrafast melting. By comparing the observed time-dependence of the atomic disordering and the dedicated theoretical simulations, we interpret that the energy transfer from the excited electrons to ions via electron-ion coupling (thermal effect) as well as a strong modification of the interatomic potential due to electron excitations (non-thermal effect) trigger the ultrafast atomic disordering. Our finding of the interplay of thermal and non-thermal effects in the x-ray-induced melting demonstrates that accurate modeling of intense x-ray interactions with matter is essential to ensure a correct interpretation of experiments using intense x-ray laser pulses.

The process of annotating histological gigapixel-sized whole slide images (WSIs) at the pixel level for the purpose of training a supervised segmentation model is time-consuming. Region-based active learning (AL) involves training the model on a limited number of annotated image regions instead of requesting annotations of the entire images. These annotation regions are iteratively selected, with the goal of optimizing model performance while minimizing the annotated area. The standard method for region selection evaluates the informativeness of all square regions of a specified size and then selects a specific quantity of the most informative regions. We find that the efficiency of this method highly depends on the choice of AL step size (i.e., the combination of region size and the number of selected regions per WSI), and a suboptimal AL step size can result in redundant annotation requests or inflated computation costs. This paper introduces a novel technique for selecting annotation regions adaptively, mitigating the reliance on this AL hyperparameter. Specifically, we dynamically determine each region by first identifying an informative area and then detecting its optimal bounding box, as opposed to selecting regions of a uniform predefined shape and size as in the standard method. We evaluate our method using the task of breast cancer metastases segmentation on the public CAMELYON16 dataset and show that it consistently achieves higher sampling efficiency than the standard method across various AL step sizes. With only 2.6\% of tissue area annotated, we achieve full annotation performance and thereby substantially reduce the costs of annotating a WSI dataset. The source code is available at https://github.com/DeepMicroscopy/AdaptiveRegionSelection.

The conversion between different spherical image and video projection formats requires highly accurate resampling techniques in order to minimize the inevitable loss of information. Suitable resampling algorithms such as nearest neighbor, linear or cubic resampling are readily available. However, no generally applicable resampling technique exploits the special properties of spherical images so far. Thus, we propose a novel viewport-adaptive resampling (VAR) technique that takes the spherical characteristics of the underlying resampling problem into account. VAR can be applied to any mesh-to-mesh capable resampling algorithm and shows significant gains across all tested techniques. In combination with frequency-selective resampling, VAR outperforms conventional cubic resampling by more than 2 dB in terms of WS-PSNR. A visual inspection and the evaluation of further metrics such as PSNR and SSIM support the positive results., Comment: 5 pages, 3 figures, 2 tables, accepted for IEEE International Conference on Image Processing 2023 (IEEE ICIP 2023)

We demonstrate machine learning assisted design of a two-qubit gate in a Rydberg tweezer system. Two low-energy hyperfine states in each of the atoms represent the logical qubit and a Rydberg state acts as an auxiliary state to induce qubit interaction. Utilizing a hybrid quantum-classical optimizer, we generate optimal pulse sequences that implement a CNOT gate with high fidelity, for experimentally realistic parameters and protocols, as well as realistic limitations. We show that local control of single qubit operations is sufficient for performing quantum computation on a large array of atoms. We generate optimized strategies that are robust for both the strong-coupling, blockade regime of the Rydberg states, but also for the weak-coupling limit. Thus, we show that Rydberg-based quantum information processing in the weak-coupling limit is a desirable approach, being robust and optimal, with current technology., Comment: 11 pages, 5 figures

Graph Neural Network (GNN) research has highlighted a relationship between high homophily (i.e., the tendency of nodes of the same class to connect) and strong predictive performance in node classification. However, recent work has found the relationship to be more nuanced, demonstrating that simple GNNs can learn in certain heterophilous settings. To resolve these conflicting findings and align closer to real-world datasets, we go beyond the assumption of a global graph homophily level and study the performance of GNNs when the local homophily level of a node deviates from the global homophily level. Through theoretical and empirical analysis, we systematically demonstrate how shifts in local homophily can introduce performance degradation, leading to performance discrepancies across local homophily levels. We ground the practical implications of this work through granular analysis on five real-world datasets with varying global homophily levels, demonstrating that (a) GNNs can fail to generalize to test nodes that deviate from the global homophily of a graph, and (b) high local homophily does not necessarily confer high performance for a node. We further show that GNNs designed for globally heterophilous graphs can alleviate performance discrepancy by improving performance across local homophily levels, offering a new perspective on how these GNNs achieve stronger global performance., Comment: 30 pages

With the increasing demand of capturing our environment in three-dimensions for AR/ VR applications and autonomous driving among others, the importance of high-resolution point clouds rises. As the capturing process is a complex task, point cloud upsampling is often desired. We propose Frequency-Selective Upsampling (FSU), an upsampling scheme that upsamples geometry and attribute information of point clouds jointly in a sequential manner with overlapped support areas. The point cloud is partitioned into blocks with overlapping support area first. Then, a continuous frequency model is generated that estimates the point cloud's surface locally. The model is sampled at new positions for upsampling. In a subsequent step, another frequency model is created that models the attribute signal. Here, knowledge from the geometry upsampling is exploited for a simplified projection of the points in two dimensions. The attribute model is evaluated for the upsampled geometry positions. In our extensive evaluation, we evaluate geometry and attribute upsampling independently and show joint results. The geometry results show best performances for our proposed FSU in terms of point-to-plane error and plane-to-plane angular similarity. Moreover, FSU outperforms other color upsampling schemes by 1.9 dB in terms of color PSNR. In addition, the visual appearance of the point clouds clearly increases with FSU., Comment: 10 pages, 10 figures, Under Review at IEEE TMM Special Issue on Point Cloud Processing and Understanding

The seismically excited motion of high-Q pendula in gravitational-wave observatories sets a sensitivity limit to sub-audio gravitational-wave frequencies. Here, we report on the use of machine learning to predict the motion of a high-Q pendulum with a resonance frequency of 1.4Hz that is driven by natural seismic activity. We achieve a reduction of the displacement power spectral density of 40dB at the resonant frequency 1.4Hz and 6dB at 11Hz. Our result suggests that machine learning is able to significantly reduce seismically induced test mass motion in gravitational-wave detectors in combination with corrective feed-forward techniques., Comment: 10 pages, 7 figures

We propose a new geometrically unfitted finite element method based on discontinuous Trefftz ansatz spaces. Trefftz methods allow for a reduction in the number of degrees of freedom in discontinuous Galerkin methods, thereby, the costs for solving arising linear systems significantly. This work shows that they are also an excellent way to reduce the number of degrees of freedom in an unfitted setting. We present a unified analysis of a class of geometrically unfitted discontinuous Galerkin methods with different stabilisation mechanisms to deal with small cuts between the geometry and the mesh. We cover stability and derive a-priori error bounds, including errors arising from geometry approximation for the class of discretisations for a model Poisson problem in a unified manner. The analysis covers Trefftz and full polynomial ansatz spaces, alike. Numerical examples validate the theoretical findings and demonstrate the potential of the approach.

Neural networks became the standard technique for image classification throughout the last years. They are extracting image features from a large number of images in a training phase. In a following test phase, the network is applied to the problem it was trained for and its performance is measured. In this paper, we focus on image classification. The amount of visual data that is interpreted by neural networks grows with the increasing usage of neural networks. Mostly, the visual data is transmitted from the application side to a central server where the interpretation is conducted. If the transmission is disturbed, losses occur in the transmitted images. These losses have to be reconstructed using postprocessing. In this paper, we incorporate the widely applied bilinear and bicubic interpolation and the high-quality reconstruction Frequency-Selective Reconstruction (FSR) for the reconstruction of corrupted images. However, we propose to use Frequency-Selective Mesh-to-Grid Resampling (FSMR) for the joint reconstruction and resizing of corrupted images. The performance in terms of classification accuracy of EfficientNetB0, DenseNet121, DenseNet201, ResNet50 and ResNet152 is examined. Results show that the reconstruction with FSMR leads to the highest classification accuracy for most networks. Average improvements of up to 6.7 percentage points are possible for DenseNet121., Comment: IEEE 24th International Workshop on Multimedia Signal Processing 2022

High frame rates are desired in many fields of application. As in many cases the frame repetition rate of an already captured video has to be increased, frame rate up-conversion (FRUC) is of high interest. We conduct a motion compensated approach. From two neighboring frames, the motion is estimated and the neighboring pixels are shifted along the motion vector into the frame to be reconstructed. For displaying, these irregularly distributed mesh pixels have to be resampled onto regularly spaced grid positions. We use the model-based key point agnostic frequency-selective mesh-to-grid resampling (AFSMR) for this task and show that AFSMR works best for applications that contain irregular meshes with varying densities. AFSMR gains up to 3.2 dB in contrast to the already high performing frequency-selective mesh-to-grid resampling (FSMR). Additionally, AFSMR increases the run time by a factor of 11 relative to FSMR., Comment: 5 pages, 4 figures, IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2021