Search

Improved surgical skill is generally associated with improved patient outcomes, although assessment is subjective; labour-intensive; and requires domain specific expertise. Automated data driven metrics can alleviate these difficulties, as demonstrated by existing machine learning instrument tracking models in minimally invasive surgery. However, these models have been tested on limited datasets of laparoscopic surgery, with a focus on isolated tasks and robotic surgery. In this paper, a new public dataset is introduced, focusing on simulated surgery, using the nasal phase of endoscopic pituitary surgery as an exemplar. Simulated surgery allows for a realistic yet repeatable environment, meaning the insights gained from automated assessment can be used by novice surgeons to hone their skills on the simulator before moving to real surgery. PRINTNet (Pituitary Real-time INstrument Tracking Network) has been created as a baseline model for this automated assessment. Consisting of DeepLabV3 for classification and segmentation; StrongSORT for tracking; and the NVIDIA Holoscan SDK for real-time performance, PRINTNet achieved 71.9% Multiple Object Tracking Precision running at 22 Frames Per Second. Using this tracking output, a Multilayer Perceptron achieved 87% accuracy in predicting surgical skill level (novice or expert), with the "ratio of total procedure time to instrument visible time" correlated with higher surgical skill. This therefore demonstrates the feasibility of automated surgical skill assessment in simulated endoscopic pituitary surgery. The new publicly available dataset can be found here: https://doi.org/10.5522/04/26511049., Comment: 7 pages, 6 figures

Regression is a fundamental prediction task common in data-centric engineering applications that involves learning mappings between continuous variables. In many engineering applications (e.g.\ structural health monitoring), feature-label pairs used to learn such mappings are of limited availability which hinders the effectiveness of traditional supervised machine learning approaches. The current paper proposes a methodology for overcoming the issue of data scarcity by combining active learning with hierarchical Bayesian modelling. Active learning is an approach for preferentially acquiring feature-label pairs in a resource-efficient manner. In particular, the current work adopts a risk-informed approach that leverages contextual information associated with regression-based engineering decision-making tasks (e.g.\ inspection and maintenance). Hierarchical Bayesian modelling allow multiple related regression tasks to be learned over a population, capturing local and global effects. The information sharing facilitated by this modelling approach means that information acquired for one engineering system can improve predictive performance across the population. The proposed methodology is demonstrated using an experimental case study. Specifically, multiple regressions are performed over a population of machining tools, where the quantity of interest is the surface roughness of the workpieces. An inspection and maintenance decision process is defined using these regression tasks which is in turn used to construct the active-learning algorithm. The novel methodology proposed is benchmarked against an uninformed approach to label acquisition and independent modelling of the regression tasks. It is shown that the proposed approach has superior performance in terms of expected cost -- maintaining predictive performance while reducing the number of inspections required., Comment: 19 pages, 12 figures, 3 tables, submitted to Data-Centric Engineering

Robotic-assisted surgery (RAS) relies on accurate depth estimation for 3D reconstruction and visualization. While foundation models like Depth Anything Models (DAM) show promise, directly applying them to surgery often yields suboptimal results. Fully fine-tuning on limited surgical data can cause overfitting and catastrophic forgetting, compromising model robustness and generalization. Although Low-Rank Adaptation (LoRA) addresses some adaptation issues, its uniform parameter distribution neglects the inherent feature hierarchy, where earlier layers, learning more general features, require more parameters than later ones. To tackle this issue, we introduce Depth Anything in Robotic Endoscopic Surgery (DARES), a novel approach that employs a new adaptation technique, Vector Low-Rank Adaptation (Vector-LoRA) on the DAM V2 to perform self-supervised monocular depth estimation in RAS scenes. To enhance learning efficiency, we introduce Vector-LoRA by integrating more parameters in earlier layers and gradually decreasing parameters in later layers. We also design a reprojection loss based on the multi-scale SSIM error to enhance depth perception by better tailoring the foundation model to the specific requirements of the surgical environment. The proposed method is validated on the SCARED dataset and demonstrates superior performance over recent state-of-the-art self-supervised monocular depth estimation techniques, achieving an improvement of 13.3% in the absolute relative error metric. The code and pre-trained weights are available at https://github.com/mobarakol/DARES., Comment: 11 pages

Studies show that the model-independent, fully non-perturbative covariant cosmographic approach is suitable for analyzing the local Universe $(z\lesssim 0.1)$. However, accurately characterizing large and inhomogeneous mass distributions requires the fourth-order term in the redshift expansion of the covariant luminosity distance $d_L(z,\boldsymbol{n})$. We calculate the covariant snap parameter $\mathbb{S}$ and its spherical harmonic multipole moments using the matter expansion tensor and the evolution equations for lightray bundles. The fourth-order term adds 36 degrees of freedom, since the highest independent multipole of the snap is the 32-pole (dotriacontapole) $(\ell=5)$. Including this term helps to de-bias estimations of the covariant deceleration parameter. Given that observations suggest axially symmetric anisotropies in the Hubble diagram for $z \lesssim 0.1$ and theory shows that only a subset of multipoles contributes to the signal, we demonstrate that only 12 degrees of freedom are needed for a model-independent description of the local universe. We use an analytical axisymmetric model of the local Universe, with data that matches the Zwicky Transient Facility survey, in order to provide a numerical example of the amplitude of the snap multipoles and to forecast precision., Comment: 13 + 7 pages

We present HUP-3D, a 3D multi-view multi-modal synthetic dataset for hand-ultrasound (US) probe pose estimation in the context of obstetric ultrasound. Egocentric markerless 3D joint pose estimation has potential applications in mixed reality based medical education. The ability to understand hand and probe movements programmatically opens the door to tailored guidance and mentoring applications. Our dataset consists of over 31k sets of RGB, depth and segmentation mask frames, including pose related ground truth data, with a strong emphasis on image diversity and complexity. Adopting a camera viewpoint-based sphere concept allows us to capture a variety of views and generate multiple hand grasp poses using a pre-trained network. Additionally, our approach includes a software-based image rendering concept, enhancing diversity with various hand and arm textures, lighting conditions, and background images. Furthermore, we validated our proposed dataset with state-of-the-art learning models and we obtained the lowest hand-object keypoint errors. The dataset and other details are provided with the supplementary material. The source code of our grasp generation and rendering pipeline will be made publicly available., Comment: https://conferences.miccai.org/2024/en/

Conformal prediction is a non-parametric technique for constructing prediction intervals or sets from arbitrary predictive models under the assumption that the data is exchangeable. It is popular as it comes with theoretical guarantees on the marginal coverage of the prediction sets and the split conformal prediction variant has a very low computational cost compared to model training. We study the robustness of split conformal prediction in a data contamination setting, where we assume a small fraction of the calibration scores are drawn from a different distribution than the bulk. We quantify the impact of the corrupted data on the coverage and efficiency of the constructed sets when evaluated on "clean" test points, and verify our results with numerical experiments. Moreover, we propose an adjustment in the classification setting which we call Contamination Robust Conformal Prediction, and verify the efficacy of our approach using both synthetic and real datasets.

Reconstructing 2D freehand Ultrasound (US) frames into 3D space without using a tracker has recently seen advances with deep learning. Predicting good frame-to-frame rigid transformations is often accepted as the learning objective, especially when the ground-truth labels from spatial tracking devices are inherently rigid transformations. Motivated by a) the observed nonrigid deformation due to soft tissue motion during scanning, and b) the highly sensitive prediction of rigid transformation, this study investigates the methods and their benefits in predicting nonrigid transformations for reconstructing 3D US. We propose a novel co-optimisation algorithm for simultaneously estimating rigid transformations among US frames, supervised by ground-truth from a tracker, and a nonrigid deformation, optimised by a regularised registration network. We show that these two objectives can be either optimised using meta-learning or combined by weighting. A fast scattered data interpolation is also developed for enabling frequent reconstruction and registration of non-parallel US frames, during training. With a new data set containing over 357,000 frames in 720 scans, acquired from 60 subjects, the experiments demonstrate that, due to an expanded thus easier-to-optimise solution space, the generalisation is improved with the added deformation estimation, with respect to the rigid ground-truth. The global pixel reconstruction error (assessing accumulative prediction) is lowered from 18.48 to 16.51 mm, compared with baseline rigid-transformation-predicting methods. Using manually identified landmarks, the proposed co-optimisation also shows potentials in compensating nonrigid tissue motion at inference, which is not measurable by tracker-provided ground-truth. The code and data used in this paper are made publicly available at https://github.com/QiLi111/NR-Rec-FUS., Comment: Accepted at MICCAI 2024

Upcoming galaxy surveys aim to map the Universe with unprecedented precision, depth and sky coverage. The galaxy bispectrum is a prime source of information as it allows us to probe primordial non-Gaussianity (PNG), a key factor in differentiating various models of inflation. On the scales where local PNG is strongest, Doppler and other relativistic effects become important and need to be included. We investigate the detectability of relativistic and local PNG contributions in the galaxy bispectrum. We compute the signal-to-noise ratio for the detection of the bispectrum including such effects. Furthermore, we perform information matrix forecasts on the local PNG parameter $f_{\rm NL}$ and on the parametrised amplitudes of the relativistic corrections. Finally, we quantify the bias on the measurement of $f_{\rm NL}$ that arises from neglecting relativistic effects. Our results show that detections of both first- and second-order relativistic effects are promising with forthcoming spectroscopic survey specifications -- and are largely unaffected by the uncertainty in $f_{\rm NL}$. Conversely, we show for the first time that neglecting relativistic corrections in the galaxy bispectrum can lead to $>\!1.5\sigma(f_{\rm NL})$ shift on the detected value of $f_{\rm NL}$, highlighting the importance of including relativistic effects in our modelling., Comment: 28 pages, 8 figures, comments are welcome

The galaxy bispectrum provides access to correlations among different scales that cannot be captured by the power spectrum alone, and with the Stage-IV galaxy surveys it enables the possibility of detecting both primordial non-Gaussianity (PNG) and general relativistic effects. Accounting for wide-separation corrections, which arise from the loss of symmetry in the correlation of widely separated points on the past light cone, is essential for their accurate modelling and detection. These corrections can be included perturbatively to the standard bispectrum and we compute them analytically for a generalised line of sight. We include the radial contribution to the wide-separations corrections for the first time. We show that the first-order corrections entering the odd multipoles with respect to the line of sight are large, up to $10 \%$ of the bispectrum monopole, and need to be included when considering the leading-order relativistic effects that could be detectable with surveys like DESI and Euclid. The second-order wide-separation and relativistic contributions, including their mixing terms, have implications for analysis of PNG and we show, for the local type, they can mimic $f_{\rm NL}$ of order 10 in the squeezed limit. We present full analytic expressions for all these contributions to the local bispectrum and its multipoles with a publicly available code for their computation., Comment: 36 pages, 17 figures. Prepared for submission to JCAP

Upcoming large-scale structure surveys will be able to measure new features in the galaxy two point correlation function. Relativistic effects appear on large scales as subtle corrections to redshift-space distortions, showing up as a dipole and octupole when cross-correlating two different tracers of dark matter. The dipole and octupole are very sensitive to the evolution and magnification biases of the observed tracers which are hard to model accurately as they depend upon the derivative of the luminosity function at the flux limit of the survey. We show that splitting a galaxy population into bright and faint samples allows us to cross-correlate these and constrain both the evolution bias and magnification bias of the two samples -- using the relativistic odd multipoles of the correlation function, together with the even Newtonian multipoles. Although the octupole has much lower signal-to-noise than the dipole, it significantly improves the constraints by breaking parameter degeneracies. We illustrate this in the case of a futuristic survey with the Square Kilometre Array, and demonstrate how splitting the samples in different ways can help improve constraints. This method is quite general and can be used on different types of tracers to improve knowledge of their luminosity functions. Furthermore, the signal-to-noise of the dipole and octupole peaks on intermediate scales, which means that that they can deliver a clean measurement of the magnification bias and evolution bias without contamination from local primordial non-Gaussianities or from systematics on very large scales., Comment: 34 pages, 10 figures, 7 tables

Galaxy surveys contain information on the largest scales via wide-angle and relativistic contributions. By combining two different galaxy populations, we can suppress the strong cosmic variance on ultra-large scales and thus enhance the detectability of the signals. The relativistic Doppler and Sachs-Wolfe effects are of a similar magnitude to the leading wide-angle corrections, so that it is important to treat them together, especially since they can partially cancel. The power spectra depend on the choice of line of sight for each galaxy pair and we present results for a general line of sight. Then we estimate the detection significance of the auto- and cross-power spectra for a variety of cases. We use two futuristic galaxy samples based on a `beyond-DESI' survey and a SKA Phase 2 survey, covering 15,000\,deg$^2$ up to $z=1$. We find a detection significance for the total relativistic wide-angle effects that ranges from $\sim 5\sigma$ to $>15\sigma$, depending on the line-of-sight configuration., Comment: 23 pages, 9 figures, 'version accepted by JCAP'

A major goal of cosmology is to understand the nature of the field(s) which drove primordial Inflation. Through future observations, the statistics of large-scale structure will allow us to probe primordial non-Gaussianity of the curvature perturbation at the end of Inflation. We show how a new correlation statistic can significantly improve these constraints over conventional methods. Next-generation radio telescope arrays are under construction which will map the density field of neutral hydrogen to high redshifts. These telescopes can operate as an interferometer, able to probe small scales, or as a collection of single dishes, combining signals to map the large scales. We show how to fuse these these operating modes in order to measure the squeezed bispectrum with higher precision and greater economy. This leads to constraints on primordial non-Gaussianity that will improve on measurements by Planck, and out-perform other surveys such as Euclid. We forecast that $\sigma(f_{\mathrm{NL}}^{\mathrm{loc}})\sim 3$, achieved by using a small subset, $\mathcal{O}(10^2-10^3)$, of the total number of accessible triangles. The proposed method can be used for any signal that peaks in squeezed configurations., Comment: 13 pages, 6 figures

Weakly-supervised segmentation (WSS) methods, reliant on image-level labels indicating object presence, lack explicit correspondence between labels and regions of interest (ROIs), posing a significant challenge. Despite this, WSS methods have attracted attention due to their much lower annotation costs compared to fully-supervised segmentation. Leveraging reinforcement learning (RL) self-play, we propose a novel WSS method that gamifies image segmentation of a ROI. We formulate segmentation as a competition between two agents that compete to select ROI-containing patches until exhaustion of all such patches. The score at each time-step, used to compute the reward for agent training, represents likelihood of object presence within the selection, determined by an object presence detector pre-trained using only image-level binary classification labels of object presence. Additionally, we propose a game termination condition that can be called by either side upon exhaustion of all ROI-containing patches, followed by the selection of a final patch from each. Upon termination, the agent is incentivised if ROI-containing patches are exhausted or disincentivised if an ROI-containing patch is found by the competitor. This competitive setup ensures minimisation of over- or under-segmentation, a common problem with WSS methods. Extensive experimentation across four datasets demonstrates significant performance improvements over recent state-of-the-art methods. Code: https://github.com/s-sd/spurl/tree/main/wss

Visual Question Answering (VQA) within the surgical domain, utilizing Large Language Models (LLMs), offers a distinct opportunity to improve intra-operative decision-making and facilitate intuitive surgeon-AI interaction. However, the development of LLMs for surgical VQA is hindered by the scarcity of diverse and extensive datasets with complex reasoning tasks. Moreover, contextual fusion of the image and text modalities remains an open research challenge due to the inherent differences between these two types of information and the complexity involved in aligning them. This paper introduces PitVQA, a novel dataset specifically designed for VQA in endonasal pituitary surgery and PitVQA-Net, an adaptation of the GPT2 with a novel image-grounded text embedding for surgical VQA. PitVQA comprises 25 procedural videos and a rich collection of question-answer pairs spanning crucial surgical aspects such as phase and step recognition, context understanding, tool detection and localization, and tool-tissue interactions. PitVQA-Net consists of a novel image-grounded text embedding that projects image and text features into a shared embedding space and GPT2 Backbone with an excitation block classification head to generate contextually relevant answers within the complex domain of endonasal pituitary surgery. Our image-grounded text embedding leverages joint embedding, cross-attention and contextual representation to understand the contextual relationship between questions and surgical images. We demonstrate the effectiveness of PitVQA-Net on both the PitVQA and the publicly available EndoVis18-VQA dataset, achieving improvements in balanced accuracy of 8% and 9% over the most recent baselines, respectively. Our code and dataset is available at https://github.com/mobarakol/PitVQA., Comment: 10 pages, 3 figures

The advances in deep generative models have greatly accelerate the process of video procession such as video enhancement and synthesis. Learning spatio-temporal video models requires to capture the temporal dynamics of a scene, in addition to the visual appearance of individual frames. Illumination consistency, which reflects the variations of illumination in the dynamic video sequences, play a vital role in video processing. Unfortunately, to date, no well-accepted quantitative metric has been proposed for video illumination consistency evaluation. In this paper, we propose a illumination histogram consistency (IHC) metric to quantitatively and automatically evaluate the illumination consistency of the video sequences. IHC measures the illumination variation of any video sequence based on the illumination histogram discrepancies across all the frames in the video sequence. Specifically, given a video sequence, we first estimate the illumination map of each individual frame using the Retinex model; Then, using the illumination maps, the mean illumination histogram of the video sequence is computed by the mean operation across all the frames; Next, we compute the illumination histogram discrepancy between each individual frame and the mean illumination histogram and sum up all the illumination histogram discrepancies to represent the illumination variations of the video sequence. Finally, we obtain the IHC score from the illumination histogram discrepancies via normalization and subtraction operations. Experiments are conducted to illustrate the performance of the proposed IHC metric and its capability to measure the illumination variations in video sequences. The source code is available on \url{https://github.com/LongChenCV/IHC-Metric}.

Purpose: The recent Segment Anything Model (SAM) has demonstrated impressive performance with point, text or bounding box prompts, in various applications. However, in safety-critical surgical tasks, prompting is not possible due to (i) the lack of per-frame prompts for supervised learning, (ii) it is unrealistic to prompt frame-by-frame in a real-time tracking application, and (iii) it is expensive to annotate prompts for offline applications. Methods: We develop Surgical-DeSAM to generate automatic bounding box prompts for decoupling SAM to obtain instrument segmentation in real-time robotic surgery. We utilise a commonly used detection architecture, DETR, and fine-tuned it to obtain bounding box prompt for the instruments. We then empolyed decoupling SAM (DeSAM) by replacing the image encoder with DETR encoder and fine-tune prompt encoder and mask decoder to obtain instance segmentation for the surgical instruments. To improve detection performance, we adopted the Swin-transformer to better feature representation. Results: The proposed method has been validated on two publicly available datasets from the MICCAI surgical instruments segmentation challenge EndoVis 2017 and 2018. The performance of our method is also compared with SOTA instrument segmentation methods and demonstrated significant improvements with dice metrics of 89.62 and 90.70 for the EndoVis 2017 and 2018. Conclusion: Our extensive experiments and validations demonstrate that Surgical-DeSAM enables real-time instrument segmentation without any additional prompting and outperforms other SOTA segmentation methods., Comment: 8 pages, 2 figures

Radio signals propagating via the solar corona and solar wind are significantly affected by compressive waves, impacting properties of solar bursts as well as sources viewed through the turbulent solar atmosphere. While static fluctuations scatter radio waves elastically, moving, turbulent or oscillating density irregularities act to broaden the frequency of the scattered waves. Using a new anisotropic density fluctuation model from the kinetic scattering theory for solar radio bursts, we deduce the plasma velocities required to explain observations of spacecraft signal frequency broadening. The inferred velocities are consistent with motions that are dominated by the solar wind at distances $\gtrsim 10$ $R_\odot$, but the levels of frequency broadening for $\lesssim 10$ $R_\odot$ require additional radial speeds $\sim (100-300)$ km s$^{-1}$ and/or transverse speeds $\sim (20-70)$ km s$^{-1}$. The inferred radial velocities also appear consistent with the sound or proton thermal speeds, while the speeds perpendicular to the radial direction are consistent with non-thermal motions measured via coronal Doppler-line broadening, interpreted as Alfv\'enic fluctuations. Landau damping of parallel propagating ion-sound (slow MHD) waves allow an estimate of the proton heating rate. The energy deposition rates due to ion-sound wave damping peak at a heliocentric distance of $\sim(1-3)$ $R_\odot$ are comparable to the rates available from a turbulent cascade of Alfv\'enic waves at large scales, suggesting a coherent picture of energy transfer, via the cascade or/and parametric decay of Alfv\'en waves to the small scales where heating takes place., Comment: 27 pages, 5 figures, submitted to The Astrophysical Journal

Recent measurements of the 4-point correlation function in large-scale galaxy surveys have found apparent evidence of parity violation in the distribution of galaxies. This cannot happen via dynamical gravitational effects in general relativity. If such a violation arose from physics in the early Universe it could indicate important new physics beyond the standard model, and would be at odds with most models of inflation. It is therefore now timely to consider the galaxy trispectrum in more detail. While the intrinsic 4-point correlation function, or equivalently the trispectrum, its Fourier counterpart, is parity invariant, the observed trispectrum must take redshift-space distortions into account. Although the standard Newtonian correction also respects parity invariance, we show that sub-leading relativistic corrections do not. We demonstrate that these can be significant at intermediate linear scales and are dominant over the Newtonian parity-invariant part around the equality scale and above. Therefore when observing the galaxy 4-point correlation function, we should expect to detect parity violation on large scales., Comment: 8 pages, 6 figures. v2 has minor errors corrected and additional material and figures. Version to appear in Physical Review Letters

In this paper we propose a reinforcement learning based weakly supervised system for localisation. We train a controller function to localise regions of interest within an image by introducing a novel reward definition that utilises non-binarised classification probability, generated by a pre-trained binary classifier which classifies object presence in images or image crops. The object-presence classifier may then inform the controller of its localisation quality by quantifying the likelihood of the image containing an object. Such an approach allows us to minimize any potential labelling or human bias propagated via human labelling for fully supervised localisation. We evaluate our proposed approach for a task of cancerous lesion localisation on a large dataset of real clinical bi-parametric MR images of the prostate. Comparisons to the commonly used multiple-instance learning weakly supervised localisation and to a fully supervised baseline show that our proposed method outperforms the multi-instance learning and performs comparably to fully-supervised learning, using only image-level classification labels for training., Comment: Accepted at ISBI 2024 (21st IEEE International Symposium on Biomedical Imaging)

Anisotropies in the distance-redshift relation of cosmological sources are expected due to large-scale inhomogeneities in the local Universe. When the observed sources are tracing a large-scale matter flow in a general spacetime geometry, the distance-redshift relation with its anisotropies can be described with a geometrical prediction that generalises the well-known Friedmann-Lema\^itre-Robertson-Walker result. Furthermore, it turns out that a finite set of multipole coefficients contain the full information about a finite-order truncation of the distance-redshift relation of a given observer. The multipoles of the distance-redshift relation are interesting new cosmological observables that have a direct physical interpretation in terms of kinematical quantities of the underlying matter flow. Using light cones extracted from $N$-body simulations we quantify the anisotropies expected in a $\Lambda$ cold dark matter cosmology by running a Markov chain Monte Carlo analysis on the observed data. In this observational approach the survey selection implements an implicit smoothing scale over which the effective rest frame of matter is fitted. The perceived anisotropy therefore depends significantly on the redshift range and distribution of sources. We find that the multipoles of the expansion rate, as well as the observer's velocity with respect to the large-scale matter flow, can be determined robustly with our approach., Comment: 17 pages, 11 figures; v2: minor revision, accepted version

For training registration networks, weak supervision from segmented corresponding regions-of-interest (ROIs) have been proven effective for (a) supplementing unsupervised methods, and (b) being used independently in registration tasks in which unsupervised losses are unavailable or ineffective. This correspondence-informing supervision entails cost in annotation that requires significant specialised effort. This paper describes a semi-weakly-supervised registration pipeline that improves the model performance, when only a small corresponding-ROI-labelled dataset is available, by exploiting unlabelled image pairs. We examine two types of augmentation methods by perturbation on network weights and image resampling, such that consistency-based unsupervised losses can be applied on unlabelled data. The novel WarpDDF and RegCut approaches are proposed to allow commutative perturbation between an image pair and the predicted spatial transformation (i.e. respective input and output of registration networks), distinct from existing perturbation methods for classification or segmentation. Experiments using 589 male pelvic MR images, labelled with eight anatomical ROIs, show the improvement in registration performance and the ablated contributions from the individual strategies. Furthermore, this study attempts to construct one of the first computational atlases for pelvic structures, enabled by registering inter-subject MRs, and quantifies the significant differences due to the proposed semi-weak supervision with a discussion on the potential clinical use of example atlas-derived statistics.

Augmented reality for laparoscopic liver resection is a visualisation mode that allows a surgeon to localise tumours and vessels embedded within the liver by projecting them on top of a laparoscopic image. Preoperative 3D models extracted from CT or MRI data are registered to the intraoperative laparoscopic images during this process. In terms of 3D-2D fusion, most of the algorithms make use of anatomical landmarks to guide registration. These landmarks include the liver's inferior ridge, the falciform ligament, and the occluding contours. They are usually marked by hand in both the laparoscopic image and the 3D model, which is time-consuming and may contain errors if done by a non-experienced user. Therefore, there is a need to automate this process so that augmented reality can be used effectively in the operating room. We present the Preoperative-to-Intraoperative Laparoscopic Fusion Challenge (P2ILF), held during the Medical Imaging and Computer Assisted Interventions (MICCAI 2022) conference, which investigates the possibilities of detecting these landmarks automatically and using them in registration. The challenge was divided into two tasks: 1) A 2D and 3D landmark detection task and 2) a 3D-2D registration task. The teams were provided with training data consisting of 167 laparoscopic images and 9 preoperative 3D models from 9 patients, with the corresponding 2D and 3D landmark annotations. A total of 6 teams from 4 countries participated, whose proposed methods were evaluated on 16 images and two preoperative 3D models from two patients. All the teams proposed deep learning-based methods for the 2D and 3D landmark segmentation tasks and differentiable rendering-based methods for the registration task. Based on the experimental outcomes, we propose three key hypotheses that determine current limitations and future directions for research in this domain., Comment: 24 pages

Patchick 99 is a candidate globular cluster located in the direction of the Galactic bulge, with a proper motion almost identical to the field and extreme field star contamination. A recent analysis suggests it is a low-luminosity globular cluster with a population of RR Lyrae stars. We present new spectra of stars in and around Patchick 99, targeting specifically the 3 RR Lyrae stars associated with the cluster as well as the other RR Lyrae stars in the field. A sample of 53 giant stars selected from proper motions and a position on CMD are also observed. The three RR Lyrae stars associated with the cluster have similar radial velocities and distances, and two of the targeted giants also have radial velocities in this velocity regime and [Fe/H] metallicities that are slightly more metal-poor than the field. Therefore, if Patchick 99 is a bonafide globular cluster, it would have a radial velocity of -92+/-10 km s-1, a distance of 6.7+/-0.4 kpc (as determined from the RR Lyrae stars), and an orbit that confines it to the inner bulge., Comment: Accepted to The Astrophysical Journal Letters. Replaced due to a typo in the title

The disagreement between low- and high-redshift measurements of the Hubble parameter is emerging as a serious challenge to the standard model of cosmology. We develop a covariant cosmographic analysis of the Hubble parameter in a general spacetime, which is fully model-independent and can thus be used as part of a robust assessment of the tension. Here our focus is not on the tension but on understanding the relation between the physical expansion rate and its measurement by observers -- which is critical for model-independent measurements and tests. We define the physical Hubble parameter and its multipoles in a general spacetime and derive for the first time the covariant boost transformation of the multipoles measured by a heliocentric observer. The analysis is extended to the covariant deceleration parameter. Current cosmographic measurements of the expansion anisotropy contain discrepancies and disagreements, some of which may arise because the correct transformations for a moving observer are not applied. A heliocentric observer will detect a dipole, generated not only by a Doppler effect, but also by an aberration effect due to shear. In principle, the observer can measure both the intrinsic shear anisotropy and the velocity of the observer relative to the matter -- without any knowledge of peculiar velocities, which are gauge dependent and do not arise in a covariant approach. The practical implementation of these results is investigated in a follow-up paper. We further show that the standard cosmographic relation between the Hubble parameter, the redshift and the luminosity distance (or magnitude) is not invariant under boosts and holds only in the matter frame. A moving observer who applies the standard cosmographic relation should correct the luminosity distance by a redshift factor -- otherwise an incorrect dipole and a spurious octupole are predicted., Comment: Version accepted by JCAP: additional results and clarifications

The Milky Way Bulge extra-tidal star survey (MWBest) is a spectroscopic survey with the goal of identifying stripped globular cluster stars from inner Galaxy clusters. In this way, an indication of the fraction of metal-poor bulge stars that originated from globular clusters can be determined. We observed and analyzed stars in and around BH 261, an understudied globular cluster in the bulge. From seven giants within the tidal radius of the cluster, we measured an average heliocentric radial velocity of = -61 +- 2.6 km/s with a radial velocity dispersion of \sigma = 6.1 +- 1.9 km/s. The large velocity dispersion may have arisen from tidal heating in the cluster's orbit about the Galactic center, or because BH 261 has a high dynamical mass as well as a high mass-to-light ratio. From spectra of five giants, we measure an average metallicity of <[Fe/H]> = -1.1 +- 0.2 dex. We also spectroscopically confirm an RR Lyrae star in BH 261, which yields a distance to the cluster of 7.1 +- 0.4~kpc. Stars with 3D velocities and metallicities consistent with BH 261 reaching to ~0.5 degrees from the cluster are identified. A handful of these stars are also consistent with the spatial distribution of that potential debris from models focussing on the most recent disruption of the cluster., Comment: accepted for publication in The Astronomical Journal

The Blanco DECam Bulge Survey (BDBS) provides near-ultraviolet to near-infrared photometry for ~250 million unique stars. By combining BDBS photometry with the latest Gaia astrometry, we characterize the chemo-dynamics of red clump stars across the BDBS footprint, using an unprecedented sample size and sky coverage. We construct a sample of ~2.3 million red clump giants in the bulge with photometric metallicities, BDBS photometric distances, and proper motions. We study the kinematics of the red clump stars as a function of sky position and metallicity, by investigating proper motion rotation curves, velocity dispersions, and proper motion correlations across the southern Galactic bulge. We find that metal-poor red clump stars exhibit lower rotation amplitudes, at ~29 km s$^{-1}$ kpc^{-1}. The peak of the angular velocity is ~39 km s^{-1} kpc^{-1} for [Fe/H] ~ -0.2 dex, exhibiting declining rotation at higher [Fe/H]. The velocity dispersion is higher for metal-poor stars, while metal-rich stars show a steeper gradient with Galactic latitude, with a maximum dispersion at low latitudes along the bulge minor axis. Only metal-rich stars ([Fe/H] >~ -0.5 dex) show clear signatures of the bar in their kinematics, while the metal-poor population exhibits isotropic motions with an axisymmetric pattern around Galactic longitude l = 0. This work reports the largest sample of bulge stars with distance, metallicity, and astrometry and shows clear kinematic differences with metallicity. The global kinematics over the bulge agrees with earlier studies. However, we see striking changes with increasing metallicity and for the first time, see kinematic differences for stars with [Fe/H]>-0.5, suggesting that the bar itself may have kinematics that depends on metallicity., Comment: 12 pages, Accepted for publication in A&A

Objective: Reconstructing freehand ultrasound in 3D without any external tracker has been a long-standing challenge in ultrasound-assisted procedures. We aim to define new ways of parameterising long-term dependencies, and evaluate the performance. Methods: First, long-term dependency is encoded by transformation positions within a frame sequence. This is achieved by combining a sequence model with a multi-transformation prediction. Second, two dependency factors are proposed, anatomical image content and scanning protocol, for contributing towards accurate reconstruction. Each factor is quantified experimentally by reducing respective training variances. Results: 1) The added long-term dependency up to 400 frames at 20 frames per second (fps) indeed improved reconstruction, with an up to 82.4% lowered accumulated error, compared with the baseline performance. The improvement was found to be dependent on sequence length, transformation interval and scanning protocol and, unexpectedly, not on the use of recurrent networks with long-short term modules; 2) Decreasing either anatomical or protocol variance in training led to poorer reconstruction accuracy. Interestingly, greater performance was gained from representative protocol patterns, than from representative anatomical features. Conclusion: The proposed algorithm uses hyperparameter tuning to effectively utilise long-term dependency. The proposed dependency factors are of practical significance in collecting diverse training data, regulating scanning protocols and developing efficient networks. Significance: The proposed new methodology with publicly available volunteer data and code for parametersing the long-term dependency, experimentally shown to be valid sources of performance improvement, which could potentially lead to better model development and practical optimisation of the reconstruction application., Comment: Accepted to IEEE Transactions on Biomedical Engineering (TBME, 2023)

At present, most surface-quality prediction methods can only perform single-task prediction which results in under-utilised datasets, repetitive work and increased experimental costs. To counter this, the authors propose a Bayesian hierarchical model to predict surface-roughness measurements for a turning machining process. The hierarchical model is compared to multiple independent Bayesian linear regression models to showcase the benefits of partial pooling in a machining setting with respect to prediction accuracy and uncertainty quantification., Comment: Submitted to International Workshop on Structural Health Monitoring 2023, Stanford University, California, USA

Third-generation gravitational wave (GW) observatories such as the Einstein Telescope and Cosmic Explorer, together with the LSST survey at the Vera Rubin Observatory, will yield an abundance of extra-galactic transient objects. This opens the exciting possibility of using GW sources and Supernovae Type Ia (SNIa) as luminosity distance tracers of large-scale structure for the first time. The large volumes accessible to these surveys imply that we may need to include relativistic corrections, such as lensing and Doppler magnification. However, the amplitude of these effects depends on the magnification and evolution biases of the transient sources, which are not yet understood. In this paper we develop comprehensive frameworks to address and model these biases for both populations of transient objects; in particular, we define how to compute these biases for GW sources. We then analyse the impact of magnification and evolution biases on the relativistic corrections and on the angular power spectrum of these sources. We show that correct modelling and implementation of these biases is crucial for measuring the cross-correlations of transient sources at higher redshifts., Comment: 28 pages, 14 figures, 4 tables, comments welcome

The use of deep learning to undertake shape analysis of the complexities of the human head holds great promise. However, there have traditionally been a number of barriers to accurate modelling, especially when operating on both a global and local level. In this work, we will discuss the application of the Swap Disentangled Variational Autoencoder (SD-VAE) with relevance to Crouzon, Apert and Muenke syndromes. Although syndrome classification is performed on the entire mesh, it is also possible, for the first time, to analyse the influence of each region of the head on the syndromic phenotype. By manipulating specific parameters of the generative model, and producing procedure-specific new shapes, it is also possible to simulate the outcome of a range of craniofacial surgical procedures. This opens new avenues to advance diagnosis, aids surgical planning and allows for the objective evaluation of surgical outcomes.

Denoising diffusion models have found applications in image segmentation by generating segmented masks conditioned on images. Existing studies predominantly focus on adjusting model architecture or improving inference, such as test-time sampling strategies. In this work, we focus on improving the training strategy and propose a novel recycling method. During each training step, a segmentation mask is first predicted given an image and a random noise. This predicted mask, which replaces the conventional ground truth mask, is used for denoising task during training. This approach can be interpreted as aligning the training strategy with inference by eliminating the dependence on ground truth masks for generating noisy samples. Our proposed method significantly outperforms standard diffusion training, self-conditioning, and existing recycling strategies across multiple medical imaging data sets: muscle ultrasound, abdominal CT, prostate MR, and brain MR. This holds for two widely adopted sampling strategies: denoising diffusion probabilistic model and denoising diffusion implicit model. Importantly, existing diffusion models often display a declining or unstable performance during inference, whereas our novel recycling consistently enhances or maintains performance. We show that, under a fair comparison with the same network architectures and computing budget, the proposed recycling-based diffusion models achieved on-par performance with non-diffusion-based supervised training. By ensembling the proposed diffusion and the non-diffusion models, significant improvements to the non-diffusion models have been observed across all applications, demonstrating the value of this novel training method. This paper summarizes these quantitative results and discusses their values, with a fully reproducible JAX-based implementation, released at https://github.com/mathpluscode/ImgX-DiffSeg., Comment: Accepted for publication at the Journal of Machine Learning for Biomedical Imaging (MELBA) https://melba-journal.org/2023:016

We propose Boundary-RL, a novel weakly supervised segmentation method that utilises only patch-level labels for training. We envision the segmentation as a boundary detection problem, rather than a pixel-level classification as in previous works. This outlook on segmentation may allow for boundary delineation under challenging scenarios such as where noise artefacts may be present within the region-of-interest (ROI) boundaries, where traditional pixel-level classification-based weakly supervised methods may not be able to effectively segment the ROI. Particularly of interest, ultrasound images, where intensity values represent acoustic impedance differences between boundaries, may also benefit from the boundary delineation approach. Our method uses reinforcement learning to train a controller function to localise boundaries of ROIs using a reward derived from a pre-trained boundary-presence classifier. The classifier indicates when an object boundary is encountered within a patch, as the controller modifies the patch location in a sequential Markov decision process. The classifier itself is trained using only binary patch-level labels of object presence, which are the only labels used during training of the entire boundary delineation framework, and serves as a weak signal to inform the boundary delineation. The use of a controller function ensures that a sliding window over the entire image is not necessary. It also prevents possible false-positive or -negative cases by minimising number of patches passed to the boundary-presence classifier. We evaluate our proposed approach for a clinically relevant task of prostate gland segmentation on trans-rectal ultrasound images. We show improved performance compared to other tested weakly supervised methods, using the same labels e.g., multiple instance learning., Comment: Accepted to MICCAI Workshop MLMI 2023 (14th International Conference on Machine Learning in Medical Imaging)

Three-dimensional (3D) freehand ultrasound (US) reconstruction without using any additional external tracking device has seen recent advances with deep neural networks (DNNs). In this paper, we first investigated two identified contributing factors of the learned inter-frame correlation that enable the DNN-based reconstruction: anatomy and protocol. We propose to incorporate the ability to represent these two factors - readily available during training - as the privileged information to improve existing DNN-based methods. This is implemented in a new multi-task method, where the anatomical and protocol discrimination are used as auxiliary tasks. We further develop a differentiable network architecture to optimise the branching location of these auxiliary tasks, which controls the ratio between shared and task-specific network parameters, for maximising the benefits from the two auxiliary tasks. Experimental results, on a dataset with 38 forearms of 19 volunteers acquired with 6 different scanning protocols, show that 1) both anatomical and protocol variances are enabling factors for DNN-based US reconstruction; 2) learning how to discriminate different subjects (anatomical variance) and predefined types of scanning paths (protocol variance) both significantly improve frame prediction accuracy, volume reconstruction overlap, accumulated tracking error and final drift, using the proposed algorithm., Comment: Accepted to Advances in Simplifying Medical UltraSound (ASMUS) workshop at MICCAI 2023

Solar radio bursts are strongly affected by radio-wave scattering on density inhomogeneities, changing their observed time characteristics, sizes, and positions. The same turbulence causes angular broadening and scintillation of galactic and extra-galactic compact radio sources observed through the solar atmosphere. Using large-scale simulations of radio-wave transport, the characteristics of anisotropic density turbulence from $0.1 \, R_\odot$ to $1$ au are explored. For the first time, a profile of heliospheric density fluctuations is deduced that accounts for the properties of extra-solar radio sources, solar radio bursts, and in-situ density fluctuation measurements in the solar wind at $1$ au. The radial profile of the spectrum-weighted mean wavenumber of density fluctuations (a quantity proportional to the scattering rate of radio-waves) is found to have a broad maximum at around $(4-7) \, R_\odot$, where the slow solar wind becomes supersonic. The level of density fluctuations at the inner scale (which is consistent with the proton resonance scale) decreases with heliocentric distance as $\langle\delta{n_i}^2 \rangle (r) \simeq 2 \times 10^7 \, (r/R_\odot-1)^{-3.7}$ cm$^{-6}$. Due to scattering, the apparent positions of solar burst sources observed at frequencies between $0.1$ and $300$ MHz are computed to be essentially cospatial and to have comparable sizes, for both fundamental and harmonic emission. Anisotropic scattering is found to account for the shortest solar radio burst decay times observed, and the required wavenumber anisotropy is $q_\parallel/q_\perp =0.25-0.4$, depending on whether fundamental or harmonic emission is involved. The deduced radio-wave scattering rate paves the way to quantify intrinsic solar radio burst characteristics., Comment: 27 pages, 12 figures