Search

Multiple instance learning (MIL) has become a standard paradigm for weakly supervised classification of whole slide images (WSI). However, this paradigm relies on the use of a large number of labelled WSIs for training. The lack of training data and the presence of rare diseases present significant challenges for these methods. Prompt tuning combined with the pre-trained Vision-Language models (VLMs) is an effective solution to the Few-shot Weakly Supervised WSI classification (FSWC) tasks. Nevertheless, applying prompt tuning methods designed for natural images to WSIs presents three significant challenges: 1) These methods fail to fully leverage the prior knowledge from the VLM's text modality; 2) They overlook the essential multi-scale and contextual information in WSIs, leading to suboptimal results; and 3) They lack exploration of instance aggregation methods. To address these problems, we propose a Multi-Scale and Context-focused Prompt Tuning (MSCPT) method for FSWC tasks. Specifically, MSCPT employs the frozen large language model to generate pathological visual language prior knowledge at multi-scale, guiding hierarchical prompt tuning. Additionally, we design a graph prompt tuning module to learn essential contextual information within WSI, and finally, a non-parametric cross-guided instance aggregation module has been introduced to get the WSI-level features. Based on two VLMs, extensive experiments and visualizations on three datasets demonstrated the powerful performance of our MSCPT., Comment: 11 pages, 5 figures, 5tables

Accurate and robust multimodal multi-task perception is crucial for modern autonomous driving systems. However, current multimodal perception research follows independent paradigms designed for specific perception tasks, leading to a lack of complementary learning among tasks and decreased performance in multi-task learning (MTL) due to joint training. In this paper, we propose MaskBEV, a masked attention-based MTL paradigm that unifies 3D object detection and bird's eye view (BEV) map segmentation. MaskBEV introduces a task-agnostic Transformer decoder to process these diverse tasks, enabling MTL to be completed in a unified decoder without requiring additional design of specific task heads. To fully exploit the complementary information between BEV map segmentation and 3D object detection tasks in BEV space, we propose spatial modulation and scene-level context aggregation strategies. These strategies consider the inherent dependencies between BEV segmentation and 3D detection, naturally boosting MTL performance. Extensive experiments on nuScenes dataset show that compared with previous state-of-the-art MTL methods, MaskBEV achieves 1.3 NDS improvement in 3D object detection and 2.7 mIoU improvement in BEV map segmentation, while also demonstrating slightly leading inference speed., Comment: Accepted to ACM MM 2024

Vision-based 3D semantic scene completion (SSC) describes autonomous driving scenes through 3D volume representations. However, the occlusion of invisible voxels by scene surfaces poses challenges to current SSC methods in hallucinating refined 3D geometry. This paper proposes HybridOcc, a hybrid 3D volume query proposal method generated by Transformer framework and NeRF representation and refined in a coarse-to-fine SSC prediction framework. HybridOcc aggregates contextual features through the Transformer paradigm based on hybrid query proposals while combining it with NeRF representation to obtain depth supervision. The Transformer branch contains multiple scales and uses spatial cross-attention for 2D to 3D transformation. The newly designed NeRF branch implicitly infers scene occupancy through volume rendering, including visible and invisible voxels, and explicitly captures scene depth rather than generating RGB color. Furthermore, we present an innovative occupancy-aware ray sampling method to orient the SSC task instead of focusing on the scene surface, further improving the overall performance. Extensive experiments on nuScenes and SemanticKITTI datasets demonstrate the effectiveness of our HybridOcc on the SSC task., Comment: Accepted to IEEE RAL

Survival prediction is a complex ordinal regression task that aims to predict the survival coefficient ranking among a cohort of patients, typically achieved by analyzing patients' whole slide images. Existing deep learning approaches mainly adopt multiple instance learning or graph neural networks under weak supervision. Most of them are unable to uncover the diverse interactions between different types of biological entities(\textit{e.g.}, cell cluster and tissue block) across multiple scales, while such interactions are crucial for patient survival prediction. In light of this, we propose a novel multi-scale heterogeneity-aware hypergraph representation framework. Specifically, our framework first constructs a multi-scale heterogeneity-aware hypergraph and assigns each node with its biological entity type. It then mines diverse interactions between nodes on the graph structure to obtain a global representation. Experimental results demonstrate that our method outperforms state-of-the-art approaches on three benchmark datasets. Code is publicly available at \href{https://github.com/Hanminghao/H2GT}{https://github.com/Hanminghao/H2GT}., Comment: 9 pages, 6 figures, accepted by ICME2024

Augmented reality for laparoscopic liver resection is a visualisation mode that allows a surgeon to localise tumours and vessels embedded within the liver by projecting them on top of a laparoscopic image. Preoperative 3D models extracted from CT or MRI data are registered to the intraoperative laparoscopic images during this process. In terms of 3D-2D fusion, most of the algorithms make use of anatomical landmarks to guide registration. These landmarks include the liver's inferior ridge, the falciform ligament, and the occluding contours. They are usually marked by hand in both the laparoscopic image and the 3D model, which is time-consuming and may contain errors if done by a non-experienced user. Therefore, there is a need to automate this process so that augmented reality can be used effectively in the operating room. We present the Preoperative-to-Intraoperative Laparoscopic Fusion Challenge (P2ILF), held during the Medical Imaging and Computer Assisted Interventions (MICCAI 2022) conference, which investigates the possibilities of detecting these landmarks automatically and using them in registration. The challenge was divided into two tasks: 1) A 2D and 3D landmark detection task and 2) a 3D-2D registration task. The teams were provided with training data consisting of 167 laparoscopic images and 9 preoperative 3D models from 9 patients, with the corresponding 2D and 3D landmark annotations. A total of 6 teams from 4 countries participated, whose proposed methods were evaluated on 16 images and two preoperative 3D models from two patients. All the teams proposed deep learning-based methods for the 2D and 3D landmark segmentation tasks and differentiable rendering-based methods for the registration task. Based on the experimental outcomes, we propose three key hypotheses that determine current limitations and future directions for research in this domain., Comment: 24 pages

Accurate tissue segmentation of thick-slice fetal brain magnetic resonance (MR) scans is crucial for both reconstruction of isotropic brain MR volumes and the quantification of fetal brain development. However, this task is challenging due to the use of thick-slice scans in clinically-acquired fetal brain data. To address this issue, we propose to leverage high-quality isotropic fetal brain MR volumes (and also their corresponding annotations) as guidance for segmentation of thick-slice scans. Due to existence of significant domain gap between high-quality isotropic volume (i.e., source data) and thick-slice scans (i.e., target data), we employ a domain adaptation technique to achieve the associated knowledge transfer (from high-quality volumes to thick-slice scans). Specifically, we first register the available high-quality isotropic fetal brain MR volumes across different gestational weeks to construct longitudinally-complete source data. To capture domain-invariant information, we then perform Fourier decomposition to extract image content and style codes. Finally, we propose a novel Cycle-Consistent Domain Adaptation Network (C2DA-Net) to efficiently transfer the knowledge learned from high-quality isotropic volumes for accurate tissue segmentation of thick-slice scans. Our C2DA-Net can fully utilize a small set of annotated isotropic volumes to guide tissue segmentation on unannotated thick-slice scans. Extensive experiments on a large-scale dataset of 372 clinically acquired thick-slice MR scans demonstrate that our C2DA-Net achieves much better performance than cutting-edge methods quantitatively and qualitatively., Comment: 10 pages, 9 figures, 5 tables, Fetal MRI, Brain tissue segmentation, Unsupervised domain adaptation, Cycle-consistency

We investigate the second order energy density perturbation $\delta^{(2)}$ induced by small-scale Gaussian and local-type non-Gaussian primordial curvature perturbations. The relative abundance of primordial black hole is calculated in terms of the probability density function of total energy density perturbation $\delta_r=\delta^{(1)}+\frac{1}{2}\delta^{(2)}$. The effects of second order density perturbation greatly reduce the upper bounds of small-scale power spectra of primordial curvature perturbations by one to two orders of magnitude. For log-normal primordial power spectrum, its amplitude $A_{\zeta}$ is constrained to be about $A_{\zeta}\sim 3\times10^{-3}$. And for local-type non-Gaussianity with $f_{\mathrm{NL}}=10$, the upper bound of $A_{\zeta}$ is about $2.5\times10^{-4}$., Comment: 5 pages, 4 figures

Recently, from 12 $\gamma$-ray Galactic sources, the LHAASO has detected ultrahigh-energy photons up to 1.4PeV. The $\gamma$-ray spectra of the sources J2226+6057, J1908+0621, J1825-1326 and the suggested origin pulsars near the sources have been published. In our previous work, we studied the hadronic $\gamma$-ray spectra of the sources J2226+6057, J1908+0621, J1825-1326 in terms of the Hertzian dipole model of pulsar. In this paper, we investigate the possibility of the leptonic origin of the $\gamma$-ray. We use the Hertzian dipole model to describe the pulsars around the sources. The electrons around the pulsars can be accelerated to PeV by the electromagnetic fields of pulsars. Under the assumption that the initial electrons are uniform distributed in a spherical shell between $10^{7}$ to $10^{9}$m around the pulsar, we obtain the energy distribution of electrons. The leptonic $\gamma$-ray spectra can be calculated through inverse Compton scattering processes. The leptonic $\gamma$-ray can roughly conform to the observation of LHAASO.

We study the second order scalar and density perturbations generated by the Gaussian curvature perturbations and primordial gravitational waves in the radiation-dominated era. After presenting all the possible second-order source terms, we obtain the explicit expressions of the kernel functions and the power spectra of the second order scalar perturbations. It shows that the primordial gravitational waves might affect the second order energy density perturbation significantly. The effects of the primordial gravitational waves are studied in terms of different kinds of primordial power spectra.

The process of \acp{PBH} formation would be inevitably accompanied by \acp{SIGW}. This strong correlation between \acp{PBH} and \acp{SIGW} signals could be a promising approach to detecting \acp{PBH} in the upcoming \ac{GW} experiments, such as \ac{LISA}. We investigate the third order \acp{SIGW} during a \ac{RD} era in the case of a monochromatic primordial power spectrum $\mathcal{P}_{\zeta}=A_{\zeta}k_*\delta\left(k-k_*\right)$. For \ac{LISA} observations, the relations between \ac{SNR} and monochromatic primordial power spectrum are studied systematically. It shows that the effects of third order \acp{SIGW} extend the cutoff frequency from $2f_*$ to $3f_*$ and lead to about $200\%$ increase of the \ac{SNR} for frequency band from $10^{-5}$Hz to $1.6\times 10^{-3}$Hz corresponding to \acp{PBH} with mass range $4\times 10^{-12}M_{\odot} \sim 10^{-7}M_{\odot}$. We find that there exists a critical value $A_*=1.76\times 10^{-2}$ for the amplitude of the monochromatic primordial power spectra, such that when $A_{\zeta}>A_*$, the energy density of third order \acp{SIGW} will be larger than the energy density of second order \acp{SIGW}.

We investigate the second order gravitational waves induced by the primordial scalar and tensor perturbations during radiation-dominated era. The explicit expressions of the power spectra of the second order GWs are presented. We calculate the energy density spectra of the second order GWs for a monochromatic primordial power spectra. For large $k$ $\left( k>k_* \right)$, the effects of the primordial tensor perturbation with tensor-to-scalar ratio $r=A_{h}/A_{\zeta}=0.2$ lead to an around $50\% $ increase of the signal-to-noise ratio (SNR) for LISA observations.

The LHAASO Collaboration has observed ultrahigh-energy photons up to $1.4$PeV from $12$ $\gamma$-ray Galactic sources. In particular, the $\gamma$-ray spectra of the sources J2226+6057, J1908+0621, J1825-1326 have been published. We investigate the possibility of suggested origin pulsars near the sources as the PeVatrons. The pulsar is described by a rotating magnetic dipole. Assuming protons are uniform distributed out of the light cylinders, we obtain the Lorentz distribution of proton energy spectrum. It is found that the protons around pulsar could be accelerated to PeV at short times. The hadronic $\gamma$-ray spectra of the suggested origin pulsars are in good agreement with the LHAASO observed $\gamma$-ray spectra of the sources J2226+6057, J1908+0621, J1825-1326., Comment: This paper has been accepcted by MNRAS

The damping effect of the free-streaming neutrinos on the second order gravitational waves is investigated in detail. We solve the Boltzmann equation and give the anisotropic stress induced by neutrinos to second order. The first order tensor and its coupling with scalar perturbations induced gravitational waves are considered. We give the analytic equations of the damping kernel functions and finally obtain the energy density spectrum. The results show that the free-streaming neutrinos suppress the density spectrum significantly for low frequency gravitational waves and enlarge the logarithmic slope $n$ in the infrared region ($k \ll k_*$) of the spectrum. For the spectrum of $k_*\sim 10^{-7}$Hz, the damping effect in the range of $k

The cosmological background of higher order vector modes can be generated by the first order scalar perturbations. We investigate the second order and the third order vector modes systematically. The explicit expressions of two point functions $\langle V^{(n),\lambda}V^{(n),\lambda'} \rangle$$\left(n=2,3\right)$ and power spectra corresponded are presented. In the case of a monochromatic primordial power spectrum, the second order vector modes do not exist. However, the third order vector modes can be generated by a monochromatic primordial power spectrum. And it is found that the third order vector modes sourced by the second order scalar perturbations dominate the two point function $\langle V^{(3),\lambda}V^{(3),\lambda'} \rangle$ and power spectrum corresponded.

Since the gravitational waves were detected by LIGO and Virgo, it has been promising that lots of information about the primordial Universe could be learned by further observations on stochastic gravitational waves background. The studies on gravitational waves induced by primordial curvature perturbations are of great interest. The aim of this paper is to investigate the third order induced gravitational waves. Based on the theory of cosmological perturbations, the first order scalar induces the second order scalar, vector and tensor perturbations. At the next iteration, the first order scalar, the second order scalar, vector and tensor perturbations all induce the third order tensor perturbations. We present the energy density spectrum of the third order gravitational waves for a monochromatic primordial power spectrum. The shape of the energy density spectrum of the third order gravitational waves is different from that of the second order scalar induced gravitational waves. And it is found that the third order gravitational waves sourced by the second order scalar perturbations dominate the energy density spectrum., Comment: 33 pages, 4 figures

As an attractive candidate for dark matter, the primordial black holes (PBHs) in the mass range ($10^{15} \sim 10^{16}$)$\mathrm{g}$ could be detected via their Hawking radiation, including neutrinos and antineutrinos of three flavors. In this paper, we investigate the possibility to constrain the PBH as dark matter by measuring (anti)neutrino signals at the large liquid-scintillator detector of Jiangmen Underground Neutrino Observatory (JUNO). Among six available detection channels, the inverse beta decay $\overline{\nu}^{}_e + p \to e^+ + n$ is shown to be most sensitive to the fraction $f^{}_{\rm PBH}$ of PBHs contributing to the dark matter abundance. Given the PBH mass $M^{}_{\rm PBH} = 10^{15}~{\rm g}$, we find that JUNO will be able to place an upper bound $f^{}_{\rm PBH} \lesssim 3\times 10^{-5}$, which is 20 times better than the current best limit $f^{}_{\rm PBH} \lesssim 6\times 10^{-4}$ from Super-Kamiokande. For heavier PBHs with a lower Hawking temperature, the (anti)neutrinos become less energetic, leading to a relatively weaker bound., Comment: 11 pages, 7 figures

Predicting the mutation status of genes in tumors is of great clinical significance. Recent studies have suggested that certain mutations may be noninvasively predicted by studying image features of the tumors from Computed Tomography (CT) data. Currently, this kind of image feature identification method mainly relies on manual processing to extract generalized image features alone or machine processing without considering the morphological differences of the tumor itself, which makes it difficult to achieve further breakthroughs. In this paper, we propose a pyramid focusing network (PFNet) for mutation prediction and classification based on CT images. Firstly, we use Space Pyramid Pooling to collect semantic cues in feature maps from multiple scales according to the observation that the shape and size of the tumors are varied.Secondly, we improve the loss function based on the consideration that the features required for proper mutation detection are often not obvious in cross-sections of tumor edges, which raises more attention to these hard examples in the network. Finally, we devise a training scheme based on data augmentation to enhance the generalization ability of networks. Extensively verified on clinical gastric CT datasets of 20 testing volumes with 63648 CT images, our method achieves the accuracy of 94.90% in predicting the HER-2 genes mutation status of at the CT image., Comment: The carelessness of writing the paper led to the incorrect experimental results shown in the paper.In addition, the experimental design shown in the paper is incomplete, and I need to revise it substantially before submitting it