Search

Weed damage in rice fields is one of the main causes of reduced rice yields and quality. Accurate and efficient weed identification is the prerequisite for realizing intelligent and precise weeding in paddies. Recently, Vision Transformers (ViTs) have emerged with superior performance on computer vision tasks compared to the convolutional neural network (CNN)-based models. However, the lack of fully labeled weed datasets hinders the potential application of deep learning models in weed identification. To address the above issues, this study customizes a novel point-supervised instance segmentation network (PIS-Net) for weakly supervised instance segmentation of weeds in rice fields. More correctly, we first propose a novel instance segmentation point labeling scheme that utilizes randomly generated annotation points within each instance, aiming to decrease both labeling time and difficulty. Additionally, to make optimal use of point labels, this study puts forth a mask generation strategy based on the adaptive selection of pyramid levels. In this sense, the network model can flexibly choose the pyramid level expected to generate the most suitable instance mask based on the network's reliability. Finally, we establish the pseudo label refinement network (PLR-Net) to refine rough instance masks. The proposed PIS-Net utilizes 13 randomly generated annotation points for each instance, yet achieving an AP of 38.5 and an AP50 of 68.3, which is superior to the baseline mask-R-CNN with an AP of 8.2 and AP50 of 6.9, achieving 90 % fully supervised performance. This method effectively utilizes point labels, annotated with high efficiency, as a robust source of weak supervision to address challenges in weed data annotation and the low accuracy of existing weakly supervised models. Experiments show that the point annotation scheme of the PIS-Net is faster than full-object mask annotation, and the AP is also higher than the current semi-supervised weed segmentation model, enjoying high potentials in practical paddy fields.

Remote sensing (RS) instance segmentation is an important but challenging task due to multi-oriented, densely arranged objects and lack of mask annotation. Compared with redundant horizontal bounding-box (HBB) and expensive pixel-level annotation, oriented bounding box (OBB) annotations can provide compact object depicts with lower annotation costs. Therefore, we propose the first weakly supervised remote sensing instance segmentation method with OBB supervision (namely OBBInst) to reduce the annotation burden and make full use of existing abundant OBB annotations. Based on BoxInst (a high-performance instance segmentation method with box annotations), OBBInst has customized a framework for OBB annotation to unify the incompatibility between existing HBB-based and OBB-based methods. In addition, we propose an oriented projection method with a corresponding loss function to achieve more precise target depicts of OBB annotation. Moreover, we propose an edge similarity loss to incorporate Canny edge prior into deep learning framework for more precise edge identification of densely arranged objects. We have conducted extensive experiments on iSAID and HRSC datasets, and the experimental results demonstrate that OBBInst can achieve the state-of-the-art performance as compared to existing box-supervised methods. In addition, OBBInst dramatically narrows the performance gap between weakly and fully supervised instance segmentation (23.9% vs. 35.6% in iSAID dataset and 79.5% vs. 84.9% in HRSC dataset).

In the field of histopathology, many studies on the classification of whole slide images (WSIs) using artificial intelligence (AI) technology have been reported. We have studied the disease progression assessment of glioma. Adult-type diffuse gliomas, a type of brain tumor, are classified into astrocytoma, oligodendroglioma, and glioblastoma. Astrocytoma and oligodendroglioma are also called low grade glioma (LGG), and glioblastoma is also called glioblastoma multiforme (GBM). LGG patients frequently have isocitrate dehydrogenase (IDH) mutations. Patients with IDH mutations have been reported to have a better prognosis than patients without IDH mutations. Therefore, IDH mutations are an essential indicator for the classification of glioma. That is why we focused on the IDH1 mutation. In this paper, we aimed to classify the presence or absence of the IDH1 mutation using WSIs and clinical data of glioma patients. Ensemble learning between the WSIs model and the clinical data model is used to classify the presence or absence of IDH1 mutation. By using slide level labels, we combined patch-based imaging information from hematoxylin and eosin (H & E) stained WSIs, along with clinical data using deep image feature extraction and machine learning classifier for predicting IDH1 gene mutation prediction versus wild-type across cohort of 546 patients. We experimented with different deep learning (DL) models including attention-based multiple instance learning (ABMIL) models on imaging data along with gradient boosting machine (LightGBM) for the clinical variables. Further, we used hyperparameter optimization to find the best overall model in terms of classification accuracy. We obtained the highest area under the curve (AUC) of 0.823 for WSIs, 0.782 for clinical data, and 0.852 for ensemble results using MaxViT and LightGBM combination, respectively. Our experimental results indicate that the overall accuracy of the AI models can be improved by using both clinical data and images., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 The Author(s). Published by Elsevier Ltd.. All rights reserved.)

The current methods of auto-segmenting medical images are limited due to insufficient and ambiguous pathonmorphological labeling. In clinical practice, rough classification labels (such as disease or normal) are more commonly used than precise segmentation masks. However, there is still much to be explored regarding utilizing these weak clinical labels to accurately determine the lesion mask and guide medical image segmentation. In this paper, we proposed a weakly supervised medical image segmentation model to directly generate the lesion mask through a class activation map (CAM) guided cycle-consistency label-activated region transferring network. Cycle-consistency enforces that the mappings between the two domains should be reversible, which ensures that the original image can be reconstructed from the translated image. We developed a complementary branches fusion module to address the issue of blurry boundaries in CAM-guided segmentation. The complementary branch preserves the original semantic information of the non-lesion region and perfectly fuses the transferred feature of the lesion region with a complementary mask-constrained fake image generation process to clear the boundary of the lesion and non-lesion regions. This module allows the class transformation to focus solely on the label-activated region, resulting in more explicit segmentation. This model can accurately identify different region of medical images at the pixel-level while preserving the overall semantic structure semantion. It organizes disease labels and corresponding regions during image synthesis. Our method utilizes a joint discrimination strategy that significantly enhances the precision of the produced lesion mask. Extensive experiments of the proposed method on BraTs, ISIC and COVID-19 datasets demonstrate superior performance over existing state-of-the-art methods. The code and datasets are available at: https://github.com/mlcb-jlu/MedImgSeg., Competing Interests: Declaration of competing interest We declare that we have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 Elsevier Ltd. All rights reserved.)

The frequency of basal cell carcinoma (BCC) cases is putting an increasing strain on dermatopathologists. BCC is the most common type of skin cancer, and its incidence is increasing rapidly worldwide. AI can play a significant role in reducing the time and effort required for BCC diagnostics and thus improve the overall efficiency of the process. To train such an AI system in a fully-supervised fashion however, would require a large amount of pixel-level annotation by already strained dermatopathologists. Therefore, in this study, our primary objective was to develop a weakly-supervised for the identification of basal cell carcinoma (BCC) and the stratification of BCC into low-risk and high-risk categories within histopathology whole-slide images (WSI). We compared Clustering-constrained Attention Multiple instance learning (CLAM) with StreamingCLAM and hypothesized that the latter would be the superior approach. A total of 5147 images were used to train and validate the models, which were subsequently tested on an internal set of 949 images and an external set of 183 images. The labels for training were automatically extracted from free-text pathology reports using a rule-based approach. All data has been made available through the COBRA dataset. The results showed that both the CLAM and StreamingCLAM models achieved high performance for the detection of BCC, with an area under the ROC curve (AUC) of 0.994 and 0.997, respectively, on the internal test set and 0.983 and 0.993 on the external dataset. Furthermore, the models performed well on risk stratification, with AUC values of 0.912 and 0.931, respectively, on the internal set, and 0.851 and 0.883 on the external set. In every single metric the StreamingCLAM model outperformed the CLAM model or is on par. The performance of both models was comparable to that of two pathologists who scored 240 BCC positive slides. Additionally, in the public test set, StreamingCLAM demonstrated a comparable AUC of 0.958, markedly superior to CLAM's 0.803. This difference was statistically significant and emphasized the strength and better adaptability of the StreamingCLAM approach., Competing Interests: Declaration of competing interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Stephan Dooper reports financial support was provided by Innovative Medicines Initiative. Geert Litjens reports financial support was provided by Innovative Medicines Initiative. Witali Aswolinskiy reports financial support was provided by European Union. Geert Litjens reports a relationship with Dutch Research Council that includes: funding grants. Geert Litjens reports a relationship with Dutch Cancer Association that includes: funding grants. Geert Litjens reports a relationship with HealthHolland that includes: funding grants. Geert Litjens reports a relationship with European Research Council that includes: funding grants. Geert Litjens reports a relationship with European Union that includes: funding grants. Geert Litjens reports a relationship with Canon Healthcare IT that includes: board membership. Geert Litjens reports a relationship with Aiosyn BV that includes: equity or stocks., (Copyright © 2024 The Authors. Published by Elsevier B.V. All rights reserved.)

Uterine segmentation of endometrial cancer MR images can be a valuable diagnostic tool for gynecologists. However, uterine segmentation based on deep learning relies on artificial pixel-level annotation, which is time-consuming, laborious and subjective. To reduce the dependence on pixel-level annotation, a method of weakly supervised uterine segmentation on endometrial cancer MRI slices is proposed, which only requires scribble label and is enhanced by pseudo-label technology, exponential geodesic distance loss and input disturbance strategy. Specifically, the limitations caused by the shortage of supervision are addressed by dynamically mixing the two outputs of the dual branch network to generate pseudo-labels, expanding supervision information and promoting mutual supervision training. On the other hand, considering the large difference of grayscale intensity between the uterus and surrounding tissues, the exponential geodesic distance loss is introduced to enhance the ability of the network to capture the edge of the uterus. Input disturbance strategies are incorporated to adapt to the flexible and variable characteristics of the uterus and further improve the segmentation performance of the network. The proposed method is evaluated on MRI images from 135 cases of endometrial cancer. Compared with other four weakly supervised segmentation methods, the performance of the proposed method is the best, whose mean DI, HD 95 , Recall, Precision, ADP are 92.8%, 11.632, 92.7%, 93.6%, 6.5% and increasing by 2.1%, 9.144, 0.6%, 2.4%, 2.9% respectively. The experimental results demonstrate that the proposed method is more effective than other weakly supervised methods and achieves similar performance as those fully supervised., Competing Interests: Declaration of competing interest There is no conflict of interests in this work., (Copyright © 2023 Elsevier Ltd. All rights reserved.)

Recently, convolutional neural networks (CNNs) directly using whole slide images (WSIs) for tumor diagnosis and analysis have attracted considerable attention, because they only utilize the slide-level label for model training without any additional annotations. However, it is still a challenging task to directly handle gigapixel WSIs, due to the billions of pixels and intra-variations in each WSI. To overcome this problem, in this paper, we propose a novel end-to-end interpretable deep MIL framework for WSI analysis, by using a two-branch deep neural network and a multi-scale representation attention mechanism to directly extract features from all patches of each WSI. Specifically, we first divide each WSI into bag-, patch- and cell-level images, and then assign the slide-level label to its corresponding bag-level images, so that WSI classification becomes a MIL problem. Additionally, we design a novel multi-scale representation attention mechanism, and embed it into a two-branch deep network to simultaneously mine the bag with a correct label, the significant patches and their cell-level information. Extensive experiments demonstrate the superior performance of the proposed framework over recent state-of-the-art methods, in term of classification accuracy and model interpretability. All source codes are released at: https://github.com/xhangchen/MRAN/., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2023 Elsevier B.V. All rights reserved.)

At the end of 2019, a novel coronavirus, COVID-19, was ravaging the world, wreaking havoc on public health and the global economy. Today, although Reverse Transcription-Polymerase Chain Reaction (RT-PCR) is the gold standard for COVID-19 clinical diagnosis, it is a time-consuming and labor-intensive procedure. Simultaneously, an increasing number of individuals are seeking for better alternatives to RT-PCR. As a result, automated identification of COVID-19 lung infection in computed tomography (CT) images may help traditional diagnostic approaches in determining the severity of the disease. Unfortunately, a shortage of labeled training sets makes using AI deep learning algorithms to accurately segregate diseased regions in CT scan challenging. We design a simple and effective weakly supervised learning strategy for COVID-19 CT image segmentation to overcome the segmentation issue in the absence of adequate labeled data, namely LLC-Net. Unlike others weakly supervised work that uses a complex training procedure, our LLC-Net is relatively easy and repeatable. We propose a Local Self-Coherence Mechanism to accomplish label propagation based on lesion area labeling characteristics for weak labels that cannot offer comprehensive lesion areas, hence forecasting a more complete lesion area. Secondly, when the COVID-19 training samples are insufficient, the Scale Transform for Self-Correlation is designed to optimize the robustness of the model to ensure that the CT images are consistent in the prediction results from different angles. Finally, in order to constrain the segmentation accuracy of the lesion area, the Lesion Infection Edge Attention Module is used to improve the information expression ability of edge modeling. Experiments on public datasets demonstrate that our method is more effective than other weakly supervised methods and achieves a new state-of-the-art performance., Competing Interests: The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (© 2022 Elsevier Ltd. All rights reserved.)

An important challenge and limiting factor in deep learning methods for medical imaging segmentation is the lack of available of annotated data to properly train models. For the specific task of tumor segmentation, the process entails clinicians labeling every slice of volumetric scans for every patient, which becomes prohibitive at the scale of datasets required to train neural networks to optimal performance. To address this, we propose a novel semi-supervised framework that allows training any segmentation (encoder-decoder) model using only information readily available in radiological data, namely the presence of a tumor in the image, in addition to a few annotated images. Specifically, we conjecture that a generative model performing domain translation on this weak label - healthy vs diseased scans - helps achieve tumor segmentation. The proposed GenSeg method first disentangles tumoral tissue from healthy "background" tissue. The latent representation is separated into (1) the common background information across both domains, and (2) the unique tumoral information. GenSeg then achieves diseased-to-healthy image translation by decoding a healthy version of the image from just the common representation, as well as a residual image that allows adding back the tumors. The same decoder that produces this residual tumor image, also outputs a tumor segmentation. Implicit data augmentation is achieved by re-using the same framework for healthy-to-diseased image translation, where a residual tumor image is produced from a prior distribution. By performing both image translation and segmentation simultaneously, GenSeg allows training on only partially annotated datasets. To test the framework, we trained U-Net-like architectures using GenSeg and evaluated their performance on 3 variants of a synthetic task, as well as on 2 benchmark datasets: brain tumor segmentation in MRI (derived from BraTS) and liver metastasis segmentation in CT (derived from LiTS). Our method outperforms the baseline semi-supervised (autoencoder and mean teacher) and supervised segmentation methods, with improvements ranging between 8-14% Dice score on the brain task and 5-8% on the liver task, when only 1% of the training images were annotated. These results show the proposed framework is ideal at addressing the problem of training deep segmentation models when a large portion of the available data is unlabeled and unpaired, a common issue in tumor segmentation., Competing Interests: Declaration of Competing Interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Eugene Vorontsov reports financial support was provided by NVIDIA Corp. Eugene Vorontsov, Pavlo Molchanov has patent pending to Nvidia., (Copyright © 2022 Elsevier B.V. All rights reserved.)

Objective: Electronic Health Record (EHR) based phenotyping is a crucial yet challenging problem in the biomedical field. Though clinicians typically determine patient-level diagnoses via manual chart review, the sheer volume and heterogeneity of EHR data renders such tasks challenging, time-consuming, and prohibitively expensive, thus leading to a scarcity of clinical annotations in EHRs. Weakly supervised learning algorithms have been successfully applied to various EHR phenotyping problems, due to their ability to leverage information from large quantities of unlabeled samples to better inform predictions based on a far smaller number of patients. However, most weakly supervised methods are subject to the challenge to choose the right cutoff value to generate an optimal classifier. Furthermore, since they only utilize the most informative features (i.e., main ICD and NLP counts) they may fail for episodic phenotypes that cannot be consistently detected via ICD and NLP data. In this paper, we propose a label-efficient, weakly semi-supervised deep learning algorithm for EHR phenotyping (WSS-DL), which overcomes the limitations above., Materials and Methods: WSS-DL classifies patient-level disease status through a series of learning stages: 1) generating silver standard labels, 2) deriving enhanced-silver-standard labels by fitting a weakly supervised deep learning model to data with silver standard labels as outcomes and high dimensional EHR features as input, and 3) obtaining the final prediction score and classifier by fitting a supervised learning model to data with a minimal number of gold standard labels as the outcome, and the enhanced-silver-standard labels and a minimal set of most informative EHR features as input. To assess the generalizability of WSS-DL across different phenotypes and medical institutions, we apply WSS-DL to classify a total of 17 diseases, including both acute and chronic conditions, using EHR data from three healthcare systems. Additionally, we determine the minimum quantity of training labels required by WSS-DL to outperform existing supervised and semi-supervised phenotyping methods., Results: The proposed method, in combining the strengths of deep learning and weakly semi-supervised learning, successfully leverages the crucial phenotyping information contained in EHR features from unlabeled samples. Indeed, the deep learning model's ability to handle high-dimensional EHR features allows it to generate strong phenotype status predictions from silver standard labels. These predictions, in turn, provide highly effective features in the final logistic regression stage, leading to high phenotyping accuracy in notably small subsets of labeled data (e.g. n = 40 labeled samples)., Conclusion: Our method's high performance in EHR datasets with very small numbers of labels indicates its potential value in aiding doctors to diagnose rare diseases as well as conditions susceptible to misdiagnosis., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2022. Published by Elsevier Inc.)

[S U M M A R Y] Weakly supervised segmentation for medical images ease the reliance of models on pixel-level annotation while advancing the field of computer-aided diagnosis. However, the differences in nodule size in thyroid ultrasound images and the limitations of class activation maps in weakly supervised segmentation methods typically lead to under- and/or over-segmentation problems in real predictions. To alleviate this problem, we propose a weakly supervised segmentation neural network approach. This new method is based on a dual branch soft erase module that expands the foreground response region while constraining the erroneous expansion of the foreground region by the enhancement of background features. The sensitivity of this neural network to the nodule scale size is further enhanced by the scale feature adaptation module, which in turn generates integral and high-quality segmentation masks. In addition, while the nodule area can be significantly expanded through soft erase module and scale feature adaptation module, the activation effect in the nodule edge area is still not satisfactory, so that we further add an edge-based attention mechanism to strengthen the nodule edge segmentation effect. The results of experiments performed on the thyroid ultrasound image dataset showed that our new approach significantly outperformed existing weakly supervised semantic segmentation methods, e.g., 5.9% and 6.3% more accurate than the second-based results in terms of Jaccard and Dice coefficients, respectively., (Copyright © 2022 Elsevier Ltd. All rights reserved.)

Digital pathology is a field of increasing interest and requires automated systems for processing huge amounts of digital data. The development of supervised-learning based automated systems is aggravated by the fact that image properties can change from slide to slide. In this work, the focus is on the segmentation of the glomeruli constituting the most important regions-of-interest in renal histopathology. We propose and investigate a two-stage pipeline consisting of a weakly supervised patch-based detection and a precise segmentation. The proposed methods do not need any previously obtained training data. For adapting and optimizing this model, a kernel two-sample test is applied. For the segmentation stage, unsupervised segmentation methods including level-set and polygon-fitting approaches are adapted, combined and evaluated. Overall, with the best performing polygon-fitting segmentation method, 51% of glomeruli were segmented with sufficient accuracy (DSC > 0.8). 42% of the detections were false positives. Due to the difficult application scenario in combination with the small required training corpus, the obtained performance is assessed as good. Strategies for increasing the segmentation performance even further are discussed in detail., (Copyright © 2017 Elsevier Ltd. All rights reserved.)