Search

Visual representations based on circular shapes are frequently used in visualization applications. One example are circos plots within bioinformatics, which bend graphs into a wheel of information with connective lines running through the center like spokes. The results are aesthetically appealing and impressive visualizations that fit long data sequences into a small quadratic space. However, the authors' experiences are that when asked, a visualization researcher would generally advise against making visualizations with radial layouts. Upon reviewing the literature we found that there is evidence that circular layouts are preferable in some cases, but we found no clear evidence for what layout is preferable for matrices and connective data in particular, which both are common data types in circos plots. In this work, we thus performed a user study to compare circular and linear layouts. The tasks are inspired by genomics data, but our results generalize to many other application areas, involving comparison and connective data. To build the prototype we utilized Gosling, a grammar for visualizing genomics data. We contribute empirical evidence on the suitedness of linear versus circular layouts, adding to the specific and general knowledge concerning perception of circular graphs. In addition, we contribute a case study evaluation of the grammar Gosling as a rapid prototyping language, confirming its utility and providing guidance on suitable areas for future development., Funding Agencies|Knut and Alice Wallenberg Foundation

Purpose: Uncertainty estimation has gained significant attention in recent years for its potential to enhance the performance of deep learning (DL) algorithms in medical applications and even potentially address domain shift challenges. However, it is not straightforward to incorporate uncertainty estimation with a DL system to achieve a tangible positive effect. The objective of our work is to evaluate if the proposed spatial uncertainty aggregation (SUA) framework may improve the effectiveness of uncertainty estimation in segmentation tasks. We evaluate if SUA boosts the observed correlation between the uncertainty estimates and false negative (FN) predictions. We also investigate if the observed benefits can translate to tangible improvements in segmentation performance. Approach: Our SUA framework processes negative prediction regions from a segmentation algorithm and detects FNs based on an aggregated uncertainty score. It can be utilized with many existing uncertainty estimation methods to boost their performance. We compare the SUA framework with a baseline of processing individual pixel's uncertainty independently. Results The results demonstrate that SUA is able to detect FN regions. It achieved F beta=0.5 of 0.92 on the in-domain and 0.85 on the domain-shift test data compared with 0.81 and 0.48 achieved by the baseline uncertainty, respectively. We also demonstrate that SUA yields improved general segmentation performance compared with utilizing the baseline uncertainty. Conclusions: We propose the SUA framework for incorporating and utilizing uncertainty estimates for FN detection in DL segmentation algorithms for histopathology. The evaluation confirms the benefits of our approach compared with assessing pixel uncertainty independently., Funding Agencies|Swedish e-Science Research Center; VINNOVA [2021-01420]; Knut and Alice Wallenberg Foundation

In this perspective article we discuss a certain type of research on visualization for bioinformatics data, namely, methods targeting clinical use. We argue that in this subarea additional complex challenges come into play, particularly so in genomics. We here describe four such challenge areas, elicited from a domain characterization effort in clinical genomics. We also list opportunities for visualization research to address clinical challenges in genomics that were uncovered in the case study. The findings are shown to have parallels with experiences from the diagnostic imaging domain., Funding Agencies|This research has been funded by the Swedish Foundation for Strategic Research, grant ID20-0092, and the Knut and Alice Wallenberg Foundation through Grant KAW 2019.0024. [ID20-0092]; Swedish Foundation for Strategic Research [KAW 2019.0024]; Knut and Alice Wallenberg Foundation

In this perspective article we discuss a certain type of research on visualization for bioinformatics data, namely, methods targeting clinical use. We argue that in this subarea additional complex challenges come into play, particularly so in genomics. We here describe four such challenge areas, elicited from a domain characterization effort in clinical genomics. We also list opportunities for visualization research to address clinical challenges in genomics that were uncovered in the case study. The findings are shown to have parallels with experiences from the diagnostic imaging domain., Funding Agencies|This research has been funded by the Swedish Foundation for Strategic Research, grant ID20-0092, and the Knut and Alice Wallenberg Foundation through Grant KAW 2019.0024. [ID20-0092]; Swedish Foundation for Strategic Research [KAW 2019.0024]; Knut and Alice Wallenberg Foundation

In this perspective article we discuss a certain type of research on visualization for bioinformatics data, namely, methods targeting clinical use. We argue that in this subarea additional complex challenges come into play, particularly so in genomics. We here describe four such challenge areas, elicited from a domain characterization effort in clinical genomics. We also list opportunities for visualization research to address clinical challenges in genomics that were uncovered in the case study. The findings are shown to have parallels with experiences from the diagnostic imaging domain., Funding Agencies|This research has been funded by the Swedish Foundation for Strategic Research, grant ID20-0092, and the Knut and Alice Wallenberg Foundation through Grant KAW 2019.0024. [ID20-0092]; Swedish Foundation for Strategic Research [KAW 2019.0024]; Knut and Alice Wallenberg Foundation

In this perspective article we discuss a certain type of research on visualization for bioinformatics data, namely, methods targeting clinical use. We argue that in this subarea additional complex challenges come into play, particularly so in genomics. We here describe four such challenge areas, elicited from a domain characterization effort in clinical genomics. We also list opportunities for visualization research to address clinical challenges in genomics that were uncovered in the case study. The findings are shown to have parallels with experiences from the diagnostic imaging domain., Funding Agencies|This research has been funded by the Swedish Foundation for Strategic Research, grant ID20-0092, and the Knut and Alice Wallenberg Foundation through Grant KAW 2019.0024. [ID20-0092]; Swedish Foundation for Strategic Research [KAW 2019.0024]; Knut and Alice Wallenberg Foundation

The discussion on artificial intelligence (AI) solutions in diagnostic imaging has matured in recent years. The potential value of AI adoption is well established, as are the potential risks associated. Much focus has, rightfully, been on regulatory certification of AI products, with the strong incentive of being an enabling step for the commercial actors. It is, however, becoming evident that regulatory approval is not enough to ensure safe and effective AI usage in the local setting. In other words, care providers need to develop and implement quality assurance (QA) approaches for AI solutions in diagnostic imaging. The domain of AI-specific QA is still in an early development phase. We contribute to this development by describing the current landscape of QA-for-AI approaches in medical imaging, with focus on radiology and pathology. We map the potential quality threats and review the existing QA approaches in relation to those threats. We propose a practical categorization of QA approaches, based on key characteristics corresponding to means, situation, and purpose. The review highlights the heterogeneity of methods and practices relevant for this domain and points to targets for future research efforts., Funding Agencies|Linkoping University; Vinnova [2021-01420]

The discussion on artificial intelligence (AI) solutions in diagnostic imaging has matured in recent years. The potential value of AI adoption is well established, as are the potential risks associated. Much focus has, rightfully, been on regulatory certification of AI products, with the strong incentive of being an enabling step for the commercial actors. It is, however, becoming evident that regulatory approval is not enough to ensure safe and effective AI usage in the local setting. In other words, care providers need to develop and implement quality assurance (QA) approaches for AI solutions in diagnostic imaging. The domain of AI-specific QA is still in an early development phase. We contribute to this development by describing the current landscape of QA-for-AI approaches in medical imaging, with focus on radiology and pathology. We map the potential quality threats and review the existing QA approaches in relation to those threats. We propose a practical categorization of QA approaches, based on key characteristics corresponding to means, situation, and purpose. The review highlights the heterogeneity of methods and practices relevant for this domain and points to targets for future research efforts., Funding Agencies|Linkoping University; Vinnova [2021-01420]

In this perspective article we discuss a certain type of research on visualization for bioinformatics data, namely, methods targeting clinical use. We argue that in this subarea additional complex challenges come into play, particularly so in genomics. We here describe four such challenge areas, elicited from a domain characterization effort in clinical genomics. We also list opportunities for visualization research to address clinical challenges in genomics that were uncovered in the case study. The findings are shown to have parallels with experiences from the diagnostic imaging domain., Funding Agencies|This research has been funded by the Swedish Foundation for Strategic Research, grant ID20-0092, and the Knut and Alice Wallenberg Foundation through Grant KAW 2019.0024. [ID20-0092]; Swedish Foundation for Strategic Research [KAW 2019.0024]; Knut and Alice Wallenberg Foundation

Multiple-instance learning (MIL) is an attractive approach for digital pathology applications as it reduces the costs related to data collection and labelling. However, it is not clear how sensitive MIL is to clinically realistic domain shifts, i.e., differences in data distribution that could negatively affect performance, and if already existing metrics for detecting domain shifts work well with these algorithms. We trained an attention-based MIL algorithm to classify whether a whole-slide image of a lymph node contains breast tumour metastases. The algorithm was evaluated on data from a hospital in a different country and various subsets of this data that correspond to different levels of domain shift. Our contributions include showing that MIL for digital pathology is affected by clinically realistic differences in data, evaluating which features from a MIL model are most suitable for detecting changes in performance, and proposing an unsupervised metric named Fréchet Domain Distance (FDD) for quantification of domain shifts. Shift measure performance was evaluated through the mean Pearson correlation to change in classification performance, where FDD achieved 0.70 on 10-fold cross-validation models. The baselines included Deep ensemble, Difference of Confidence, and Representation shift which resulted in 0.45, -0.29, and 0.56 mean Pearson correlation, respectively. FDD could be a valuable tool for care providers and vendors who need to verify if a MIL system is likely to perform reliably when implemented at a new site, without requiring any additional annotations from pathologists., Funding: Swedish e-Science Research Center; VINNOVA; CENIIT career development program at Linkoping University; Wallenberg AI, WASP - Knut and Alice Wallenberg Foundation

Computational pathology, a developing area of primarily deep learning (DL) solutions aiming to aid pathologists at their daily tasks, has shown promising results in research settings. In recent years, uncertainty estimation has gained substantial recognition as having high potential to bring value to DL algorithms for medical applications. But it is not trivial how to incorporate it with a DL system to obtain a real positive impact. In this work we propose a framework to spatially aggregated epistemic uncertainty in order to detect false negatives produced by a segmentation algorithm of breast cancer metastases. We show a strong correlation between the false negative segmentation areas and the aggregated uncertainty values. Furthermore, the results include examples of reducing false negatives, where the uncertainty approach led to detection of some tumour metastases that had been missed., Funding Agencies|Swedish e-Science Research Center; VINNOVA [2017-02447]; Zenith career development program at Linkoping University; Autonomous Systems and Software Program (WASP) - Knut and Alice Wallenberg Foundation

Purpose: Multiple vendors are currently offering artificial intelligence (AI) computer-aided systems for triage detection, diagnosis, and risk prediction of breast cancer based on screening mammography. There is an imminent need to establish validation platforms that enable fair and transparent testing of these systems against external data.Approach: We developed validation of artificial intelligence for breast imaging (VAI-B), a platform for independent validation of AI algorithms in breast imaging. The platform is a hybrid solution, with one part implemented in the cloud and another in an on-premises environment at Karolinska Institute. Cloud services provide the flexibility of scaling the computing power during inference time, while secure on-premises clinical data storage preserves their privacy. A MongoDB database and a python package were developed to store and manage the data on-premises. VAI-B requires four data components: radiological images, AI inferences, radiologist assessments, and cancer outcomes.Results: To pilot test VAI-B, we defined a case-control population based on 8080 patients diagnosed with breast cancer and 36,339 healthy women based on the Swedish national quality registry for breast cancer. Images and radiological assessments from more than 100,000 mammography examinations were extracted from hospitals in three regions of Sweden. The images were processed by AI systems from three vendors in a virtual private cloud to produce abnormality scores related to signs of cancer in the images. A total of 105,706 examinations have been processed and stored in the database.Conclusions: We have created a platform that will allow downstream evaluation of AI systems for breast cancer detection, which enables faster development cycles for participating vendors and safer AI adoption for participating hospitals. The platform was designed to be scalable and ready to be expanded should a new vendor want to evaluate their system or should a new hospital wish, Funding: Regional Cancer Centers in Collaboration and Vinnova [21/00060, 2021-02617]; Medtechlabs, Stockholm, Sweden

The discussion on artificial intelligence (AI) solutions in diagnostic imaging has matured in recent years. The potential value of AI adoption is well established, as are the potential risks associated. Much focus has, rightfully, been on regulatory certification of AI products, with the strong incentive of being an enabling step for the commercial actors. It is, however, becoming evident that regulatory approval is not enough to ensure safe and effective AI usage in the local setting. In other words, care providers need to develop and implement quality assurance (QA) approaches for AI solutions in diagnostic imaging. The domain of AI-specific QA is still in an early development phase. We contribute to this development by describing the current landscape of QA-for-AI approaches in medical imaging, with focus on radiology and pathology. We map the potential quality threats and review the existing QA approaches in relation to those threats. We propose a practical categorization of QA approaches, based on key characteristics corresponding to means, situation, and purpose. The review highlights the heterogeneity of methods and practices relevant for this domain and points to targets for future research efforts., Funding Agencies|Linkoping University; Vinnova [2021-01420]

The discussion on artificial intelligence (AI) solutions in diagnostic imaging has matured in recent years. The potential value of AI adoption is well established, as are the potential risks associated. Much focus has, rightfully, been on regulatory certification of AI products, with the strong incentive of being an enabling step for the commercial actors. It is, however, becoming evident that regulatory approval is not enough to ensure safe and effective AI usage in the local setting. In other words, care providers need to develop and implement quality assurance (QA) approaches for AI solutions in diagnostic imaging. The domain of AI-specific QA is still in an early development phase. We contribute to this development by describing the current landscape of QA-for-AI approaches in medical imaging, with focus on radiology and pathology. We map the potential quality threats and review the existing QA approaches in relation to those threats. We propose a practical categorization of QA approaches, based on key characteristics corresponding to means, situation, and purpose. The review highlights the heterogeneity of methods and practices relevant for this domain and points to targets for future research efforts., Funding Agencies|Linkoping University; Vinnova [2021-01420]

The discussion on artificial intelligence (AI) solutions in diagnostic imaging has matured in recent years. The potential value of AI adoption is well established, as are the potential risks associated. Much focus has, rightfully, been on regulatory certification of AI products, with the strong incentive of being an enabling step for the commercial actors. It is, however, becoming evident that regulatory approval is not enough to ensure safe and effective AI usage in the local setting. In other words, care providers need to develop and implement quality assurance (QA) approaches for AI solutions in diagnostic imaging. The domain of AI-specific QA is still in an early development phase. We contribute to this development by describing the current landscape of QA-for-AI approaches in medical imaging, with focus on radiology and pathology. We map the potential quality threats and review the existing QA approaches in relation to those threats. We propose a practical categorization of QA approaches, based on key characteristics corresponding to means, situation, and purpose. The review highlights the heterogeneity of methods and practices relevant for this domain and points to targets for future research efforts., Funding Agencies|Linkoping University; Vinnova [2021-01420]

We present PLIS, a publicly available, open-source software for the determination of protein-ligand dissociation constants that can be used to characterize biological processes or to shed light on biophysical aspects of interactions. PLIS can analyze data from titration experiments monitored by for instance fluorescence spectroscopy or from nuclear magnetic resonance relaxation dispersion experiments. In addition to analysis of experimental data, PLIS includes functionality for generation of synthetic data, useful for understanding how different parameters effect the data in order to better analyze experiments.

Deep learning (DL) has shown great potential in digital pathology applications. The robustness of a diagnostic DL-based solution is essential for safe clinical deployment. In this work we evaluate if adding uncertainty estimates for DL predictions in digital pathology could result in increased value for the clinical applications, by boosting the general predictive performance or by detecting mispredictions. We compare the effectiveness of model-integrated methods (MC dropout and Deep ensembles) with a model-agnostic approach (Test time augmentation, TTA). Moreover, four uncertainty metrics are compared. Our experiments focus on two domain shift scenarios: a shift to a different medical center and to an underrepresented subtype of cancer. Our results show that uncertainty estimates increase reliability by reducing a models sensitivity to classification threshold selection as well as by detecting between 70 and 90% of the mispredictions done by the model. Overall, the deep ensembles method achieved the best performance closely followed by TTA., Funding Agencies|Swedish e-Science Research Center; VINNOVA [2017-02447]

Deep learning (DL) has shown great potential in digital pathology applications. The robustness of a diagnostic DL-based solution is essential for safe clinical deployment. In this work we evaluate if adding uncertainty estimates for DL predictions in digital pathology could result in increased value for the clinical applications, by boosting the general predictive performance or by detecting mispredictions. We compare the effectiveness of model-integrated methods (MC dropout and Deep ensembles) with a model-agnostic approach (Test time augmentation, TTA). Moreover, four uncertainty metrics are compared. Our experiments focus on two domain shift scenarios: a shift to a different medical center and to an underrepresented subtype of cancer. Our results show that uncertainty estimates increase reliability by reducing a models sensitivity to classification threshold selection as well as by detecting between 70 and 90% of the mispredictions done by the model. Overall, the deep ensembles method achieved the best performance closely followed by TTA., Funding Agencies|Swedish e-Science Research Center; VINNOVA [2017-02447]

Unsupervised learning has made substantial progress over the last few years, especially by means of contrastive self-supervised learning. The dominating dataset for benchmarking self-supervised learning has been ImageNet, for which recent methods are approaching the performance achieved by fully supervised training. The ImageNet dataset is however largely object-centric, and it is not clear yet what potential those methods have on widely different datasets and tasks that are not object-centric, such as in digital pathology.While self-supervised learning has started to be explored within this area with encouraging results, there is reason to look closer at how this setting differs from natural images and ImageNet. In this paper we make an in-depth analysis of contrastive learning for histopathology, pin-pointing how the contrastive objective will behave differently due to the characteristics of histopathology data. Using SimCLR and H&E stained images as a representative setting for contrastive self-supervised learning in histopathology, we bring forward a number of considerations, such as view generation for the contrastive objectiveand hyper-parameter tuning. In a large battery of experiments, we analyze how the downstream performance in tissue classification will be affected by these considerations. The results point to how contrastive learning can reduce the annotation effort within digital pathology, but that the specific dataset characteristics need to be considered. To take full advantage of the contrastive learning objective, different calibrations of view generation and hyper-parameters are required. Our results pave the way for realizing the full potential of self-supervised learning for histopathology applications. Code and trained models are available at https://github.com/k-stacke/ssl-pathology.

Unsupervised learning has made substantial progress over the last few years, especially by means of contrastive self-supervised learning. The dominating dataset for benchmarking self-supervised learning has been ImageNet, for which recent methods are approaching the performance achieved by fully supervised training. The ImageNet dataset is however largely object-centric, and it is not clear yet what potential those methods have on widely different datasets and tasks that are not object-centric, such as in digital pathology.While self-supervised learning has started to be explored within this area with encouraging results, there is reason to look closer at how this setting differs from natural images and ImageNet. In this paper we make an in-depth analysis of contrastive learning for histopathology, pin-pointing how the contrastive objective will behave differently due to the characteristics of histopathology data. Using SimCLR and H&E stained images as a representative setting for contrastive self-supervised learning in histopathology, we bring forward a number of considerations, such as view generation for the contrastive objectiveand hyper-parameter tuning. In a large battery of experiments, we analyze how the downstream performance in tissue classification will be affected by these considerations. The results point to how contrastive learning can reduce the annotation effort within digital pathology, but that the specific dataset characteristics need to be considered. To take full advantage of the contrastive learning objective, different calibrations of view generation and hyper-parameters are required. Our results pave the way for realizing the full potential of self-supervised learning for histopathology applications. Code and trained models are available at https://github.com/k-stacke/ssl-pathology.

Deep learning (DL) has shown great potential in digital pathology applications. The robustness of a diagnostic DL-based solution is essential for safe clinical deployment. In this work we evaluate if adding uncertainty estimates for DL predictions in digital pathology could result in increased value for the clinical applications, by boosting the general predictive performance or by detecting mispredictions. We compare the effectiveness of model-integrated methods (MC dropout and Deep ensembles) with a model-agnostic approach (Test time augmentation, TTA). Moreover, four uncertainty metrics are compared. Our experiments focus on two domain shift scenarios: a shift to a different medical center and to an underrepresented subtype of cancer. Our results show that uncertainty estimates increase reliability by reducing a models sensitivity to classification threshold selection as well as by detecting between 70 and 90% of the mispredictions done by the model. Overall, the deep ensembles method achieved the best performance closely followed by TTA., Funding Agencies|Swedish e-Science Research Center; VINNOVA [2017-02447]

We present PLIS, a publicly available, open-source software for the determination of protein-ligand dissociation constants that can be used to characterize biological processes or to shed light on biophysical aspects of interactions. PLIS can analyze data from titration experiments monitored by for instance fluorescence spectroscopy or from nuclear magnetic resonance relaxation dispersion experiments. In addition to analysis of experimental data, PLIS includes functionality for generation of synthetic data, useful for understanding how different parameters effect the data in order to better analyze experiments.

We present PLIS, a publicly available, open-source software for the determination of protein-ligand dissociation constants that can be used to characterize biological processes or to shed light on biophysical aspects of interactions. PLIS can analyze data from titration experiments monitored by for instance fluorescence spectroscopy or from nuclear magnetic resonance relaxation dispersion experiments. In addition to analysis of experimental data, PLIS includes functionality for generation of synthetic data, useful for understanding how different parameters effect the data in order to better analyze experiments.

Simple Summary Pathology is a cornerstone in cancer diagnostics, and digital pathology and artificial intelligence-driven image analysis could potentially save time and enhance diagnostic accuracy. For clinical implementation of artificial intelligence, a major question is whether the computer models maintain high performance when applied to new settings. We tested the generalizability of a highly accurate deep learning model for breast cancer metastasis detection in sentinel lymph nodes from, firstly, unseen sentinel node data and, secondly, data with a small change in surgical indication, in this case lymph nodes from axillary dissections. Model performance dropped in both settings, particularly on axillary dissection nodes. Retraining of the model was needed to mitigate the performance drop. The study highlights the generalization challenge of clinical implementation of AI models, and the possibility that retraining might be necessary. Poor generalizability is a major barrier to clinical implementation of artificial intelligence in digital pathology. The aim of this study was to test the generalizability of a pretrained deep learning model to a new diagnostic setting and to a small change in surgical indication. A deep learning model for breast cancer metastases detection in sentinel lymph nodes, trained on CAMELYON multicenter data, was used as a base model, and achieved an AUC of 0.969 (95% CI 0.926-0.998) and FROC of 0.838 (95% CI 0.757-0.913) on CAMELYON16 test data. On local sentinel node data, the base model performance dropped to AUC 0.929 (95% CI 0.800-0.998) and FROC 0.744 (95% CI 0.566-0.912). On data with a change in surgical indication (axillary dissections) the base model performance indicated an even larger drop with a FROC of 0.503 (95%CI 0.201-0.911). The model was retrained with addition of local data, resulting in about a 4% increase for both AUC and FROC for sentinel nodes, and an increase of 11% in AUC and 49% in FROC for axillary nodes. Patho, Funding Agencies|Vinnova [2017-02447]

Therapeutic monoclonal antibodies (mAbs) provide new means for treatments of a wide range of diseases and comprise a large fraction of all new approved drugs. Production of mAbs is expensive compared to conventional drug production, primarily due to the complex processes involved. The affinity purification step is dominating the cost of goods in mAb manufacturing. Process intensification and automation could reduce costs, but the lack of real-time process analytical technologies (PAT) complicates this development. We show a specific and robust fiber optical localized surface plasmon resonance (LSPR) sensor technology that is optimized for in-line product detection in the effluent in affinity capture steps. The sensor system comprises a flow cell and a replaceable sensor chip functionalized with biorecognition elements for specific analyte detection. The high selectivity of the sensor enable detection of mAbs in complex sample matrices at concentrations below 2.5 mu g mL(-1). In place regeneration of the sensor chips allowed for continuous monitoring of multiple consecutive chromatographic separation cycles. Excellent performance was obtained at different purification scales with flow rates up to 200 mL min(-1). This sensor technology facilitates efficient column loading, optimization, and control of chromatography systems, which can pave the way for continuous operation and automation of protein purification steps., Funding Agencies|European Unions Horizon 2020 research and innovation program under the Marie Skodowska-Curie grant [841373]; Swedish Innovation Agency (VINNOVA) [2016-04120, 2019-00130]

Simple Summary Pathology is a cornerstone in cancer diagnostics, and digital pathology and artificial intelligence-driven image analysis could potentially save time and enhance diagnostic accuracy. For clinical implementation of artificial intelligence, a major question is whether the computer models maintain high performance when applied to new settings. We tested the generalizability of a highly accurate deep learning model for breast cancer metastasis detection in sentinel lymph nodes from, firstly, unseen sentinel node data and, secondly, data with a small change in surgical indication, in this case lymph nodes from axillary dissections. Model performance dropped in both settings, particularly on axillary dissection nodes. Retraining of the model was needed to mitigate the performance drop. The study highlights the generalization challenge of clinical implementation of AI models, and the possibility that retraining might be necessary. Poor generalizability is a major barrier to clinical implementation of artificial intelligence in digital pathology. The aim of this study was to test the generalizability of a pretrained deep learning model to a new diagnostic setting and to a small change in surgical indication. A deep learning model for breast cancer metastases detection in sentinel lymph nodes, trained on CAMELYON multicenter data, was used as a base model, and achieved an AUC of 0.969 (95% CI 0.926-0.998) and FROC of 0.838 (95% CI 0.757-0.913) on CAMELYON16 test data. On local sentinel node data, the base model performance dropped to AUC 0.929 (95% CI 0.800-0.998) and FROC 0.744 (95% CI 0.566-0.912). On data with a change in surgical indication (axillary dissections) the base model performance indicated an even larger drop with a FROC of 0.503 (95%CI 0.201-0.911). The model was retrained with addition of local data, resulting in about a 4% increase for both AUC and FROC for sentinel nodes, and an increase of 11% in AUC and 49% in FROC for axillary nodes. Patho, Funding Agencies|Vinnova [2017-02447]

Therapeutic monoclonal antibodies (mAbs) provide new means for treatments of a wide range of diseases and comprise a large fraction of all new approved drugs. Production of mAbs is expensive compared to conventional drug production, primarily due to the complex processes involved. The affinity purification step is dominating the cost of goods in mAb manufacturing. Process intensification and automation could reduce costs, but the lack of real-time process analytical technologies (PAT) complicates this development. We show a specific and robust fiber optical localized surface plasmon resonance (LSPR) sensor technology that is optimized for in-line product detection in the effluent in affinity capture steps. The sensor system comprises a flow cell and a replaceable sensor chip functionalized with biorecognition elements for specific analyte detection. The high selectivity of the sensor enable detection of mAbs in complex sample matrices at concentrations below 2.5 mu g mL(-1). In place regeneration of the sensor chips allowed for continuous monitoring of multiple consecutive chromatographic separation cycles. Excellent performance was obtained at different purification scales with flow rates up to 200 mL min(-1). This sensor technology facilitates efficient column loading, optimization, and control of chromatography systems, which can pave the way for continuous operation and automation of protein purification steps., Funding Agencies|European Unions Horizon 2020 research and innovation program under the Marie Skodowska-Curie grant [841373]; Swedish Innovation Agency (VINNOVA) [2016-04120, 2019-00130]

Therapeutic monoclonal antibodies (mAbs) provide new means for treatments of a wide range of diseases and comprise a large fraction of all new approved drugs. Production of mAbs is expensive compared to conventional drug production, primarily due to the complex processes involved. The affinity purification step is dominating the cost of goods in mAb manufacturing. Process intensification and automation could reduce costs, but the lack of real-time process analytical technologies (PAT) complicates this development. We show a specific and robust fiber optical localized surface plasmon resonance (LSPR) sensor technology that is optimized for in-line product detection in the effluent in affinity capture steps. The sensor system comprises a flow cell and a replaceable sensor chip functionalized with biorecognition elements for specific analyte detection. The high selectivity of the sensor enable detection of mAbs in complex sample matrices at concentrations below 2.5 mu g mL(-1). In place regeneration of the sensor chips allowed for continuous monitoring of multiple consecutive chromatographic separation cycles. Excellent performance was obtained at different purification scales with flow rates up to 200 mL min(-1). This sensor technology facilitates efficient column loading, optimization, and control of chromatography systems, which can pave the way for continuous operation and automation of protein purification steps., Funding Agencies|European Unions Horizon 2020 research and innovation program under the Marie Skodowska-Curie grant [841373]; Swedish Innovation Agency (VINNOVA) [2016-04120, 2019-00130]

Simple Summary Pathology is a cornerstone in cancer diagnostics, and digital pathology and artificial intelligence-driven image analysis could potentially save time and enhance diagnostic accuracy. For clinical implementation of artificial intelligence, a major question is whether the computer models maintain high performance when applied to new settings. We tested the generalizability of a highly accurate deep learning model for breast cancer metastasis detection in sentinel lymph nodes from, firstly, unseen sentinel node data and, secondly, data with a small change in surgical indication, in this case lymph nodes from axillary dissections. Model performance dropped in both settings, particularly on axillary dissection nodes. Retraining of the model was needed to mitigate the performance drop. The study highlights the generalization challenge of clinical implementation of AI models, and the possibility that retraining might be necessary. Poor generalizability is a major barrier to clinical implementation of artificial intelligence in digital pathology. The aim of this study was to test the generalizability of a pretrained deep learning model to a new diagnostic setting and to a small change in surgical indication. A deep learning model for breast cancer metastases detection in sentinel lymph nodes, trained on CAMELYON multicenter data, was used as a base model, and achieved an AUC of 0.969 (95% CI 0.926-0.998) and FROC of 0.838 (95% CI 0.757-0.913) on CAMELYON16 test data. On local sentinel node data, the base model performance dropped to AUC 0.929 (95% CI 0.800-0.998) and FROC 0.744 (95% CI 0.566-0.912). On data with a change in surgical indication (axillary dissections) the base model performance indicated an even larger drop with a FROC of 0.503 (95%CI 0.201-0.911). The model was retrained with addition of local data, resulting in about a 4% increase for both AUC and FROC for sentinel nodes, and an increase of 11% in AUC and 49% in FROC for axillary nodes. Patho, Funding Agencies|Vinnova [2017-02447]

Unsupervised learning has made substantial progress over the last few years, especially by means of contrastive self-supervised learning. The dominating dataset for benchmarking self-supervised learning has been ImageNet, for which recent methods are approaching the performance achieved by fully supervised training. The ImageNet dataset is however largely object-centric, and it is not clear yet what potential those methods have on widely different datasets and tasks that are not object-centric, such as in digital pathology.While self-supervised learning has started to be explored within this area with encouraging results, there is reason to look closer at how this setting differs from natural images and ImageNet. In this paper we make an in-depth analysis of contrastive learning for histopathology, pin-pointing how the contrastive objective will behave differently due to the characteristics of histopathology data. Using SimCLR and H&E stained images as a representative setting for contrastive self-supervised learning in histopathology, we bring forward a number of considerations, such as view generation for the contrastive objectiveand hyper-parameter tuning. In a large battery of experiments, we analyze how the downstream performance in tissue classification will be affected by these considerations. The results point to how contrastive learning can reduce the annotation effort within digital pathology, but that the specific dataset characteristics need to be considered. To take full advantage of the contrastive learning objective, different calibrations of view generation and hyper-parameters are required. Our results pave the way for realizing the full potential of self-supervised learning for histopathology applications. Code and trained models are available at https://github.com/k-stacke/ssl-pathology.

Deep learning (DL) has shown great potential in digital pathology applications. The robustness of a diagnostic DL-based solution is essential for safe clinical deployment. In this work we evaluate if adding uncertainty estimates for DL predictions in digital pathology could result in increased value for the clinical applications, by boosting the general predictive performance or by detecting mispredictions. We compare the effectiveness of model-integrated methods (MC dropout and Deep ensembles) with a model-agnostic approach (Test time augmentation, TTA). Moreover, four uncertainty metrics are compared. Our experiments focus on two domain shift scenarios: a shift to a different medical center and to an underrepresented subtype of cancer. Our results show that uncertainty estimates increase reliability by reducing a models sensitivity to classification threshold selection as well as by detecting between 70 and 90% of the mispredictions done by the model. Overall, the deep ensembles method achieved the best performance closely followed by TTA., Funding Agencies|Swedish e-Science Research Center; VINNOVA [2017-02447]

We present PLIS, a publicly available, open-source software for the determination of protein-ligand dissociation constants that can be used to characterize biological processes or to shed light on biophysical aspects of interactions. PLIS can analyze data from titration experiments monitored by for instance fluorescence spectroscopy or from nuclear magnetic resonance relaxation dispersion experiments. In addition to analysis of experimental data, PLIS includes functionality for generation of synthetic data, useful for understanding how different parameters effect the data in order to better analyze experiments.

Therapeutic monoclonal antibodies (mAbs) provide new means for treatments of a wide range of diseases and comprise a large fraction of all new approved drugs. Production of mAbs is expensive compared to conventional drug production, primarily due to the complex processes involved. The affinity purification step is dominating the cost of goods in mAb manufacturing. Process intensification and automation could reduce costs, but the lack of real-time process analytical technologies (PAT) complicates this development. We show a specific and robust fiber optical localized surface plasmon resonance (LSPR) sensor technology that is optimized for in-line product detection in the effluent in affinity capture steps. The sensor system comprises a flow cell and a replaceable sensor chip functionalized with biorecognition elements for specific analyte detection. The high selectivity of the sensor enable detection of mAbs in complex sample matrices at concentrations below 2.5 mu g mL(-1). In place regeneration of the sensor chips allowed for continuous monitoring of multiple consecutive chromatographic separation cycles. Excellent performance was obtained at different purification scales with flow rates up to 200 mL min(-1). This sensor technology facilitates efficient column loading, optimization, and control of chromatography systems, which can pave the way for continuous operation and automation of protein purification steps., Funding Agencies|European Unions Horizon 2020 research and innovation program under the Marie Skodowska-Curie grant [841373]; Swedish Innovation Agency (VINNOVA) [2016-04120, 2019-00130]

Simple Summary Pathology is a cornerstone in cancer diagnostics, and digital pathology and artificial intelligence-driven image analysis could potentially save time and enhance diagnostic accuracy. For clinical implementation of artificial intelligence, a major question is whether the computer models maintain high performance when applied to new settings. We tested the generalizability of a highly accurate deep learning model for breast cancer metastasis detection in sentinel lymph nodes from, firstly, unseen sentinel node data and, secondly, data with a small change in surgical indication, in this case lymph nodes from axillary dissections. Model performance dropped in both settings, particularly on axillary dissection nodes. Retraining of the model was needed to mitigate the performance drop. The study highlights the generalization challenge of clinical implementation of AI models, and the possibility that retraining might be necessary. Poor generalizability is a major barrier to clinical implementation of artificial intelligence in digital pathology. The aim of this study was to test the generalizability of a pretrained deep learning model to a new diagnostic setting and to a small change in surgical indication. A deep learning model for breast cancer metastases detection in sentinel lymph nodes, trained on CAMELYON multicenter data, was used as a base model, and achieved an AUC of 0.969 (95% CI 0.926-0.998) and FROC of 0.838 (95% CI 0.757-0.913) on CAMELYON16 test data. On local sentinel node data, the base model performance dropped to AUC 0.929 (95% CI 0.800-0.998) and FROC 0.744 (95% CI 0.566-0.912). On data with a change in surgical indication (axillary dissections) the base model performance indicated an even larger drop with a FROC of 0.503 (95%CI 0.201-0.911). The model was retrained with addition of local data, resulting in about a 4% increase for both AUC and FROC for sentinel nodes, and an increase of 11% in AUC and 49% in FROC for axillary nodes. Patho, Funding Agencies|Vinnova [2017-02447]

Production of therapeutic monoclonal antibodies (mAbs) is a complex process that requires extensive analytical and bioanalytical characterization to ensure high and consistent product quality. Aggregation of mAbs is common and very problematic and can result in products with altered pharmacodynamics and pharmacokinetics and potentially increased immunogenicity. Rapid detection of aggregates, however, remains very challenging using existing analytical techniques. Here, we show a real-time and label-free fiber optical nanoplasmonic biosensor system for specific detection and quantification of immunoglobulin G (IgG) aggregates exploiting Protein A mediated avidity effects. Compared to monomers, IgG aggregates were found to have substantially higher apparent affinity when binding to Protein Afunctionalized sensor chips in a specific pH range (pH 3.8-4.0). Under these conditions, aggregates and monomers showed significantly different binding and dissociation kinetics. Reliable and rapid aggregate quantification was demonstrated with a limit of detection (LOD) and limit of quantification (LOQ) of about 9 and 30 mu g/mL, respectively. Using neural network-based curve fitting, it was further possible to simultaneously quantify monomers and aggregates for aggregate concentrations lower than 30 mu g/mL. Our work demonstrates a unique avidity-based biosensor approach for fast aggregate analysis that can be used for rapid at-line quality control, including lot/batch release testing. This technology can also likely be further optimized for real-time in-line monitoring of product titers and quality, facilitating process intensification and automation., Funding Agencies|Swedish Innovation Agency (VINNOVA); Swedish Research Council [2016-04120, 2019-00130]; European Union [841373]

Simple Summary Pathology is a cornerstone in cancer diagnostics, and digital pathology and artificial intelligence-driven image analysis could potentially save time and enhance diagnostic accuracy. For clinical implementation of artificial intelligence, a major question is whether the computer models maintain high performance when applied to new settings. We tested the generalizability of a highly accurate deep learning model for breast cancer metastasis detection in sentinel lymph nodes from, firstly, unseen sentinel node data and, secondly, data with a small change in surgical indication, in this case lymph nodes from axillary dissections. Model performance dropped in both settings, particularly on axillary dissection nodes. Retraining of the model was needed to mitigate the performance drop. The study highlights the generalization challenge of clinical implementation of AI models, and the possibility that retraining might be necessary. Poor generalizability is a major barrier to clinical implementation of artificial intelligence in digital pathology. The aim of this study was to test the generalizability of a pretrained deep learning model to a new diagnostic setting and to a small change in surgical indication. A deep learning model for breast cancer metastases detection in sentinel lymph nodes, trained on CAMELYON multicenter data, was used as a base model, and achieved an AUC of 0.969 (95% CI 0.926-0.998) and FROC of 0.838 (95% CI 0.757-0.913) on CAMELYON16 test data. On local sentinel node data, the base model performance dropped to AUC 0.929 (95% CI 0.800-0.998) and FROC 0.744 (95% CI 0.566-0.912). On data with a change in surgical indication (axillary dissections) the base model performance indicated an even larger drop with a FROC of 0.503 (95%CI 0.201-0.911). The model was retrained with addition of local data, resulting in about a 4% increase for both AUC and FROC for sentinel nodes, and an increase of 11% in AUC and 49% in FROC for axillary nodes. Patho, Funding Agencies|Vinnova [2017-02447]

Therapeutic monoclonal antibodies (mAbs) provide new means for treatments of a wide range of diseases and comprise a large fraction of all new approved drugs. Production of mAbs is expensive compared to conventional drug production, primarily due to the complex processes involved. The affinity purification step is dominating the cost of goods in mAb manufacturing. Process intensification and automation could reduce costs, but the lack of real-time process analytical technologies (PAT) complicates this development. We show a specific and robust fiber optical localized surface plasmon resonance (LSPR) sensor technology that is optimized for in-line product detection in the effluent in affinity capture steps. The sensor system comprises a flow cell and a replaceable sensor chip functionalized with biorecognition elements for specific analyte detection. The high selectivity of the sensor enable detection of mAbs in complex sample matrices at concentrations below 2.5 mu g mL(-1). In place regeneration of the sensor chips allowed for continuous monitoring of multiple consecutive chromatographic separation cycles. Excellent performance was obtained at different purification scales with flow rates up to 200 mL min(-1). This sensor technology facilitates efficient column loading, optimization, and control of chromatography systems, which can pave the way for continuous operation and automation of protein purification steps., Funding Agencies|European Unions Horizon 2020 research and innovation program under the Marie Skodowska-Curie grant [841373]; Swedish Innovation Agency (VINNOVA) [2016-04120, 2019-00130]

Unsupervised learning has made substantial progress over the last few years, especially by means of contrastive self-supervised learning. The dominating dataset for benchmarking self-supervised learning has been ImageNet, for which recent methods are approaching the performance achieved by fully supervised training. The ImageNet dataset is however largely object-centric, and it is not clear yet what potential those methods have on widely different datasets and tasks that are not object-centric, such as in digital pathology.While self-supervised learning has started to be explored within this area with encouraging results, there is reason to look closer at how this setting differs from natural images and ImageNet. In this paper we make an in-depth analysis of contrastive learning for histopathology, pin-pointing how the contrastive objective will behave differently due to the characteristics of histopathology data. Using SimCLR and H&E stained images as a representative setting for contrastive self-supervised learning in histopathology, we bring forward a number of considerations, such as view generation for the contrastive objectiveand hyper-parameter tuning. In a large battery of experiments, we analyze how the downstream performance in tissue classification will be affected by these considerations. The results point to how contrastive learning can reduce the annotation effort within digital pathology, but that the specific dataset characteristics need to be considered. To take full advantage of the contrastive learning objective, different calibrations of view generation and hyper-parameters are required. Our results pave the way for realizing the full potential of self-supervised learning for histopathology applications. Code and trained models are available at https://github.com/k-stacke/ssl-pathology.

Unsupervised learning has made substantial progress over the last few years, especially by means of contrastive self-supervised learning. The dominating dataset for benchmarking self-supervised learning has been ImageNet, for which recent methods are approaching the performance achieved by fully supervised training. The ImageNet dataset is however largely object-centric, and it is not clear yet what potential those methods have on widely different datasets and tasks that are not object-centric, such as in digital pathology.While self-supervised learning has started to be explored within this area with encouraging results, there is reason to look closer at how this setting differs from natural images and ImageNet. In this paper we make an in-depth analysis of contrastive learning for histopathology, pin-pointing how the contrastive objective will behave differently due to the characteristics of histopathology data. Using SimCLR and H&E stained images as a representative setting for contrastive self-supervised learning in histopathology, we bring forward a number of considerations, such as view generation for the contrastive objectiveand hyper-parameter tuning. In a large battery of experiments, we analyze how the downstream performance in tissue classification will be affected by these considerations. The results point to how contrastive learning can reduce the annotation effort within digital pathology, but that the specific dataset characteristics need to be considered. To take full advantage of the contrastive learning objective, different calibrations of view generation and hyper-parameters are required. Our results pave the way for realizing the full potential of self-supervised learning for histopathology applications. Code and trained models are available at https://github.com/k-stacke/ssl-pathology.

Deep learning (DL) has shown great potential in digital pathology applications. The robustness of a diagnostic DL-based solution is essential for safe clinical deployment. In this work we evaluate if adding uncertainty estimates for DL predictions in digital pathology could result in increased value for the clinical applications, by boosting the general predictive performance or by detecting mispredictions. We compare the effectiveness of model-integrated methods (MC dropout and Deep ensembles) with a model-agnostic approach (Test time augmentation, TTA). Moreover, four uncertainty metrics are compared. Our experiments focus on two domain shift scenarios: a shift to a different medical center and to an underrepresented subtype of cancer. Our results show that uncertainty estimates increase reliability by reducing a models sensitivity to classification threshold selection as well as by detecting between 70 and 90% of the mispredictions done by the model. Overall, the deep ensembles method achieved the best performance closely followed by TTA., Funding Agencies|Swedish e-Science Research Center; VINNOVA [2017-02447]

Deep learning (DL) has shown great potential in digital pathology applications. The robustness of a diagnostic DL-based solution is essential for safe clinical deployment. In this work we evaluate if adding uncertainty estimates for DL predictions in digital pathology could result in increased value for the clinical applications, by boosting the general predictive performance or by detecting mispredictions. We compare the effectiveness of model-integrated methods (MC dropout and Deep ensembles) with a model-agnostic approach (Test time augmentation, TTA). Moreover, four uncertainty metrics are compared. Our experiments focus on two domain shift scenarios: a shift to a different medical center and to an underrepresented subtype of cancer. Our results show that uncertainty estimates increase reliability by reducing a models sensitivity to classification threshold selection as well as by detecting between 70 and 90% of the mispredictions done by the model. Overall, the deep ensembles method achieved the best performance closely followed by TTA., Funding Agencies|Swedish e-Science Research Center; VINNOVA [2017-02447]

We present PLIS, a publicly available, open-source software for the determination of protein-ligand dissociation constants that can be used to characterize biological processes or to shed light on biophysical aspects of interactions. PLIS can analyze data from titration experiments monitored by for instance fluorescence spectroscopy or from nuclear magnetic resonance relaxation dispersion experiments. In addition to analysis of experimental data, PLIS includes functionality for generation of synthetic data, useful for understanding how different parameters effect the data in order to better analyze experiments.

Purpose While computational methods are prevalent in aircraft conceptual design, recent advances in mechatronics and manufacturing are lowering the cost of practical experiments. Focussing on a relatively simple property, the lift curve, this study aims to increase understanding of how basic aerodynamic characteristics of a complex stealth configuration can be estimated experimentally using low-cost equipment, rapid prototyping methods and remotely piloted aircraft. Design/methodology/approach Lift curve estimates are obtained from a wind tunnel test of a three-dimensional-printed, 3.8%-scale model of a generic fighter and from flight testing a 14%-scale demonstrator using both a simple and a more advanced identification technique based on neural networks. These results are compared to a computational fluid dynamics study, a panel method and a straightforward, theoretical approach based on radical geometry simplifications. Findings Besides a good agreement in the linear region, discrepancies at high angles of attack reveal the shortcomings of each method. The remotely piloted model manages to provide consistent results beyond the physical limitations of the wind tunnel although it seems limited by instrumentation capabilities and unmodelled thrust effects. Practical implications Physical models can, even though low-cost experiments, expand the capabilities of other aerodynamic tools and contribute to reducing uncertainty when other estimations diverge. Originality/value This study highlights the limitations of commonly used aerodynamic methods and shows how low-cost prototyping and testing can complement or validate other estimations in the early study of a complex configuration., Funding Agencies|CoordenacAo de Aperfeicoamento de Pessoal de Nivel SuperiorCoordenacao de Aperfeicoamento de Pessoal de Nivel Superior (CAPES) [001]; VINNOVA [MESTA]Vinnova [2017-01502]

We present PLIS, a publicly available, open-source software for the determination of protein-ligand dissociation constants that can be used to characterize biological processes or to shed light on biophysical aspects of interactions. PLIS can analyze data from titration experiments monitored by for instance fluorescence spectroscopy or from nuclear magnetic resonance relaxation dispersion experiments. In addition to analysis of experimental data, PLIS includes functionality for generation of synthetic data, useful for understanding how different parameters effect the data in order to better analyze experiments.

The high capacity of neural networks allows fitting models to data with high precision, but makes generalization to unseen data a challenge. If a domain shift exists, i.e. differences in image statistics between training and test data, care needs to be taken to ensure reliable deployment in real-world scenarios. In digital pathology, domain shift can be manifested in differences between whole-slide images, introduced by for example differences in acquisition pipeline - between medical centers or over time. In order to harness the great potential presented by deep learning in histopathology, and ensure consistent model behavior, we need a deeper understanding of domain shift and its consequences, such that a model's predictions on new data can be trusted. This work focuses on the internal representation learned by trained convolutional neural networks, and shows how this can be used to formulate a novel measure - the representation shift - for quantifying the magnitude of model specific domain shift. We perform a study on domain shift in tumor classification of hematoxylin and eosin stained images, by considering different datasets, models, and techniques for preparing data in order to reduce the domain shift. The results show how the proposed measure has a high correlation with drop in performance when testing a model across a large number of different types of domain shifts, and how it improves on existing techniques for measuring data shift and uncertainty. The proposed measure can reveal how sensitive a model is to domain variations, and can be used to detect new data that a model will have problems generalizing to. We see techniques for measuring, understanding and overcoming the domain shift as a crucial step towards reliable use of deep learning in the future clinical pathology applications., Funding: Wallenberg AI and Autonomous Systems and Software Program (WASP-AI); research environment ELLIIT; AIDA VinnovaVinnova [2017-02447]

Artificial intelligence (AI) holds much promise for enabling highly desired imaging diagnostics improvements. One of the most limiting bottlenecks for the development of useful clinical-grade AI models is the lack of training data. One aspect is the large amount of cases needed and another is the necessity of high-quality ground truth annotation. The aim of the project was to establish and describe the construction of a database with substantial amounts of detail-annotated oncology imaging data from pathology and radiology. A specific objective was to be proactive, that is, to support undefined subsequent AI training across a wide range of tasks, such as detection, quantification, segmentation, and classification, which puts particular focus on the quality and generality of the annotations. The main outcome of this project was the database as such, with a collection of labeled image data from breast, ovary, skin, colon, skeleton, and liver. In addition, this effort also served as an exploration of best practices for further scalability of high-quality image collections, and a main contribution of the study was generic lessons learned regarding how to successfully organize efforts to construct medical imaging databases for AI training, summarized as eight guiding principles covering team, process, and execution aspects., Funding Agencies|Linkoping University; Visual Sweden [VS1702]

The high capacity of neural networks allows fitting models to data with high precision, but makes generalization to unseen data a challenge. If a domain shift exists, i.e. differences in image statistics between training and test data, care needs to be taken to ensure reliable deployment in real-world scenarios. In digital pathology, domain shift can be manifested in differences between whole-slide images, introduced by for example differences in acquisition pipeline - between medical centers or over time. In order to harness the great potential presented by deep learning in histopathology, and ensure consistent model behavior, we need a deeper understanding of domain shift and its consequences, such that a model's predictions on new data can be trusted. This work focuses on the internal representation learned by trained convolutional neural networks, and shows how this can be used to formulate a novel measure - the representation shift - for quantifying the magnitude of model specific domain shift. We perform a study on domain shift in tumor classification of hematoxylin and eosin stained images, by considering different datasets, models, and techniques for preparing data in order to reduce the domain shift. The results show how the proposed measure has a high correlation with drop in performance when testing a model across a large number of different types of domain shifts, and how it improves on existing techniques for measuring data shift and uncertainty. The proposed measure can reveal how sensitive a model is to domain variations, and can be used to detect new data that a model will have problems generalizing to. We see techniques for measuring, understanding and overcoming the domain shift as a crucial step towards reliable use of deep learning in the future clinical pathology applications., Funding: Wallenberg AI and Autonomous Systems and Software Program (WASP-AI); research environment ELLIIT; AIDA VinnovaVinnova [2017-02447]

Artificial intelligence (AI) holds much promise for enabling highly desired imaging diagnostics improvements. One of the most limiting bottlenecks for the development of useful clinical-grade AI models is the lack of training data. One aspect is the large amount of cases needed and another is the necessity of high-quality ground truth annotation. The aim of the project was to establish and describe the construction of a database with substantial amounts of detail-annotated oncology imaging data from pathology and radiology. A specific objective was to be proactive, that is, to support undefined subsequent AI training across a wide range of tasks, such as detection, quantification, segmentation, and classification, which puts particular focus on the quality and generality of the annotations. The main outcome of this project was the database as such, with a collection of labeled image data from breast, ovary, skin, colon, skeleton, and liver. In addition, this effort also served as an exploration of best practices for further scalability of high-quality image collections, and a main contribution of the study was generic lessons learned regarding how to successfully organize efforts to construct medical imaging databases for AI training, summarized as eight guiding principles covering team, process, and execution aspects., Funding Agencies|Linkoping University; Visual Sweden [VS1702]

The high capacity of neural networks allows fitting models to data with high precision, but makes generalization to unseen data a challenge. If a domain shift exists, i.e. differences in image statistics between training and test data, care needs to be taken to ensure reliable deployment in real-world scenarios. In digital pathology, domain shift can be manifested in differences between whole-slide images, introduced by for example differences in acquisition pipeline - between medical centers or over time. In order to harness the great potential presented by deep learning in histopathology, and ensure consistent model behavior, we need a deeper understanding of domain shift and its consequences, such that a model's predictions on new data can be trusted. This work focuses on the internal representation learned by trained convolutional neural networks, and shows how this can be used to formulate a novel measure - the representation shift - for quantifying the magnitude of model specific domain shift. We perform a study on domain shift in tumor classification of hematoxylin and eosin stained images, by considering different datasets, models, and techniques for preparing data in order to reduce the domain shift. The results show how the proposed measure has a high correlation with drop in performance when testing a model across a large number of different types of domain shifts, and how it improves on existing techniques for measuring data shift and uncertainty. The proposed measure can reveal how sensitive a model is to domain variations, and can be used to detect new data that a model will have problems generalizing to. We see techniques for measuring, understanding and overcoming the domain shift as a crucial step towards reliable use of deep learning in the future clinical pathology applications., Funding: Wallenberg AI and Autonomous Systems and Software Program (WASP-AI); research environment ELLIIT; AIDA VinnovaVinnova [2017-02447]

Designing useful human-AI interaction for clinical workflows remains challenging despite the impressive performance of recent AI models. One specific difficulty is a lack of successful examples demonstrating how to achieve safe and efficient workflows while mitigating AI imperfections. In this paper, we present an interactive AI-powered visual search tool that supports pathologists in cancer assessments. Our evaluation with six pathologists demonstrates that it can 1) reduce time needed with maintained quality, 2) build user trust progressively, and 3) learn and improve from use. We describe our iterative design process, model development, and key features. Through interviews, design choices are related to the overall user experience. Implications for future human-AI interaction design are discussed with respect to trust, explanations, learning from use, and collaboration strategies., Funding: Autonomous Systems and Software Program (WASP)