Back to Search
Start Over
Transferability of neural network clinical deidentification systems
- Source :
- J Am Med Inform Assoc
- Publication Year :
- 2021
- Publisher :
- Oxford University Press (OUP), 2021.
-
Abstract
- Objective Neural network deidentification studies have focused on individual datasets. These studies assume the availability of a sufficient amount of human-annotated data to train models that can generalize to corresponding test data. In real-world situations, however, researchers often have limited or no in-house training data. Existing systems and external data can help jump-start deidentification on in-house data; however, the most efficient way of utilizing existing systems and external data is unclear. This article investigates the transferability of a state-of-the-art neural clinical deidentification system, NeuroNER, across a variety of datasets, when it is modified architecturally for domain generalization and when it is trained strategically for domain transfer. Materials and Methods We conducted a comparative study of the transferability of NeuroNER using 4 clinical note corpora with multiple note types from 2 institutions. We modified NeuroNER architecturally to integrate 2 types of domain generalization approaches. We evaluated each architecture using 3 training strategies. We measured transferability from external sources; transferability across note types; the contribution of external source data when in-domain training data are available; and transferability across institutions. Results and Conclusions Transferability from a single external source gave inconsistent results. Using additional external sources consistently yielded an F1-score of approximately 80%. Fine-tuning emerged as a dominant transfer strategy, with or without domain generalization. We also found that external sources were useful even in cases where in-domain training data were available. Transferability across institutions differed by note type and annotation label but resulted in improved performance.
- Subjects :
- Artificial neural network
Computer science
Generalization
business.industry
Health Informatics
Research and Applications
Machine learning
computer.software_genre
Variety (cybernetics)
Domain (software engineering)
Deidentification
Annotation
Data Anonymization
Humans
Generalizability theory
Neural Networks, Computer
Artificial intelligence
business
computer
Test data
Subjects
Details
- ISSN :
- 1527974X
- Volume :
- 28
- Database :
- OpenAIRE
- Journal :
- Journal of the American Medical Informatics Association
- Accession number :
- edsair.doi.dedup.....b4c602acc59623af129f25901ed49ec8