Search

Rationale and objectives: Defacing research MRI brain scans is often a mandatory step. With current defacing software, there are issues with Windows compatibility and researcher doubt regarding the adequacy of preservation of brain voxels in non-T1w scans. To address this, we developed PyFaceWipe, a multiplatform software for multiple MRI contrasts, which was evaluated based on its anonymisation ability and effect on downstream processing. Materials and methods: Multiple MRI brain scan contrasts from the OASIS-3 dataset were defaced with PyFaceWipe and PyDeface and manually assessed for brain voxel preservation, remnant facial features and effect on automated face detection. Original and PyFaceWipe-defaced data from locally acquired T1w structural scans underwent volumetry with FastSurfer and brain atlas generation with ANTS. Results: 214 MRI scans of several contrasts from OASIS-3 were successfully processed with both PyFaceWipe and PyDeface. PyFaceWipe maintained complete brain voxel preservation in all tested contrasts except ASL (45%) and DWI (90%), and PyDeface in all tested contrasts except ASL (95%), BOLD (25%), DWI (40%) and T2* (25%). Manual review of PyFaceWipe showed no failures of facial feature removal. Pinna removal was less successful (6% of T1 scans showed residual complete pinna). PyDeface achieved 5.1% failure rate. Automated detection found no faces in PyFaceWipe-defaced scans, 19 faces in PyDeface scans compared with 78 from the 224 original scans. Brain atlas generation showed no significant difference between atlases created from original and defaced data in both young adulthood and late elderly cohorts. Structural volumetry dice scores were ≥ 0.98 for all structures except for grey matter which had 0.93. PyFaceWipe output was identical across the tested operating systems. Conclusion: PyFaceWipe is a promising multiplatform defacing tool, demonstrating excellent brain voxel preservation and competitive defacing in multiple MRI contrasts, performing favourably against PyDeface. ASL, BOLD, DWI and T2* scans did not produce recognisable 3D renders and hence should not require defacing. Structural volumetry dice scores (≥ 0.98) were higher than previously published FreeSurfer results, except for grey matter which were comparable. The effect is measurable and care should be exercised during studies. ANTS atlas creation showed no significant effect from PyFaceWipe defacing. [ABSTRACT FROM AUTHOR]

De-identification of DICOM images is an essential component of medical image research. While many established methods exist for the safe removal of protected health information (PHI) in DICOM metadata, approaches for the removal of PHI "burned-in" to image pixel data are typically manual, and automated high-throughput approaches are not well validated. Emerging optical character recognition (OCR) models can potentially detect and remove PHI-bearing text from medical images but are very time-consuming to run on the high volume of images found in typical research studies. We present a data processing method that performs metadata de-identification for all images combined with a targeted approach to only apply OCR to images with a high likelihood of burned-in text. The method was validated on a dataset of 415,182 images across ten modalities representative of the de-identification requests submitted at our institution over a 20-year span. Of the 12,578 images in this dataset with burned-in text of any kind, only 10 passed undetected with the method. OCR was only required for 6050 images (1.5% of the dataset). [ABSTRACT FROM AUTHOR]

A large number of images are available on the Internet with the growth of social networking services, and many of them are face photos or contain faces. It is necessary to protect the privacy of face images to prevent their malicious use by face image de-identification techniques that make face recognition difficult, which prevent the collection of specific face images using face recognition. In this paper, we propose a face image de-identification method that generates a de-identified image from an input face image by embedding facial features extracted from that of another person into the input face image. We develop the novel framework for embedding facial features into a face image and loss functions based on images and features to de-identify a face image preserving its appearance. Through a set of experiments using public face image datasets, we demonstrate that the proposed method exhibits higher de-identification performance against unknown face recognition models than conventional methods while preserving the appearance of the input face images. [ABSTRACT FROM AUTHOR]

De-identification is a process to prevent individuals from being identified from original transaction data by processing personal identification information. k-anonymization, which processes data so that at least k users have the same records, is one of the representative methods of de-identification. One of the methods of k-anonymization is adding dummy records into the data to protect users who have unique histories. For this method, the cost for k-anonymization is the difference in the number of records between the original data and the processed data, and it can be calculated only after deciding the parameter k and processing data. However, we want to calculate the cost before processing and find the optimal value of k because processing the big data with various k is very costly. In this paper, we propose a new model of transaction data that gives us a probability distribution and an expected value of values in data under the assumption that all values occur independently with uniform probability. Applying our data model, it is possible to evaluate the cost of k-anonymized data even before processing. [ABSTRACT FROM AUTHOR]

Head CT, which includes the facial region, can visualize faces using 3D reconstruction, raising concern that individuals may be identified. We developed a new de-identification technique that distorts the faces of head CT images. Head CT images that were distorted were labeled as "original images" and the others as "reference images." Reconstructed face models of both were created, with 400 control points on the facial surfaces. All voxel positions in the original image were moved and deformed according to the deformation vectors required to move to corresponding control points on the reference image. Three face detection and identification programs were used to determine face detection rates and match confidence scores. Intracranial volume equivalence tests were performed before and after deformation, and correlation coefficients between intracranial pixel value histograms were calculated. Output accuracy of the deep learning model for intracranial segmentation was determined using Dice Similarity Coefficient before and after deformation. The face detection rate was 100%, and match confidence scores were < 90. Equivalence testing of the intracranial volume revealed statistical equivalence before and after deformation. The median correlation coefficient between intracranial pixel value histograms before and after deformation was 0.9965, indicating high similarity. Dice Similarity Coefficient values of original and deformed images were statistically equivalent. We developed a technique to de-identify head CT images while maintaining the accuracy of deep-learning models. The technique involves deforming images to prevent face identification, with minimal changes to the original information. [ABSTRACT FROM AUTHOR]

Natural language processing (NLP) techniques for electronic health records have shown great potential to improve the quality of medical care. The text of radiology reports frequently constitutes a large fraction of EHR data, and can provide valuable information about patients' diagnoses, medical history, and imaging findings. The lack of a major public repository for radiological reports severely limits the development, testing, and application of new NLP tools. De-identification of protected health information (PHI) presents a major challenge to building such repositories, as many automated tools for de-identification were trained or designed for clinical notes and do not perform sufficiently well to build a public database of radiology reports. We developed and evaluated six ensemble models based on three publically available de-identification tools: MIT de-id, NeuroNER, and Philter. A set of 1023 reports was set aside as the testing partition. Two individuals with medical training annotated the test set for PHI; differences were resolved by consensus. Ensemble methods included simple voting schemes (1-Vote, 2-Votes, and 3-Votes), a decision tree, a naïve Bayesian classifier, and Adaboost boosting. The 1-Vote ensemble achieved recall of 998 / 1043 (95.7%); the 3-Votes ensemble had precision of 1035 / 1043 (99.2%). F1 scores were: 93.4% for the decision tree, 71.2% for the naïve Bayesian classifier, and 87.5% for the boosting method. Basic voting algorithms and machine learning classifiers incorporating the predictions of multiple tools can outperform each tool acting alone in de-identifying radiology reports. Ensemble methods hold substantial potential to improve automated de-identification tools for radiology reports to make such reports more available for research use to improve patient care and outcomes. [ABSTRACT FROM AUTHOR]

As healthcare data is extremely sensitive, it poses a risk of invading individuals' privacy if stored or exported without proper security measures. De-identification entails pseudonymization or anonymization of data, which are methods for temporarily or permanently removing an individual's identity. These methods are most suitable to keep user healthcare data private. Inference attacks are a commonly overlooked weakness of de-identification techniques. In this paper, I discuss a method for de-identifying Electronic Healthcare Records (EHR) using chained hashing to generate short-lived pseudonyms to reduce the impact of inference attacks, as well as a mechanism for re-identification based on information self-determination. It also removes the weaknesses of existing de-identification algorithms and resolve them by using appropriate real-time de-identification algorithm, Ephemeral Pseudonym Generation Algorithm (EPGA). [ABSTRACT FROM AUTHOR]

Background: Defacing has become mandatory for anonymization of brain MRI scans; however, concerns regarding data integrity were raised. Thus, we systematically evaluated the effect of different defacing procedures on automated brain atrophy estimation. Methods: In total, 268 Alzheimer's disease patients were included from ADNI, which included unaccelerated (n = 154), within-session unaccelerated repeat (n = 67) and accelerated 3D T1 imaging (n = 114). Atrophy maps were computed using the open-source software veganbagel for every original, unmodified scan and after defacing using afni_refacer, fsl_deface, mri_deface, mri_reface, PyDeface or spm_deface, and the root-mean-square error (RMSE) between z-scores was calculated. RMSE values derived from unaccelerated and unaccelerated repeat imaging served as a benchmark. Outliers were defined as RMSE > 75th percentile and by using Grubbs's test. Results: Benchmark RMSE was 0.28 ± 0.1 (range 0.12–0.58, 75th percentile 0.33). Outliers were found for unaccelerated and accelerated T1 imaging using the 75th percentile cutoff: afni_refacer (unaccelerated: 18, accelerated: 16), fsl_deface (unaccelerated: 4, accelerated: 18), mri_deface (unaccelerated: 0, accelerated: 15), mri_reface (unaccelerated: 0, accelerated: 2) and spm_deface (unaccelerated: 0, accelerated: 7). PyDeface performed best with no outliers (unaccelerated mean RMSE 0.08 ± 0.05, accelerated mean RMSE 0.07 ± 0.05). The following outliers were found according to Grubbs's test: afni_refacer (unaccelerated: 16, accelerated: 13), fsl_deface (unaccelerated: 10, accelerated: 21), mri_deface (unaccelerated: 7, accelerated: 20), mri_reface (unaccelerated: 7, accelerated: 6), PyDeface (unaccelerated: 5, accelerated: 8) and spm_deface (unaccelerated: 10, accelerated: 12). Conclusion: Most defacing approaches have an impact on atrophy estimation, especially in accelerated 3D T1 imaging. Only PyDeface showed good results with negligible impact on atrophy estimation. [ABSTRACT FROM AUTHOR]

A novel visual privacy protection (VPP) algorithm is proposed for camouflage of the visual identifiers in digital images. The challenge is to suffice the de-identification with a different look. This issue has been previously addressed by two classes of algorithms, i.e. modifying target features with or without a reference. The former ran well by finding a surrogate from randomized hybrid features between the target and the reference, and the latter made it by blurring or hiding the details of the region. Inspired by camouflage in animals for self-concealment, this paper presents a simple and efficient algorithm that imitates such behavior via adaptive histogram shift adjustment without a reference. The image is first blurred and segmented into several flat surfaces, then adaptively re-joined or split by shifts randomly chosen to produce the de-identified output. It is found that the success of the surface-based feature redaction depends on the shift diversity, the saliency, and coverage of the surfaces segmented, which are used as adjusting parameters for the camouflage. Extensive examples of real and synthetic images have demonstrated that our results compare favorably to those obtained by existing VPP methods, with required security, robustness, and selective reversibility. [ABSTRACT FROM AUTHOR]

There are various efforts in de-identifying patient's radiation oncology data for their uses in the advancement of research in medicine. Though the task of de-identification needs to be defined in the context of research goals and objectives, existing systems lack the flexibility of modeling data and normalization of names of attributes for accomplishing them. In this work, we describe a de-identification process of radiation and clinical oncology data, which is guided by a data model and a schema of dynamically capturing domain ontology and normalization of terminologies, defined in tune with the research goals in this area. The radiological images are obtained in DICOM format. It consists of diagnostic, radiation therapy (RT) treatment planning, RT verification, and RT response images. During the DICOM de-identification, a few crucial pieces of information are taken about the dataset. The proposed model is generic in organizing information modeling in sync with the de-identification of a patient's clinical information. The treatment and clinical data are provided in the comma-separated values (CSV) format, which follows a predefined data structure. The de-identified data is harmonized throughout the entire process. We have presented four specific case studies on four different types of cancers, namely glioblastoma multiforme, head–neck, breast, and lung. We also present experimental validation on a few patients' data in these four areas. A few aspects are taken care of during de-identification, such as preservation of longitudinal date changes (LDC), incremental de-identification, referential data integrity between the clinical and image data, de-identified data harmonization, and transformation of the data to an underlined database schema. [ABSTRACT FROM AUTHOR]

Privacy protection issues for resource description frameworks (RDFs) have emerged over the use of public government open data and the healthcare data of individuals. As these data may include personal information, they must undergo a de-identification process that deletes or replaces parts of the original data. To enable these protections, a method has been developed to apply k-anonymization to RDF data. However, sensitive RDF information anonymized using k-anonymization is not completely secure and is vulnerable to attacks. In this paper, we propose an l-diversity anatomy de-identification method that can overcome the limitations of k-anonymity and guarantee stronger privacy protection than k-anonymization. Further, as this data anonymization process is computationally time-intensive, we use Spark distributed computing to provide rapid de-identification to enhance its utility. We also propose l-diversity preservation for dynamically evolving RDF datasets. Experimental results show that our proposed distributed l-diversity algorithm processes the data more efficiently than conventional approaches. [ABSTRACT FROM AUTHOR]

We propose a de-identification system which runs in a standalone mode. The system takes care of the de-identification of radiation oncology patient's clinical and annotated imaging data including RTSTRUCT, RTPLAN, and RTDOSE. The clinical data consists of diagnosis, stages, outcome, and treatment information of the patient. The imaging data could be the diagnostic, therapy planning, and verification images. Archival of the longitudinal radiation oncology verification images like cone beam CT scans along with the initial imaging and clinical data are preserved in the process. During the de-identification, the system keeps the reference of original data identity in encrypted form. These could be useful for the re-identification if necessary. [ABSTRACT FROM AUTHOR]

Anonymization or de-identification techniques are methods for protecting the privacy of human subjects in sensitive data sets while preserving the utility of those data sets. In the case of health data, anonymization techniques may be used to remove or mask patient identities while allowing the health data content to be used by the medical and pharmaceutical research community. The efficacy of anonymization methods has come under repeated attacks and several researchers have shown that anonymized data can be re-identified to reveal the identity of the data subjects via approaches such as "linking." Nevertheless, even given these deficiencies, many government privacy policies depend on anonymization techniques as the primary approach to preserving privacy. In this report, we survey the anonymization landscape and consider the range of anonymization approaches that can be used to de-identify data containing personally identifiable information. We then review several notable government privacy policies that leverage anonymization. In particular, we review the European Union's General Data Protection Regulation (GDPR) and show that it takes a more goal-oriented approach to data privacy. It defines data privacy in terms of desired outcome (i.e., as a defense against risk of personal data disclosure), and is agnostic to the actual method of privacy preservation. And GDPR goes further to frame its privacy preservation regulations relative to the state of the art, the cost of implementation, the incurred risks, and the context of data processing. This has potential implications for the GDPR's robustness to future technological innovations - very much in contrast to privacy regulations that depend explicitly on more definite technical specifications. [ABSTRACT FROM AUTHOR]

Reusable, publicly available data is a pillar of open science and rapid advancement of cancer imaging research. Sharing data from completed research studies not only saves research dollars required to collect data, but also helps insure that studies are both replicable and reproducible. The Cancer Imaging Archive (TCIA) is a global shared repository for imaging data related to cancer. Insuring the consistency, scientific utility, and anonymity of data stored in TCIA is of utmost importance. As the rate of submission to TCIA has been increasing, both in volume and complexity of DICOM objects stored, the process of curation of collections has become a bottleneck in acquisition of data. In order to increase the rate of curation of image sets, improve the quality of the curation, and better track the provenance of changes made to submitted DICOM image sets, a custom set of tools was developed, using novel methods for the analysis of DICOM data sets. These tools are written in the programming language perl, use the open-source database PostgreSQL, make use of the perl DICOM routines in the open-source package Posda, and incorporate DICOM diagnostic tools from other open-source packages, such as dicom3tools. These tools are referred to as the “Posda Tools.” The Posda Tools are open source and available via git at https://github.com/UAMS-DBMI/PosdaTools. In this paper, we briefly describe the Posda Tools and discuss the novel methods employed by these tools to facilitate rapid analysis of DICOM data, including the following: (1) use a database schema which is more permissive, and differently normalized from traditional DICOM databases; (2) perform integrity checks automatically on a bulk basis; (3) apply revisions to DICOM datasets on an bulk basis, either through a web-based interface or via command line executable perl scripts; (4) all such edits are tracked in a revision tracker and may be rolled back; (5) a UI is provided to inspect the results of such edits, to verify that they are what was intended; (6) identification of DICOM Studies, Series, and SOP instances using “nicknames” which are persistent and have well-defined scope to make expression of reported DICOM errors easier to manage; and (7) rapidly identify potential duplicate DICOM datasets by pixel data is provided; this can be used, e.g., to identify submission subjects which may relate to the same individual, without identifying the individual. [ABSTRACT FROM AUTHOR]

Combining datasets into larger and separate datasets is becoming increasingly common, and personal identifiers are often removed in order to maintain participant anonymity. Views of research participants on the use of de-identified data in large research datasets are important for future projects, such as the Precision Medicine Initiative and Cancer Moonshot Initiative. This quantitative study set in the USA examines participant preferences and evaluates differences by demographics and cancer history. Study participants were recruited from the Northwest Cancer Genetics Registry and included cancer patients, their relatives, and controls. A secure online survey was administered to 450 participants. While the majority participants were not concerned about personal identification when participating in a genetic study using de-identified data, they expressed their concern that researchers protect their privacy and information. Most participants expressed a desire that their data should be available for as many research studies as possible, and in doing so, they would increase their chance of receiving personal health information. About 20% of participants felt that a link should not be maintained between the participant and their de-identified data. Reasons to maintain a link included an ability to return individual health results and an ability to support further research. Knowledge of participants' attitudes regarding the use of data into a research repository and the maintenance of a link to de-identified data is critical to the success of recruitment into future genomic research projects. [ABSTRACT FROM AUTHOR]

Clinical data sharing between healthcare institutions, and between practitioners is often hindered by privacy protection requirements. This problem is critical in collaborative scenarios where data sharing is fundamental for establishing a workflow among parties. The anonymization of patient information burned in DICOM images requires elaborate processes somewhat more complex than simple de-identification of textual information. Usually, before sharing, there is a need for manual removal of specific areas containing sensitive information in the images. In this paper, we present a pipeline for ultrasound medical image de-identification, provided as a free anonymization REST service for medical image applications, and a Software-as-a-Service to streamline automatic de-identification of medical images, which is freely available for end-users. The proposed approach applies image processing functions and machine-learning models to bring about an automatic system to anonymize medical images. To perform character recognition, we evaluated several machine-learning models, being Convolutional Neural Networks (CNN) selected as the best approach. For accessing the system quality, 500 processed images were manually inspected showing an anonymization rate of 89.2%. The tool can be accessed at and it is available with the most recent version of Google Chrome, Mozilla Firefox and Safari. A Docker image containing the proposed service is also publicly available for the community. [ABSTRACT FROM AUTHOR]

To investigate possible de-identification methodologies within the Cross-Enterprise Document Sharing for imaging (XDS-I) environment in order to provide strengthened support for image data exchange as part of clinical research projects. De-identification, using anonymization or pseudonymization, is the most common method to perform information removal within DICOM data. However, it is not a standard part of the XDS-I profiles. Different methodologies were observed to define how and where de-identification should take place within an XDS environment used for scientific research. De-identification service can be placed in three locations within the XDS-I framework: 1) within the Document Source, 2) between the Document Source and Document Consumer, and 3) within the Document Consumer. First method has a potential advantage with respect to the exposure of the images to outside systems but has drawbacks with respect to additional hardware and configuration requirements. Second and third method have big concern in exposing original documents with all identifiable data being intact after leaving the Document Source. De-identification within the Document Source has more advantages compared to the other methods. On the contrary, it is less recommended to perform de-identification within the Document Consumer since it has the highest risk of the exposure of patients identity due to the fact that images are exposed without de-identification during the transfers. [ABSTRACT FROM AUTHOR]

Data sharing is increasingly recognized as critical to cross-disciplinary research and to assuring scientific validity. Despite National Institutes of Health and National Science Foundation policies encouraging data sharing by grantees, little data sharing of clinical data has in fact occurred. A principal reason often given is the potential of inadvertent violation of the Health Insurance Portability and Accountability Act privacy regulations. While regulations specify the components of private health information that should be protected, there are no commonly accepted methods to de-identify clinical data objects such as images. This leads institutions to take conservative risk-averse positions on data sharing. In imaging trials, where images are coded according to the Digital Imaging and Communications in Medicine (DICOM) standard, the complexity of the data objects and the flexibility of the DICOM standard have made it especially difficult to meet privacy protection objectives. The recent release of DICOM Supplement 142 on image de-identification has removed much of this impediment. This article describes the development of an open-source software suite that implements DICOM Supplement 142 as part of the National Biomedical Imaging Archive (NBIA). It also describes the lessons learned by the authors as NBIA has acquired more than 20 image collections encompassing over 30 million images. [ABSTRACT FROM AUTHOR]

The objective of this study was to develop a method to hide information in a portable electronic health record (PEHR). In compliance with HIPAA guidelines, a hiding function for personal identifiers in a PEHR was implemented by recognizing and hiding techniques. The method emphasizes the feasibility of embedding a hiding function in a PEHR. The hiding function could be executed to hide identifiers in offline mode. The present study showed that embedding the hiding function in a PEHR is a practical way not only satisfies data confidentiability needs, but also meets patient’s personal privacy requirements. The effects of executing the hiding function would be the same as through de-identification or anonymization process. [ABSTRACT FROM AUTHOR]

From 2002–2004, the Lung Screening Study (LSS) of the National Lung Screening Trial (NLST) enrolled 34,614 participants, aged 55–74 years, at increased risk for lung cancer due to heavy cigarette smoking. Participants, randomized to standard chest X-ray (CXR) or computed tomography (CT) arms at ten screening centers, received up to three imaging screens for lung cancer at annual intervals. Participant medical histories and radiologist-interpreted screening results were transmitted to the LSS coordinating center, while all images were retained at local screening centers. From 2005–2007, all CT exams were uniformly de-identified and delivered to a central repository, the CT Image Library (CTIL), on external hard drives (94%) or CD/DVD (5.9%), or over a secure Internet connection (0.1%). Of 48,723 CT screens performed, only 176 (0.3%) were unavailable (lost, corrupted, compressed) while 48,547 (99.7%) were delivered to the CTIL. Described here is the experience organizing, implementing, and adapting the clinical-trial workflow surrounding the image retrieval, de-identification, delivery, and archiving of available LSS–NLST CT exams for the CTIL, together with the quality assurance procedures associated with those collection tasks. This collection of CT exams, obtained in a specific, well-defined participant population under a common protocol at evenly spaced intervals, and its attending demographic and clinical information, are now available to lung-disease investigators and developers of computer-aided-diagnosis algorithms. The approach to large scale, multi-center trial CT image collection detailed here may serve as a useful model, while the experience reported should be valuable in the planning and execution of future equivalent endeavors. [ABSTRACT FROM AUTHOR]

Medical image processing methods and algorithms, developed by researchers, need to be validated and tested. Test data would ideally be real clinical data especially that clinical data is varied and exists in large volumes. Nowadays, clinical data is accessible electronically and has important value for researchers. However, the usage of clinical data for research purposes should respect data confidentiality, patient right to privacy, and patient consent. In fact, clinical data is nominative given that it contains information about the patient such as name, age, and identification number. Evidently, clinical data needs to be de-identified to be exported to research databases. However, the same patient is usually followed during a long period of time. The disease progression and the diagnostic evolution represent extremely valuable information for researchers as well. Our objective is to build a research database from de-identified clinical data while enabling the data set to be easily incremented by exporting new pseudonymous data, acquired over a long period of time. Pseudonymization is data de-identification, such that data belonging to an individual in the clinical environment still belong to the same individual in the de-identified research version. In this paper, we explore various software architectures to enable the implementation of an imaging research database that can be incremented in time. We also evaluate their security and discuss their security pitfalls. As most imaging data accessible electronically is available with the digital imaging and communication in medicine (DICOM) standard, we propose a de-identification scheme that closely follows DICOM recommendations. Our work can be used to enable electronic health record (EHR) secondary usage such as public surveillance and research, while maintaining patient confidentiality. [ABSTRACT FROM AUTHOR]

The objective of this study is to discover how to maximise the use of health research data without breaching the patient’s confidentiality. To achieve this, we have examined available options identified in enhancing information security of health research data. It has been noted that there are gaps in information security for health research data. Our novel approach, searching through encrypted data is discussed in this paper. [ABSTRACT FROM AUTHOR]

The increasing reliance on electronic health record (EHR) in areas such as medical research should be addressed by using ample safeguards for patient privacy. These records often tend to be big data, and given that a significant portion is stored as free (unstructured) text, we decided to examine relevant work on automated free text de-identification with recurrent neural network (RNN) and conditional random field (CRF) approaches. Both methods involve machine learning and are widely used for the removal of protected health information (PHI) from free text. The outcome of our survey work produced several informative findings. Firstly, RNN models, particularly long short-term memory (LSTM) algorithms, generally outperformed CRF models and also other systems, namely rule-based algorithms. Secondly, hybrid or ensemble systems containing joint LSTM-CRF models showed no advantage over individual LSTM and CRF models. Thirdly, overfitting may be an issue when customized de-identification datasets are used during model training. Finally, statistical validation of performance scores and diversity during experimentation were largely ignored. In our comprehensive survey, we also identify major research gaps that should be considered for future work. [ABSTRACT FROM AUTHOR]