Back to Search Start Over

Sequential Brain CT Image Captioning Based on the Pre-Trained Classifiers and a Language Model

Authors :
Jin-Woo Kong
Byoung-Doo Oh
Chulho Kim
Yu-Seop Kim
Source :
Applied Sciences, Vol 14, Iss 3, p 1193 (2024)
Publication Year :
2024
Publisher :
MDPI AG, 2024.

Abstract

Intracerebral hemorrhage (ICH) is a severe cerebrovascular disorder that poses a life-threatening risk, necessitating swift diagnosis and treatment. While CT scans are the most effective diagnostic tool for detecting cerebral hemorrhage, their interpretation typically requires the expertise of skilled professionals. However, in regions with a shortage of such experts or situations with time constraints, delays in diagnosis may occur. In this paper, we propose a method that combines a pre-trained CNN classifier and GPT-2 to generate text for sequentially acquired ICH CT images. Initially, CNN undergoes fine-tuning by learning the presence of ICH in publicly available single CT images, and subsequently, it extracts feature vectors (i.e., matrix) from 3D ICH CT images. These vectors are input along with text into GPT-2, which is trained to generate text for consecutive CT images. In experiments, we evaluated the performance of four models to determine the most suitable image captioning model: (1) In the N-gram-based method, ReseNet50V2 and DenseNet121 showed relatively high scores. (2) In the embedding-based method, DenseNet121 exhibited the best performance. (3) Overall, the models showed good performance in BERT score. Our proposed method presents an automatic and valuable approach for analyzing 3D ICH CT images, contributing to the efficiency of ICH diagnosis and treatment.

Details

Language :
English
ISSN :
20763417
Volume :
14
Issue :
3
Database :
Directory of Open Access Journals
Journal :
Applied Sciences
Publication Type :
Academic Journal
Accession number :
edsdoj.9b3237700e6a44d3a8a333f6ca097545
Document Type :
article
Full Text :
https://doi.org/10.3390/app14031193