Search

Chest X-ray report generation aims to generate expert-level radiology diagnostic reports that can be used in clinics on given chest X-ray images. Automatic chest X-ray report generation can assist doctors in writing radiology reports, which can not only improve the diagnostic efficiency of radiologists and meet the needs of a large number of patients but also remind some inexperienced radiologists of misdiagnoses or missed diagnoses. As a visual-language task, chest X-ray report generation can be achieved by existing image captioning methods. However, chest X-ray imagery exhibits a high degree of similarity, wherein diagnostically crucial abnormalities often manifest in minute regions of the images, posing a formidable challenge in discerning subtle visual differences and distinctive features. Further, the medical findings have been widely applied as labels to guide the image encoding module to focus on the salient regions in the image using the attention mechanism. However, existing methods neglect the relations between class features corresponding to different medical findings and the relationship between such local features and global features of the image. To tackle these challenges, we propose dual graph convolution networks (DGCN) that novelly incorporate the grid feature graph and class feature graph into the report generation process. In detail, the grid graph is used to explore the relationship between the grid regions on the whole image, and the class feature graph is used to mine the relationship between different medical findings (Dual GCN). Moreover, to explore the relationship between global features, such as grid features, and local features, such as class features, we introduce a multi-head attention module to help fuse different types of features and mine more useful relationship information (FFARM). Extensive experiments on the IU X-ray dataset underscore the substantial superiority of our model over the state-of-the-art chest X-ray report generation methodologies. Further ablation analyses reveal that both the class feature graph and the grid feature graph contribute significantly to enhancing the efficacy of chest X-ray report generation. [ABSTRACT FROM AUTHOR]