Indexing Metadata

1 Title of the Article Fine-grained Abnormality Detection and Natural Language Description of Medical CT Images Using Large Language Models
2 Author's name Zhongwen Zhou: Computer Science, University of California, Berkeley, CA, USA
3 Author's name Siwei Xia, Mengying Shu, Hong Zhou
4 Subject Computer Science
5 Keyword(s) Medical Image Analysis, Large Language Models, Fine-Grained Abnormality Detection, Natural Language Description Generation
6 Abstract

Medical report generation demands accurate abnormality detection and precise description generation from CT images. While large language models have shown promising results in natural language processing tasks, their application in medical imaging analysis faces challenges due to the complexity of fine-grained feature detection and the requirement for domain-specific knowledge. This paper presents a novel framework integrating large language models with specialized medical image processing techniques for fine-grained abnormality detection and natural language description generation. Our approach incorporates a multi-modal knowledge enhancement module and a hierarchical attention mechanism to bridge the gap between visual understanding and textual description. The framework employs an adapter-based architecture for efficient domain adaptation and introduces a medical knowledge-enhanced loss function to improve description accuracy. Experimental results on three public datasets demonstrate the effectiveness of our approach, achieving 94.6% detection accuracy and a BLEU-4 score of 0.421 for description generation, surpassing current state-of-the-art methods. The system shows particular strength in handling subtle abnormalities, with a 91.2% average precision in fine-grained detection tasks. Comprehensive ablation studies validate the contribution of each component, while qualitative analysis demonstrates the clinical relevance of generated descriptions. The proposed framework represents a significant advancement in automated medical image analysis, offering potential benefits for clinical workflow optimization and diagnostic support.

 

7 Publisher Innovative Research Publication
8 Journal Name; vol., no. International Journal of Innovative Research in Computer Science & Technology (IJIRCST); Volume-12 Issue-6
9 Publication Date November 2024
10 Type Peer-reviewed Article
11 Format PDF
12 Uniform Resource Identifier https://ijircst.org/view_abstract.php?title=Fine-grained-Abnormality-Detection-and-Natural-Language-Description-of-Medical-CT-Images-Using-Large-Language-Models&year=2024&vol=12&primary=QVJULTEzMjc=
13 Digital Object Identifier(DOI) 10.55524/ijircst.2024.12.6.8   https://doi.org/10.55524/ijircst.2024.12.6.8
14 Language English
15 Page No 52-62

Indexed by

Crossref logo