Combining visual and textual informations for enhancing image retrieval systems in radiological practices

Project Summary

The field of diagnostic imaging in Radiology has experienced tremendous growth both in terms of technological development (with new modalities such as MRI, PET-CT, etc.) and market expansion. This leads to an exponential increase in the production of imaging data, moving the diagnostic imaging task in a big data challenge. However, the production of a large amount of data does not automatically allow the real exploitation of its intrinsic value for healthcare. In modern hospitals, all imaging data acquired during clinical routines are stored in a picture archiving and communication system (PACS). A PACS is a medical imaging technology providing economical storage and convenient access to images from multiple modalities. Digital images linked to patient examinations are often accompanied by a medical report in text format, summarizing the radiologist’s report and the clinical data associated with the patient (age, sex, medical history, report of previous examinations, etc.). The problem with PACS systems is that they were primarily designed for archival purposes and not for image retrieval exploitation. Therefore they only allow a search by keywords (name of the patient, date of the examination, type of examination, etc.) and not by pathologies or by content of the image, and they cannot fulfill the function of diagnostic aid when the doctor is confronted with an image of difficult interpretation or of rare pathology. The objective of this project is to combine current research in computer vision and AI to implement a method making it possible to query PACS through example images in order to search for images containing similar pathological cases and to benefit radiologists as a potential decision-making aid during hospital routines.

Florence Cloppet