XDMD

These images are segmentation examples in the field of neuroradiology, the field of origin of XDMD. They show the segmentation of blood vessels in the brain in a CT scan of a stroke patient with a ventricular shunt (Scientific Reports). Based on this result, smaller vessels can be more easily inspected for the presence of occlusions, detailed measurements of the vessels in 3D can be carried out, or the blood flow can be simulated.

More neuro segmentation examples of the cranial cavity, the hemispheres, the cerebral vasculature, the white and gray matter, and the cerebrospinal fluid, all from different patients. The vessel segmentation has been overlaid with a color map based on the flow arrival times to enhance the visualization of potential flow abnormalities (American Journal of Neuroradiology). The white and gray matter is an example of soft tissue segmentation in CT which normally is the domain of MR imaging. The method was the world’s first (Scientific Reports).

An example of abdominal fat segmentation in CT, with the top-left image showing the delineation of the abdominal region, and the remaining images showing subcutaneous fat in brown and visceral fat in green (MICCAI). These fat types play different roles in the pathophysiology of various diseases.

No, AI does not solve everything. Except, perhaps, until AI can discover new physics, a sign of truly understanding our world and universe. We are also impressed and excited by AI developments. The first time interacting with a large language model (LLM), such as ChatGPT, can feel like magic. LLMs belong to the class of generative AI, which include models capable of generating text, images, audio, and video. The potential of generative AI in medicine is large, but eloquence and realism can be misleading. First, it is not always obvious when results are wrong. Ensuring correctness is a real hard problem (and what is correct?) making patient safety a concern. Second, only big tech companies have the resources to build these models, raising patient privacy concerns. Finally, it may obscure the fact that under the hood quality data and manual annotating (now called ‘reinforcement learning with human feedback’) are still crucial for training.

Data is crucial. A large language model trained on blog posts about alternative medicine will generate different answers than one trained on medical scientific literature. But almost certainly they have been trained on both, so how does that work? It is the sheer scale of things. Training of any serious generative model requires massive compute and massive data. At some point the model switches from memorization to generalization, a phenomenon called grokking. That is when the magic happens. The model, also called a foundation model, can then be fine-tuned for a specific application.

Foundation models play an important role in medical imaging. Examples of foundation models are TotalSegmentator and SAM. They work because images have more commonalities than not. For example, scanners operate on the same physical principles regardless of manufacturer, anatomy is similar regardless of age or race, and segmentation tasks use similar image gradients regardless of organ. Earlier models did not exhibit grokking, later models do and enable zero-shot learning, which allows segmenting unseen images without additional training. Foundation models are excellent starting points for segmentation, but require fine-tuning for your application since your patient data is unique. Do you trust SAM trained on images scraped from the internet to segment images of your patients? Probably not. Also with foundation models, there is no free lunch.

To go from here to a commercially available product requires regulatory approval, clinical evidence, monitoring, evaluation, and many other steps, which is beyond the scope of XDMD. If you are interested in software as a medical device then we highly recommend Hardian Health and Romion Health for their advice and insights.