Segmentation

XDMD builds solutions for (almost) all and the most difficult segmentation problems and considers all anatomies, modalities, and dimensionalities. We do that by developing machine learning algorithms trained on the data of your patients scanned on your scanner and supervised by your expertise, at the quality level required for your application, whether this is a proof-of-concept, a prototype to address a research question, or first-version product that should work in clinical practice.

Go directly to our vision or approach

Details

These images are segmentation examples in the field of neuroradiology, the field of origin of XDMD. They show the segmentation of blood vessels in the brain in a CT scan of a stroke patient with a ventricular shunt (Scientific Reports). Based on this result, smaller vessels can be more easily inspected for the presence of occlusions, detailed measurements of the vessels in 3D can be carried out, or the blood flow can be simulated.

Details

More neuro segmentation examples of the cranial cavity, the hemispheres, the cerebral vasculature, the white and gray matter, and the cerebrospinal fluid, all from different patients. The vessel segmentation has been overlaid with a color map based on the flow arrival times to enhance the visualization of potential flow abnormalities (American Journal of Neuroradiology). The white and gray matter is an example of soft tissue segmentation in CT which normally is the domain of MR imaging. The method was the world’s first (Scientific Reports).

Details

An example of abdominal fat segmentation in CT, with the top-left image showing the delineation of the abdominal region, and the remaining images showing subcutaneous fat in brown and visceral fat in green (MICCAI). These fat types play different roles in the pathophysiology of various diseases.

Why segmentation?

Segmentation enables: visualization, quantification, characterization of anatomy, detection of pathology, characterization of pathology, surgical planning, surgical navigation, surgical scene understanding, treatment response monitoring, hemodynamic simulation, morpho-dynamic simulation, etcetera.

Our vision

Segmentation is essentially about image understanding. Therefore, human experts are always needed for interpretation, usually in the form of manual annotation. A machine learning algorithm must first see examples of annotated images before it can automatically segment unseen images. This process is called supervised learning. Segmentation requires:

Damned finest coffee

Data

Reference standard

Image processing

Neural networks

These are the basic building blocks. Despite all the amazing developments in AI, the brilliant new architectures, the massive computer power, we still need quality and representative data, and a clear understanding of what we are looking at, before AI potentially can take over the task. There is no shortcut. Also, behind every block there is a complete world of scientific progress and technical innovation. The field of medical imaging is constantly changing making an image (and its interpretation!) essentially a snapshot in space and time.

Snapshot in spacetime

A few simple examples illustrate its importance. Let’s say a new segmentation method (a new AI model) has been developed on CT images with a certain resolution. If technological advancements enable scanning at much higher resolution, for example with photon-counting CT, then this model cannot be used as is on the new imaging data. Or if an AI model has been trained on an adult population, then it cannot be used as is on a pediatric population. All factors contributing to image creation, as well as who is looking, influence the AI model. Therefore not only the image is a snapshot in spacetime, the AI model itself is as well: both are static. As local data and environments change, AI model performance declines over time, requiring monitoring when deployed in clinical practice. Segmentation is thus an engineering problem and cannot be solved like solving a mathematical problem.

But AI solves everything

No, AI does not solve everything. Except, perhaps, until AI can discover new physics, a sign of truly understanding our world and universe. We are also impressed and excited by AI developments. The first time interacting with a large language model (LLM), such as ChatGPT, can feel like magic. LLMs belong to the class of generative AI, which include models capable of generating text, images, audio, and video. The potential of generative AI in medicine is large, but eloquence and realism can be misleading. First, it is not always obvious when results are wrong. Ensuring correctness is a real hard problem (and what is correct?) making patient safety a concern. Second, only big tech companies have the resources to build these models, raising patient privacy concerns. Finally, it may obscure the fact that under the hood quality data and manual annotating (now called ‘reinforcement learning with human feedback’) are still crucial for training.

Data is crucial. A large language model trained on blog posts about alternative medicine will generate different answers than one trained on medical scientific literature. But almost certainly they have been trained on both, so how does that work? It is the sheer scale of things. Training of any serious generative model requires massive compute and massive data. At some point the model switches from memorization to generalization, a phenomenon called grokking. That is when the magic happens. The model, also called a foundation model, can then be fine-tuned for a specific application.

Foundation models

Foundation models play an important role in medical imaging. Examples of foundation models are TotalSegmentator and SAM. They work because images have more commonalities than not. For example, scanners operate on the same physical principles regardless of manufacturer, anatomy is similar regardless of age or race, and segmentation tasks use similar image gradients regardless of organ. Earlier models did not exhibit grokking, later models do and enable zero-shot learning, which allows segmenting unseen images without additional training. Foundation models are excellent starting points for segmentation, but require fine-tuning for your application since your patient data is unique. Do you trust SAM trained on images scraped from the internet to segment images of your patients? Probably not. Also with foundation models, there is no free lunch.

Our approach

First and foremost we examine your images. What do you see? What do you hope to see? What is the clinical context? Which segmentation accuracy is required for your workflow?

Blueprint

XDMD adopts a systematic approach. We have devised a comprehensive questionnaire to assess your segmentation problem in full depth. We consider each building block of the pipeline (data, reference standard, image processing, neural network) from all possible perspectives. The questionnaire is regularly updated to align with the current developments in medical imaging. Your answers will indicate, for example, how to address annotation in order to achieve the required accuracy using foundation models. Or how to achieve real-time processing given the available computer resources. We solve your segmentation problem under all imposed constraints, resulting in the optimal solution. And we always strive for simplicity and elegance.

Software products

We offer the following software products:

  • dedicated tools for annotation
  • secured database infrastructure
  • image processing training algorithms
  • image processing inference algorithms
  • neural network training algorithms
  • neural network inference algorithms

The training algorithms enable continued development on-site. All products have test routines, are documented, and have direct support. Alternatively, we offer a secured cloud-based solution if that better suits your situation. Please contact us to learn more.

Medical device

To go from here to a commercially available product requires regulatory approval, clinical evidence, monitoring, evaluation, and many other steps, which is beyond the scope of XDMD. If you are interested in software as a medical device then we highly recommend Hardian Health and Romion Health for their advice and insights.