SAM but for Medical Imaging

In the realm of medical imaging, MONAI has released the equivalent of SAM but in medical imaging!
​
The new model is called VISTA - Versatile Imaging Segmentation and Annotation.
​
VISTA is a robust model, akin to SAM (Segment Anything Model), but specifically designed for medical imaging. The model is based on SAM and has been finetuned on medical data.
​
Specifically, the image encoder, prompt encoder, and mask decoder have been finetuned on medical data.
​
A notable feature of VISTA is its ability to integrate with MONAI Label. This integration facilitates data annotators in swiftly labeling their data by specifying the desired classes and organs.
​
Impressively, VISTA can recognize 104 anatomical structures and has the capacity to generalize to unknown classes.
​
The model performs particularly well on cardiovascular, organs, gastrointestinal, and muscles structures.
​
However, it's worth noting that its performance on skeleton structures is an area for improvement.​
​
Just like SAM, VISTA can be a great addition to data labeling tasks. But I don't think it will be useful for production grade automatic image segmentation.

Why do you need to know about this?

Because this model has the potential to be a game changer for annotation tasks in medical imaging. And just like we have FastSAM as a leaner and faster model which is based on SAM, we might soon have FastVISTA! Then this model can be very suitable for production in automatic 3D segmentation!

​

Federated Learning: A technique that needs more light shed on it

Let’s say you build an AI solution for hospital A.

Then you build a similar solution for hospitals B and C.

Is there a way where your AI models from the 3 hospitals can collaborate?

And can they do that without sharing any of the data on which the AI solutions in each hospital were trained on?

The answer is YES to both questions.

How?

Through federated learning or FL for short.

To me, this is one of the most exciting fields in machine learning that almost nobody talks about.

In many industries, sharing data is a hard NO. Mainly due to privacy reasons.

With federated learning you can build custom ML models for several clients and then you can use FL algorithms such as FedAvg, FedProx and FedOpt to build a new model that makes use of the models that are already built.

This is the theory, but how do you implement this in practice?

Well, you can use Nvidia FLARE (Federated Learning Application Runtime Environment).

It’s an open source framework that allows you to implement FL with ease.

Here are few things that Nvidia FLARE supports:

• Support both deep learning and traditional machine algorithms
• Support horizontal and vertical federated learning
• Built-in FL algorithms (e.g., FedAvg, FedProx, FedOpt, Scaffold, Ditto )
• Support multiple training workflows (e.g., scatter & gather, cyclic) and validation workflows (global model evaluation, cross-site validation)

Another cool thing about this framework is that it has an FL simulator!

The Simulator allows you to start a FLARE server and any number of connected clients on your local workstation or laptop, and to quickly deploy an application for testing and debugging.

Here's an example on how to use Nvidia FLARE with MONAI for 3D spleen CT segmentation.

Why do you need to know about this?

Federate learning is one of those technologies that didn't have enough spotlight in the machine learning world. But it is still one of the most interesting techniques in ML in my opinion because of the huge potential it has when it comes to applying ML in industries that have very low tolerance to sharing data, such as healthcare, banking, ...etc. If you know about it and if you spend more time digging into it with the tools mentioned above then this could be an opportunity that you can capitz

Do you like the new section called "Why do you need to know this"?

• ​Yes​
• ​No​

​