Language Model that can call APIs


Hello Reader,

Welcome to another edition of PYCAD newsletter where we cover interesting topics in Machine Learning and Computer Vision applied to Medical Imaging. The goal of this newsletter is to help you stay up-to-date and learn important concepts in this amazing field! I've got some cool insights for you below ↓

Gorilla: A Large Language Model that can call APIs

A new open source large language model can superbly find the right API call starting from a general prompt. It's called Gorilla!
​
Is this confusing? Let me explain.
​
Suppose I would like to detect cats on an image. Meaning, I would like to actually get the bounding boxes coordinates surrounding the cats on the image.
​
ChatGPT cannot do that directly. Maybe through some plugin it can do that.
​
But there are a variety of machine learning models out there that can do this.
​
A lot of them can be called through some API or library.
​
Gorilla allows you to get the necessary libraries and it gives you the necessary steps to call methods or classes from those libraries.
​
Here's an example to illustrate this.
​
You can give Gorilla the following prompt: "I want to build a robot that can detect objects in an image.".
​
Gorilla will understand that you would like to perform object detection on an image.
​
Then it will find the right model from the HuggingFace library that can achieve this. In this case it's a YOLO model that can be accessed through the class "YolosForObjectDetection".
​
It then tells you the necessary steps to load your image or camera feed to the model and make predictions on it.
​
Awesome isn't it!?
​
For more information about this model, you can read the paper, see the code on github or test the code directly on google colab.
​

Why the ML Community is Awesome!

A really insightful exchange that I lately saw on Twitter between 3 ML researchers.
​
​Researcher 1: "New paper!! We show that pre-training language-image models *solely* on synthetic images from Stable Diffusion can outperform training on real images!!".
​
​Researcher 2: "Nice work. But stable diffusion already uses a pretrained clip, wonder does that render your exploration as some kind of distillation from pretrained models?"
​
​Researcher 3: "I think that's a nice way to look at it. SD is distillation of knowledge from CLIP and LAION into generative model.
Kind of like dataset distillation but optimized with a generative objective rather than a task objective."
​
It's exchanges like these that make ML community awesome!
​
Here's the paper that they were talking about.
​


Cool AI Tools (Affiliates)

​Chatbase: An AI chatbot builder that lets you build, train, and embed smart chatbots powered by ChatGPT right on your website.

​EzyCourse​: Create and sell your courses, services, products, or online communities from one platform.

​Katteb: The first fact-checked, real-time, and localized AI writer.


​

What'd you think of today's edition?

​

Machine Learning for Medical Imaging

👉 Learn how to build AI systems for medical imaging domain by leveraging tools and techniques that I share with you! | 💡 The newsletter is read by people from: Nvidia, Baker Hughes, Harvard, NYU, Columbia University, University of Toronto and more!

Read more from Machine Learning for Medical Imaging

Hello Reader, Welcome to another edition of PYCAD newsletter where we cover interesting topics in Machine Learning and Computer Vision applied to Medical Imaging. The goal of this newsletter is to help you stay up-to-date and learn important concepts in this amazing field! I've got some cool insights for you below ↓ AI Scribes: Transforming Medical Documentation Web Application for Medical Note Generation AI-powered medical scribes are revolutionizing clinical workflows by automating...

Hello Reader, Welcome to another edition of PYCAD newsletter where we cover interesting topics in Machine Learning and Computer Vision applied to Medical Imaging. The goal of this newsletter is to help you stay up-to-date and learn important concepts in this amazing field! I've got some cool insights for you below ↓ DeepSeek: A New Player in AI for Healthcare The new open-source LLM, DeepSeek, is creating buzz for its potential to transform AI in medicine and healthcare. Designed for...

Hello Reader, Welcome to another edition of PYCAD newsletter where we cover interesting topics in Machine Learning and Computer Vision applied to Medical Imaging. The goal of this newsletter is to help you stay up-to-date and learn important concepts in this amazing field! I've got some cool insights for you below ↓ Now You Can Use Large Language Models that are HIPAA Compliant People are finding ways to use large language models in all fields. MedTech is no exception. The amount of work...