# Supervisely Serving Apps ## Overview This section covers the deployment and inference of models using **Supervisely Serving Apps**. This is a simple way to deploy a model on the Supervisely Platform with a convenient web UI. It allows you to deploy both the pretrained models, such as YOLOv8, and your own models trained in Supervisely. In this guide, you'll learn how to run a Serving App, deploy models and make predictions. ![Supervisely Serving Apps](https://1080806899-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M4BHwRbuyIoH-xoF3Gv%2Fuploads%2Fgit-blob-af58fc959d48d5a39560948638a8a1b2a053e3f8%2Fserve-app-list.jpg?alt=media) You can interact with the deployed model in different ways: 1. **Apply model in platform:** You can predict easily with NN Applying apps. They allows you to predict the entire projects or datasets: [Apply NN to Images](https://ecosystem.supervisely.com/apps/nn-image-labeling/project-dataset), [Apply NN to Videos](https://ecosystem.supervisely.com/apps/apply-nn-to-videos-project). 2. **NN Image Labeling:** You can run the [NN Image Labeling](https://ecosystem.supervisely.com/apps/nn-image-labeling/annotation-tool) app right in the labeling interface and annotate images with the help of a deployed model. 3. **Inference via API:** Each model deployed in the Supervisely Platform acts like a server and ready to process inference requests via API. You can communicate with the model using Supervisely SDK (see the [Inference API Tutorial](https://developer.supervisely.com/app-development/neural-network-integration/inference-api-tutorial)), which has convenient methods for efficient model inference on images, projects, and videos. We also support Tracking-by-detection algorithms, such as BoT-Sort. ## How to Deploy a Model ### 1. Run Serving App To deploy a model, you need to run a Serving App. You can **search** an app on the [Ecosystem](https://ecosystem.supervisely.com), or find available serving apps in the [**Neural Networks**](https://app.supervisely.com/nn/apps) section of the platform. **Find your app** ![Neural Networks Section](https://1080806899-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M4BHwRbuyIoH-xoF3Gv%2Fuploads%2Fgit-blob-6603dd52e1fa27dee8370eee037f0140c336597a%2Fnn-apps.jpg?alt=media) **Run app** Run an app by clicking the **"Run App"** button, and the app interface will be opened in a new tab. In the first time, the app will download a docker image, it may take some time. After the app is started, you'll see the Serving GUI. **Serving App Interface:** ![Serving App Interface](https://1080806899-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M4BHwRbuyIoH-xoF3Gv%2Fuploads%2Fgit-blob-3ff71bfd66b329cc5bfffb6b7178f578e3482a20%2Fserve-rtdetr-gui.jpg?alt=media) If the app did not open automatically, you can find it in the **"Apps"** section of the platform. **Open the app from the Apps section** ![Apps Section](https://1080806899-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M4BHwRbuyIoH-xoF3Gv%2Fuploads%2Fgit-blob-f2f3cffead58d21cd43ea6241916ba78353c1d42%2Fapps-section.jpg?alt=media) ### 2. Deploy Model After you've opened the app, you'll see the app's interface with the following sections: 1. **"Pretrained" and "Custom" Tabs:** In the pretrained tab, you can find the list of available pretrained checkpoints to select one. In the custom tab, you can select your own model trained in Supervisely. 2. **Device Selector:** Choose the device to load the model on. 3. **Serve Button:** Click the "Serve" button to start the deployment process. The app will download model weights, load it on device, and start the FastAPI server for API requests. After the model is deployed, you'll see a green checkmark with text *"Model has been successfully loaded on device"*, and the **"Full model info"** section will be filled with the model's information. ![Model Deployed](https://1080806899-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M4BHwRbuyIoH-xoF3Gv%2Fuploads%2Fgit-blob-581750985a677c4a9df07fe5ef8c212276eaafaf%2Fmodel-deployed.jpg?alt=media) ## How to Predict ### Apply Model in Platform After the model is deployed, you can apply it using the **NN Applying apps**. They allow you to predict the entire projects or datasets: * [Apply NN to Images](https://ecosystem.supervisely.com/apps/nn-image-labeling/project-dataset) * [Apply NN to Videos](https://ecosystem.supervisely.com/apps/apply-nn-to-videos-project) * [Apply Classifier to Images](https://ecosystem.supervisely.com/apps/apply-classification-model-to-project) * [Apply Florence-2 + SAM 2 to Images](https://ecosystem.supervisely.com/apps/apply-florence-2-to-images-project) **Search "apply" to find available apps:** ![Apply Apps](https://1080806899-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M4BHwRbuyIoH-xoF3Gv%2Fuploads%2Fgit-blob-785b728ad5cfcbd9bdb5a20680c4aa94f013d8a8%2Fsearch-apply.jpg?alt=media) ### NN Image Labeling **Run NN Image Labeling** Open Labeling Tool and run the [NN Image Labeling](https://ecosystem.supervisely.com/apps/nn-image-labeling/annotation-tool) app from the **"Apps" panel**. The app will be opened in the labeling interface. ![NN Image Labeling](https://1080806899-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M4BHwRbuyIoH-xoF3Gv%2Fuploads%2Fgit-blob-d1ed0b1afaf50064f4c0bc110e4f2c1ff88c48c8%2Fnn-image-labeling.jpg?alt=media) **Connect to the deployed model** In the app interface, select the deployed model from the list. Click the **"Connect"** button. After that, you can start annotating images with the help of the model. ![Apply Model in Labeling](https://1080806899-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M4BHwRbuyIoH-xoF3Gv%2Fuploads%2Fgit-blob-a8a9567166be32f47016fdac1eb7522b506a4b6b%2Fnn-image-labeling-2.jpg?alt=media) ### Inference via API You can automate the inference process by sending requests to the deployed model via API. A running Serving App acts like a server and can process inference requests. See the [Inference API Tutorial](https://developer.supervisely.com/app-development/neural-network-integration/inference-api-tutorial) for more details. ```python import os from dotenv import load_dotenv import supervisely as sly # Get your Serving App's task_id from the Supervisely platform task_id = 27209 # init sly.Api load_dotenv(os.path.expanduser("~/supervisely.env")) api = sly.Api() # Create Inference Session session = sly.nn.inference.Session(api, task_id=task_id) prediction = session.inference_image_id(image_id=123) predictions = session.inference_project_id(project_id=123) ``` --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.supervisely.com/neural-networks/inference-and-deployment/supervisely-serving-apps.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.