# What's Supervisely **Supervisely** is computer vision platform for researchers and companies to annotate and manage datasets, train neural networks and much more. Unlike other platforms, Supervisely is [built like OS](/ecosystem): instead of having a huge monolith, Supervisely creates a foundation for developing and running applications called Supervisely Apps. Supervisely is available [online](https://app.supervisely.com/signup) for free, as well as an on-premise edition for enterprises. ## With Supervisely you can * [Label](/labeling/labeling-toolbox) **images**, **videos**, **3D point clouds**, **volumetric slices** and other data in the best labeling toolboxes.

Image labeling tool

* **Manage** and **track** annotation workflow at scale with [teams](/collaboration/teams), workspaces, roles and [labeling jobs](/labeling/jobs).
* **Train** [neural networks](/neural-networks/overview) on your custom datasets or use pre-trained models to speed up manual labeling. ![](/files/DtTr4Xd6k9v1FtCzfrjm) * Use the best machine learning tools, visualize and improve your data with hundreds of applications from [Ecosystem](https://ecosystem.supervisely.com/)
### What's next? The best way to explore Supervisely is to try it out - so don't wait and [create an account](https://app.supervisely.com/signup) (it's completely free!). Here are some things to start with: {% content-ref url="/pages/5xVOSLson5vl2rFLKxLT" %} [How to import](/getting-started/how-to-import) {% endcontent-ref %} ### Beyond the documentation If you are interested in learning more about Supervisely, you may find those resources interesting:
Supervisely Blog 📚Where we share tutorials and guides on the hottest topics in computer vision.https://supervisely.com/blog/
Video Course 📽️Prefer video? Watch full video course on what is Sueprvisely in 50 chapters.https://supervisely.com/what-is-supervisely/
GitHub Page 🐙Want to contribute to Supervisely? Start with our GitHub page here.https://github.com/supervisely/supervisely
# Ecosystem of Supervisely Apps How Supervisely is different from others and what are the Ecosystem and Supervisely Apps? ![](/files/SH2qs58SHzOlfaBsbDM3) Before we dive into the world of annotation, dataset management and neural networks, it is important to understand how Supervisely is different from other similar solutions. The main issue with most solutions on the market is that they build as products. It's a black box developed by some company you don't really have an impact on. As soon as your requirements go beyond basic features offered and you want to customize your experience, add something that is not in line with the software owner development plans or won't benefit other customers, you're out of luck. That is why Supervisely is building a platform instead of a product. You can think of Supervisely as an Operating System available via your web browser to help you solve computer vision tasks. The idea is to unify all the relevant tools within a single Ecosystem of Supervisely Apps: machine learning tools, UI widgets and services that may be needed to make the AI development process as smooth and fast as possible. ## Ecosystem
{% embed url="" %} The simplicity of creating Supervisely Apps has already led to the development of [hundreds of applications](https://ecosystem.supervisely.com/), ready to be run within a single click in a web browser and get the job done. Label your data, perform quality assurance, inspect every aspect of your data, collaborate easily, train and apply state-of-the-art neural networks, integrate custom models, automate routine tasks and more - like in a real AppStore, there should be an app for everything. ## Customization ![](/files/7XpZugO2aVkYzATMesOv) {% embed url="" %} Feel free to join the development of Supervisely and start building your own Supervisely Apps. You can integrate your favorite GitHub repository, machine learning model, create an import or export of a custom data format or anything in between! We welcome you at our [Developer Portal](https://developer.supervisely.com/) where you can find all the necessary information and examples on how to develop and integrate your own Supervisely Apps. Got any questions or suggestions? Feel free to chat with us in our [Supervisely Slack.](https://supervisely.com/slack) # FAQ Explore the topics to get detailed information and solutions for common queries. If you have any other questions, please don't be afraid to contact our support team.
Is Supervisely free? Supervisely offers a free Community plan with certain limitations, which is ideal for open-source projects, individuals, ML researchers and small teams. For more or full access to advanced features or to work on a commercial project, you can upgrade to a paid edition.\ Detailed information on the differences between plans, pricing, limitations and features can be found on our [Pricing page](https://supervisely.com/pricing/). If you are interested in a self-hosted or cloud solution for company with custom requirements of any size and with no limitations, please [contact us](https://supervisely.com/contact-us/).
Who has rights to the data? Your data is yours. We respect your privacy and when you create an account you don't grant us any rights to your data, except for the ones that needs for the application functioning. We don't use your data for any commercial or non-commercial purposes and share it with nobody. Supervisely uses industry-standard encryption to protect data at rest. You can learn more in the [terms of service](https://supervisely.com/terms-of-service).
Do you have an on-premises version? We do! Drop us an email at or fill the form [here](https://supervisely.com/enterprise).
What does “Supervisely” mean? The name Supervisely comes from machine learning term supervised learning - when we use a known dataset (called the training dataset) to make predictions. And, well, Supervisely is all about datasets and using them to build models.
Origins of Supervisely It's not unusual when internal tools become public projects. Supervisely was first developed as a solution to deal with everyday task of large datasets annotation. We hope that you will like Supervisely as much as we do and it'll become your favourite tool too 🎉.
How can I get started with Supervisely? Getting started with Supervisely is easy! You can sign up for a free account on our website and check out our comprehensive video tutorials, blog post guides, and documentation to get you started quickly and easily annotating your data.
What types of data can be annotated using Supervisely? Supervisely supports a variety of data types including images, videos, point clouds, point cloud episodes and DICOM. For more detailed information on supported data formats, please refer to our [How to import](/getting-started/how-to-import#supported-formats-and-modalities) section.
What are the system requirements for using Supervisely? Supervisely can be accessed via any modern web browser. However, for optimal performance, we recommend using Google Chrome. For those using the on-premises version, specific hardware and software requirements can be found in the [Installation](/agents/connect-your-computer) guide.
Can Supervisely integrate with other tools and platforms? Yes, Supervisely offers integrations with various tools and platforms, including cloud storage services like AWS and Google Cloud, as well as popular machine learning frameworks like TensorFlow and PyTorch. You can also use our API to integrate Supervisely with your own systems.
How can I import/export data in Supervisely? Supervisely allows you to easily import data in various formats such as COCO, Pascal VOC, Cityscapes and more. Exporting your annotated data is also easy and supports formats compatible with many machine learning frameworks. You can also customize the export settings to include specific metadata or annotation details as needed. Detailed guides for importing and exporting data are available in our documentation.
Can I automate annotations in Supervisely? Yes, Supervisely supports several automation tools including AI-assisted labeling and model training directly in the platform. You can use pre-trained models or upload your own to automate the annotation process.
How often is Supervisely updated with new features? We are always working to improve Supervisely and regularly release updates with new features, applications, improvements and bug fixes. You can keep up with the latest updates by reading our blog or subscribing to our newsletter.
What types of annotation tools does Supervisely offer? Supervisely provides a variety of annotation tools tailored for different data types, including bounding boxes, polygons, points, lines, masks and keypoints. Additionally, there are specialized tools for video frame annotation, 3D point cloud labeling and more.
How does Supervisely help me collaborate? Supervisely provides powerful collaboration features, including teams, shared workspaces, project sharing, role-based access control, and tools for quality assurance and data statistics. Team members can create labeling jobs, leave issues, make and review annotations, and track annotation updates in real time. The platform also supports versioning in the Pro and Enterprise editions, so you can track the history of annotations made to your data. For more information, see our [Collaboration](/collaboration/overview) section.
Can I train machine learning models directly in Supervisely? Supervisely provides built-in machine learning tools that allow you to train models directly on the platform. You can use our predefined models or bring in your own to fine-tune on your annotated datasets. In addition, the platform supports evaluation metrics and visualization tools to help you assess model performance.
Is there a community or support forum for Supervisely users? Yes, we have an active community [Slack chat](https://supervisely.slack.com/join/shared_invite/enQtNzUwMDYwNTMzODI1LWJlNTFiM2VkYzQ1ZDg1NmU4MWJkNzY1NDRjMDYzMWVlZDQwNzk5YzI0YTZiOWI3NDcwMjgzNDJhMDNlMzFhYzk#/shared-invite/email) where users can ask questions, share insights and get help from other users and the Supervisely team. You can join the chat through our website or directly [here](https://supervisely.slack.com/join/shared_invite/enQtNzUwMDYwNTMzODI1LWJlNTFiM2VkYzQ1ZDg1NmU4MWJkNzY1NDRjMDYzMWVlZDQwNzk5YzI0YTZiOWI3NDcwMjgzNDJhMDNlMzFhYzk#/shared-invite/email).
What is the process for reporting bugs or requesting new features in Supervisely? Users can report bugs or request new features through our support channel or our community [Slack chat](https://supervisely.slack.com/join/shared_invite/enQtNzUwMDYwNTMzODI1LWJlNTFiM2VkYzQ1ZDg1NmU4MWJkNzY1NDRjMDYzMWVlZDQwNzk5YzI0YTZiOWI3NDcwMjgzNDJhMDNlMzFhYzk#/shared-invite/email). We prioritize these reports based on their impact and relevance to our user base.
# Support Whether you're using our Community Edition or Enterprise Edition, this page is your starting point for getting help, and resolving issues quickly. If you need immediate assistance, please reach out to our support team.\ **Email:** \ **Community Edition Slack:** [Join here](https://supervisely.com/slack)\ **Enterprise Edition Slack:** for our Enterprise customers, we typically set up a private Slack channel for faster, real-time support. If you're an Enterprise user and not yet a member of your dedicated Slack channel, just let us know and we'll get you added right away. ## Community Edition or Enterprise Edition? Not sure what edition you're using? No worries - we've got you covered. * If you open Supervisely at `https://app.supervisely.com`, you're using the **Community Edition**. * If you're accessing it through any other address, you're on the **Enterprise Edition**. Enterprise users receive **high-priority support** - you can reach out to us directly, and we'll do our best to resolve your issue as soon as possible.\ If you're contacting us from a personal email address, please mention that you're an Enterprise user to help us prioritize your request. ### Cloud-hosted Enterprise Instances If your Supervisely server address ends with `enterprise.supervisely.com`, you're using an **Enterprise instance hosted in our Cloud**.\ This means the Supervisely team is actively managing the server, including updates, backups, maintenance, logs, and troubleshooting. You **do not need to collect or send a troubleshooting archive**, just contact us directly, and we'll take it from there. ## I Have an Issue If something's not working as expected in Supervisely, let's first narrow down what kind of issue you're facing: 1. You're having [trouble with an app from the Supervisely Ecosystem](#issues-with-applications) (or a custom app). 2. You see an [error message in the UI outside](#issues-with-the-platform) of the app (e.g. "Something went wrong").
Something went wrong
3. The platform isn't throwing errors, but it feels [slow or laggy](#slow-performance). 4. Things technically work, but the [behavior seems off or unexpected](#unexpected-behavior). 5. You're running into a technical issue with the [API or SDK](#python-sdk-or-api-issues). 6. You're an Enterprise technical engineer and need [help with deployment](#deployment-issues). Once you've identified the type of issue, scroll down to find the best way to get support. ### Issues with Applications If you're having trouble with an app from the Supervisely Ecosystem (or a custom app), here are some steps you can take: 1. Check the app's README section and ensure that you followed all setup instructions.
App README
2. If there are some specific error messages while running the app, please download its logs and share it with our support team. #### How to download application logs? 1. Open the `Tasks & Apps` section in the left sidebar. 2. Check if the needed session is in the `Tasks` tab.
Error Log
3. Click on the three dots icon and select the `Logs` option.
Logs Tab three dots
4. In the opened window, click the `Download Logs` button.
Download Logs
If you didn't find anything in the **Tasks tab**, then go to the **Apps tab**: 1. Open the `Apps` tab, find the needed application and click on the button under it. Then simply click on the `Open error log`.
Error Log
2. Click on the `Download Logs` button.
Download Logs
Now, when you downloaded the logs, please share them with our support team for further assistance. #### Changing the log level of application In some cases, we'll need additional logs to diagnose the issue. You can change the log level in two ways: **Option A** - Start a new app session with a different log level 1. Launch the application and click on the `Advanced settings` option. 2. Set the `Log Level` to `Debug` or `Trace` to capture more detailed logs. 3. Start the application.
Change Log Level
**Option B** - Re-run the same session with a different log level 1. Open `Tasks & Apps` and find the previous session. 2. Click `Run Again`. 3. In the relaunch dialog, open `Advanced settings` and set the `Log Level` to `Debug` or `Trace`. 4. Start the application.
Change Log Level
**What happens next** * If the app runs with saved inputs, it will automatically repeat the same scenario with the new log level. * If it does not, it will start with the same parameters as last time and be ready for work. Now, use the application to reproduce the issue. When finished, please [download the logs](#how-to-download-application-logs) and share them with our support team. ### Issues with the platform If you encounter an error in the graphical user interface while not using any specific application: * **Community Edition**: Please contact our support team with a description of the issue. * **Enterprise Edition**: Before reaching out, if you're not using [Cloud-hosted Enterprise Instance](#cloud-hosted-enterprise-instances), and the [Remote Logs](#enabling-remote-logs) feature is not enabled, please [generate a troubleshoot archive](https://docs.supervisely.com/enterprise-edition/advanced-tuning/generating_ts_archive) and share it with our support team. This helps us resolve your issue faster. #### Enabling Remote Logs When something goes wrong, the fastest way for our team to help is by seeing what the system sees.\ Remote logs allow Enterprise customers to securely send system-level logs to the Supervisely team in real time - so we can diagnose and resolve issues without needing a manual troubleshooting archive. To enable remote log forwarding, run: ```bash sudo supervisely enable-remote-logs ``` To disable it at any time, use: ```bash sudo supervisely disable-remote-logs ``` **Extended Remote Logs** For more detailed troubleshooting, you can enable extended remote logs that include more verbose log levels. To enable extended remote logs, do the following: 1. Open the Supervisely configuration file: ```bash cd $(sudo supervisely where) nano ./.env ``` 2. Find the line that starts with `SEND_LOGS_REMOTE_SERVER_MODE=`. 3. Change its value to `all`. The resulting line should look like this: ``` SEND_LOGS_REMOTE_SERVER_MODE=all ``` 4. Save the file and execute the following command to apply the changes: ```bash sudo supervisely up -d vector ``` **What's sent? Only error-level system logs.** Remote logging transmits **only error-level system logs** that are non-sensitive and necessary for technical troubleshooting, such as error traces, service failures, and critical system diagnostics. It does **not include any personal data, customer content, project files, credentials, or identifiable information**. We take customer privacy seriously. Remote logs are strictly limited to error-level entries required for technical troubleshooting and are handled securely by the Supervisely team. This ensures faster support without compromising your data integrity or confidentiality. ### Slow performance If you notice that the platform is running slowly, and you're using the Community version of Supervisely, perform a [speed test](https://docs.supervisely.com/enterprise-edition/advanced-tuning/speed_test) and contact our support team with the results. For Enterprise version of Supervisely, please do the following: 1. Open any Labeling Toolbox (such as Image Labeling Toolbox). 2. Find the Troubleshoot button on the bottom right corner and click it.
roubleshoot Button
3. Perform the Troubleshoot test, save the results (text or screenshot) and share them with our support team.
roubleshoot Button
#### Performing a speed test If you are using the Community Edition of Supervisely, you can perform a speed test to check your connection to our servers. You can use any online speed test tool, such as [Speedtest.net](https://www.speedtest.net/), and share the results with our support team.\ Note, that the speed test should be performed to specific location that matches the region of the Supervisely instance. For example, for the Community Edition, use a server located in Germany (Falkenstein). ### Unexpected behavior If you don't see any errors, but it looks like that something is not working as expected, please try the following steps: 1. Ensure that you're using latest version of Chromium-based browser (Google Chrome, Microsoft Edge, etc.). 2. Try the same in the incognito mode with all the extensions disabled. 3. Clear your browser cache and cookies. 4. Try using a different browser or device. If it did not help, check for errors in the browser's Developer Console by following these steps: #### Step 1: Open Developer Console 1. Right-click on the page and select "Inspect" (or press Ctrl+Shift+I) to open the Developer Console. #### Step 2: Check Console Tab for Errors 1. Go to the "Console" tab. 2. Look for any error messages (usually displayed in red). 3. Take a screenshot of any errors you find. #### Step 3: Check Network Tab for Failed Requests 1. Switch to the "Network" tab in the developer tools. 2. Check the "Disable cache" option (usually located at the top of the Network tab). 3. Clear the current log by clicking the clear button (🚫 icon). 4. Reload the page (press F5 or Ctrl+R). 5. Apply the "Fetch/XHR" filter to show only API requests. 6. Look for any entries marked in red or with error status codes (4xx, 5xx). 7. Click on each failed request to see details. 8. Copy the response content (in JSON format if available) and take screenshots. #### What to Share with Us When contacting our team, please include: * Screenshots from both Console and Network tabs showing any errors * JSON responses from failed network requests * A detailed description of the steps that led to the issue * Screen recording of the issue (if possible) This information will help us analyze the exact network behavior and pinpoint any failed requests. ### Python SDK or API issues If you're using our Python SDK or the API directly and encounter an issue, please follow these steps: 1. Ensure you are using the version of SDK that corresponds to your Supervisely instance. You can check the compatibility table [here](https://developer.supervisely.com/getting-started/installation#compatibility-table). 2. Check out the [SDK Reference](https://supervisely.readthedocs.io/en/latest/sdk_packages.html) to make sure you are using the correct methods and parameters. 3. If you're using the API directly check out the following documentation:\ [Community version API documentation](https://api.docs.supervisely.com/)\ For Enterprise version, the link to the documentation is: `/api-docs/` After checks, if you still encounter issues, please contact our support team with a detailed description of the problem. We also appreciate if you could provide any relevant logs, error messages or code snippets to help us assist you better. ### Deployment issues First of all, check out the Enterprise documentation: * [Installation and post-installation](https://docs.supervisely.com/enterprise-edition/get-supervisely) * [Advanced tuning](https://docs.supervisely.com/enterprise-edition/advanced-tuning) If you did not find a solution in the documentation, please contact our support team, and we will do our best to assist you. # How to import Explore various import methods on the Supervisely Platform, including importing different formats and modalities, importing from the cloud or via Ecosystem apps. {% hint style="info" %} This 5-minute tutorial is a part of introduction to Supervisely series. You can complete them one-by-one, in random order, or jump to the rest of the documentation at any moment. * How to import **(you are here)** * [How to annotate](/getting-started/how-to-annotate) * [How to invite team members](/getting-started/invite-member) * [How to connect agents](/agents/connect-your-computer) * [How to train models](/getting-started/how-to-train-models) {% endhint %} {% hint style="success" %} You can learn more about Import, such as importing different formats, import from the cloud or adding data to existing datasets in [this section.](https://github.com/supervisely/docs/blob/master/getting-started/broken-reference/README.md) {% endhint %} ### Supported formats and modalities
Image Datasets * Auto-detect annotations in [Supervisely](https://github.com/supervisely-ecosystem/import-wizard-docs/blob/master/converter_docs/images/supervisely.md), [COCO](https://github.com/supervisely-ecosystem/import-wizard-docs/blob/master/converter_docs/images/coco.md), [YOLO](https://github.com/supervisely-ecosystem/import-wizard-docs/blob/master/converter_docs/images/yolo.md), [Pascal VOC](https://github.com/supervisely-ecosystem/import-wizard-docs/blob/master/converter_docs/images/pascal.md), [Cityscapes](https://github.com/supervisely-ecosystem/import-wizard-docs/blob/master/converter_docs/images/cityscapes.md), [Images with PNG masks formats](https://github.com/supervisely-ecosystem/import-wizard-docs/blob/master/converter_docs/images/masks.md). * Import images for [Multiview](https://github.com/supervisely-ecosystem/import-wizard-docs/blob/master/converter_docs/images/multiview.md), [Multispectral](https://github.com/supervisely-ecosystem/import-wizard-docs/blob/master/converter_docs/images/multispectral.md), [Medical 2D (single)](https://github.com/supervisely-ecosystem/import-wizard-docs/blob/master/converter_docs/images/medical_2d.md) labeling. * Upload images as [links from CSV or TXT files](https://github.com/supervisely-ecosystem/import-wizard-docs/blob/master/converter_docs/images/csv.md) or [convert PDF pages to images](https://github.com/supervisely-ecosystem/import-wizard-docs/blob/master/converter_docs/images/pdf.md). * Images in any directory structure without annotations. * **Supported image formats:** `.jpg`, `.jpeg`, `jpe`, `.bmp`, `.png`, `.webp`, `.mpo`, `.tiff`, `.nrrd`, `.jfif`, `.avif`, `.heic`.
Video Datasets * Auto-detect annotations in [Supervisely](https://github.com/supervisely-ecosystem/import-wizard-docs/blob/master/converter_docs/videos/supervisely.md), DAVIS (coming soon), MOT (coming soon) formats. * Videos in any directory structure without annotations. * **Supported video formats:** `.avi`, `.mov`, `.wmv`, `.webm`, `.3gp`, `.mp4`, `.flv`. ⚠️ All videos will be converted to `.mp4` format during import.
Point Cloud Datasets * Auto-detect annotations in [Supervisely](https://github.com/supervisely-ecosystem/import-wizard-docs/blob/master/converter_docs/point_cloud/supervisely.md) format. * Point clouds in any directory structure without annotations in `PCD`, `PLY`, `LAS`, `LAZ` formats. * In Import Wizard, `PCD`, `PLY`, `LAS`, and `LAZ` files are imported natively without conversion.
Point Cloud Episode Datasets * Auto-detect annotations in [Supervisely](https://github.com/supervisely-ecosystem/import-wizard-docs/blob/master/converter_docs/point_cloud_episodes/supervisely.md) format. * Point cloud episodes without annotations in `PCD`, `PLY`, `LAS`, `LAZ` formats. * In Import Wizard, `PCD`, `PLY`, `LAS`, and `LAZ` files are imported natively without conversion.
Volume Datasets * Auto-detect annotations in [Supervisely](https://github.com/supervisely-ecosystem/import-wizard-docs/blob/master/converter_docs/volumes/supervisely.md) format. * Volumes in any directory structure without annotations in `DICOM`, `NRRD` formats.
You can always use applications to import different formats and modalities from our [Ecosystem](https://ecosystem.supervisely.com/): [Import Images](https://ecosystem.supervisely.com/apps/import-images) | [Import Videos](https://ecosystem.supervisely.com/apps/import-videos-supervisely) | [Import Pointclouds](https://ecosystem.supervisely.com/apps/import-pointcloud-pcd) | [Import Pointcloud Episodes](https://ecosystem.supervisely.com/apps/import-pointcloud-episode) | [Import DICOM Volumes](https://ecosystem.supervisely.com/apps/import-dicom-volumes) | [Import COCO Keypoints](https://ecosystem.supervisely.com/apps/import-coco-keypoints) | [Import Volumes in Supervisely format](https://ecosystem.supervisely.com/apps/import-volumes-with-anns) | [Import KITTI-360](https://ecosystem.supervisely.com/apps/import-kitti-360/supervisely_app) | [Import Multispectral Images](https://ecosystem.supervisely.com/apps/import-multispectral-images) | and [many other formats](https://ecosystem.supervisely.com/import). 🪄 Here we will look at the fastest and easiest import option! *** Let's start our journey with Supervisely by uploading our very first image. Of course, like we said before, you can import more complex dataset formats like [COCO](https://github.com/supervisely-ecosystem/import-wizard-docs/blob/master/converter_docs/images/coco.md), or modalities, such as DICOM, connect a S3 cloud and much more, but let's begin with a simple one. We assume that you have already created an account in Supervisely. If not, you can create a free account in our Community Edition [here.](https://app.supervisely.com/signup) First thing you will see after you login to Supervisely, is your [Projects](broken://pages/-M54fC5kfcVDMQPT05GQ) page where you can find your data. But there is nothing here yet - let's fix that!
1. Click the `Import Data` button. Enter a unique name for the project, keeping in mind that it must be unique in the workspace and case-sensitive. You can also add a description of the project to provide additional information or to track project updates. 2. Next, select the `Project type` by defining the content modality: images, videos, point clouds, or DICOM 3D volumes. {% hint style="warning" %} Note that you can't mix multiple content types in the same project, and this setting can't be changed later. {% endhint %} 3. Choose one of the available interfaces for labeling images (or other data modality). Our interfaces are designed for different industries and annotation scenarios. 4. After completing all required fields and selecting options, click `Create` to complete the project and begin uploading data.
5. In the modal window, drag and drop one or more images in one of the supported formats, such as `.jpg`, `.jpeg`, `.mpo`, `.bmp`, `.png`, `.webp`, `.tiff`, `.tif`, `.nrrd`, `.jfif`, `.avif`, `.heic`, `NIfTI`, `DICOM` . You can also check out the supported annotation formats. 🤗 Congratulations, the hardest part is over!
You will be redirected to the Tasks page where you can watch the progress of the application (your files are actually being uploaded to your [Team Files](https://docs.supervisely.com/data-organization/team-files)). You can click **three dots (⋮)** icon and check the application logs.
{% hint style="success" %} 🤓 **Nerd alert! Skip this section if you aren't interested how Supervisely works inside.** So what is going on here? First, Supervisely will choose one of the connected Agents and ask it to run the “Auto Import'' application. It will spawn a Docker container that will download the GitHub repository with the application code and run python code written with Supervisely SDK. It will pull your images uploaded to the Team Files in the modal window, convert them, if needed (this particular application maybe does little, but others, like Import COCO format, will transform a lot) and use API to create a Project and add images to it. {% endhint %} Once the import is finished, you will see the link to your new project in the `Output` column of the table.
All set! Now, in the [next section](/getting-started/how-to-annotate), let's annotate your uploaded images. # How to annotate Learn how to use labeling toolboxes, create you first annotations and a little bit more {% hint style="info" %} This 5-minute tutorial is a part of introduction to Supervisely series. You can complete them one-by-one, in random order, or jump to the rest of the documentation at any moment. * [How to import](/getting-started/how-to-import) * How to annotate **(you are here)** * [How to invite team members](/getting-started/invite-member) * [How to connect agents](/agents/connect-your-computer) * [How to train models](/getting-started/how-to-train-models) {% endhint %} {% hint style="success" %} You can learn more about Labeling, such as labeling of videos and 3D point clouds, using AI-assisted labeling and more in [this section.](/labeling/labeling-toolbox) {% endhint %} Once you've [uploaded your first images](/getting-started/how-to-import), let's annotate them with our multiple **annotation tools** - [Bounding Box](https://supervisely.com/blog/bounding-box-annotation-for-object-detection/), [Polygon Tool](https://supervisely.com/blog/how-to-use-polygon-anotation-tool-for-image-segmentation/), [Brush and Eraser Tool](https://supervisely.com/blog/brush/), [Mask Pen Tool](https://supervisely.com/blog/mask-pen-tool/), [Smart Tool](https://supervisely.com/blog/smarttool-annotation/), [Graph (Keypoins) Tool](https://supervisely.com/blog/animal-pose-estimation/), effectively supports 1000+ objects per image. Click on the project you've just created. This will open the list of datasets (subfolders) inside your project (you can learn more about data organization [here](/data-organization/overview)). Depending on whether you uploaded a set of folders with images or just images directly, there may be one or more datasets. {% hint style="info" %} Don't worry - you can move and copy images between datasets (and even projects) using the “three dots” (⋮) or [Data Commander.](/data-organization/data-commander) {% endhint %}
At the top of this page you will find more tabs, such as classes and tags. There you can define a meta information, shared across all of the datasets inside a particular project. Let's focus on [classes](broken://pages/-M54fC5m23ZkaIuwxE1C) first. A class allows you to define a type of your annotations. Every annotation object must have exactly one class. For example, you can define a class “Car” and limit it the shape “bounding box”: now, if you define a [labeling job](/labeling/jobs) and assign your [team member](/collaboration/members) to annotate a bunch of images with “Cars”, they will only be able to place bounding boxes and label them as “Cars” (unless you configure more classes, of course). Now, let's go to the classes tab and click the `New` button. Let's enter some title to it, select a shape (let's select “bounding box” for this one) and click `Save`. {% hint style="info" %} You can select “Any Shape” - that will allow to mark annotations of any shape with this class, so can have both “bounding box” and “mask” marked as this class at the same time. Be worried, that could potentially create issues when you try to train a neural network. {% endhint %}
Awesome! Now, switch back to the **Data** tab and click on any dataset. Once you're inside the dataset, selecting any image or clicking the `Annotate` button will bring up a dialog asking you to choose a labeling toolbox ( yeah, we have lots!). Select an “Image labeling toolbox 2.0”. A new tab should appear with your dataset in the labeling toolbox. {% hint style="info" %} You can label all data in your project at once by using the `Annotate` button on the dataset page. {% endhint %}
{% hint style="info" %} You can switch the dark theme to light or back at any time. {% endhint %} From the left toolbar select a tool that corresponds to your class shape, in our case, [Bounding Box Tool](https://supervisely.com/blog/bounding-box-annotation-for-object-detection/) to create bounding boxes. You can hover your cursor over the particular tool button and check the description. Now, let's hover the cursor over any object on your image and make two clicks to form a rectangle around it. You will see a new object in the right sidebar at the `Objects` tab. Done!
You can also check our blog post on how to label with bounding boxes: {% embed url="" %} You rock! Now, you can explore other labeling tools (such as [polygons](https://supervisely.com/blog/how-to-use-polygon-anotation-tool-for-image-segmentation/), [masks](https://supervisely.com/blog/smarttool-annotation/) or [skeleton shapes](https://supervisely.com/blog/human-pose-estimation/)), more advanced tools, such as video or [3D point clouds](https://supervisely.com/blog/3d-object-interpolation-in-point-clouds/) - or continue our journey and see how [collaboration](/collaboration/members) works in Supervisely. # How to invite team members Learn how to connect more people to your team and do things together {% hint style="info" %} This 5-minute tutorial is a part of introduction to Supervisely series. You can complete them one-by-one, in random order, or jump to the rest of the documentation at any moment. * [How to import](/getting-started/how-to-import) * [How to annotate](/getting-started/how-to-annotate) * How to invite team members **(you are here)** * [How to connect agents](/agents/connect-your-computer) * [How to train models](/getting-started/how-to-train-models) {% endhint %} {% hint style="success" %} If you want to learn more about collaboration, team members, labeling jobs and much more, then you need to check [this section](/collaboration/teams). {% endhint %} We have now learned how to [upload](/getting-started/how-to-import) and [annotate](/getting-started/how-to-annotate) datasets - that's great! But when it comes to a more realistic scenario of creating a computer vision dataset, you will need more than just you alone. You will need a team of people working together on the same data. So, let's invite one! First, login to Supervisely and click `Current Team members` - you will see the list of users who are members of the current [team](/collaboration/teams) (which you can find right under the `Team` drop-down list). Presumably, it's just you alone now. Let's fix that!
Click the `Invite` button at the top right corner. You will need to enter a login of an existing user and a role in the current team you want them to have. [Roles](/collaboration/members) can be used to limit which actions the user can perform in your team. You, as an admin of the team, can do anything, delete things, invite and pick other users - but, say, a user with the “labeler” role can do none of the above: they can only label existing images. Once you clicked `Add to team`, depending on your Supervisely settings, will send an invitation e-mail or add the user immediately (only available on the Enterprise Edition). Now, when the invited user will login to Supervisely, they will be able to switch between any team they are a member of. Friendship is magic! 🤝 Once you have your datasets uploaded and labeled together with your team, it's time to do something cool - train some neural networks! But, before that, we need a way to deploy training applications on a computer with GPU: and here when you will need an [agent](/agents/connect-your-computer). # How to connect agents Learn how to run model training or heavy processing on any computer in a single click {% hint style="info" %} This 5-minute tutorial is a part of introduction to Supervisely series. You can complete them one-by-one, in random order, or jump to the rest of the documentation at any moment. * [How to import](/getting-started/how-to-import) * [How to annotate](/getting-started/how-to-annotate) * [How to invite team members](/getting-started/invite-member) * How to connect agents **(you are here)** * [How to train models](/getting-started/how-to-train-models) {% endhint %} Supervisely Agent is a tiny docker container that allows you to connect your computational resources (cloud server or PC) to the platform. You can run any task from web interface (for example Neural Network training/inference/deploy). Running tasks with GPU will enhance performance and efficiency for your computer vision and deep learning projects. After you run Agent on your computer, Agent will automatically connect your server to Supervisely platform. You will see this information on the "Team Cluster" page.
{% hint style="info" %} Only you and your team members have access to your agents. So only tasks that you explicitly started yourself run on them. We will never use your nodes for our own benefit or the benefit of other users. {% endhint %}
Installation LinuxEverything you need to know about deploying Supervisely agent on Unix-based operating systems./pages/lSHXH2eHBOkzuvBj8IZU
Installation WindowsEverything you need to know about deploying Supervisely agent on Windows WSL./pages/MHMZN2d5GzCfd3f0pEWY
Installation AMI AWSIf, for some reason, your computer doesn't meet the requirements, hardware (no GPU) or software (no CUDA or nvidia-docker), there is a quick way to try training & inference with Supervisely on Amazon EC2./pages/0iVl6TX1AR0ACLPHpUqP
Installation KubernetesFollow these steps to deploy Supervisely in Kubernetes cluster/pages/GTZ7wYT3N8fQTfFrrkl1
# How to train models Learn how to use Supervisely Apps to train custom AI models, deploy them on your GPU and use in the labeling toolboxes {% hint style="info" %} This 5-minute tutorial is a part of introduction to Supervisely series. You can complete them one-by-one, in random order, or jump to the rest of the documentation at any moment. * [How to import](/getting-started/how-to-import) * [How to annotate](/getting-started/how-to-annotate) * [How to invite team members](/getting-started/invite-member) * [How to connect agents](/agents/connect-your-computer) * How to train models **(you are here)** {% endhint %} {% hint style="success" %} To learn more about training a model on your custom data and pre-labeling (getting predictions from) your images with the trained models, [watch this video tutorial](https://youtu.be/Rsr8xWJ6s9I?si=6PudzCqY1C-lMcWa). {% endhint %} We will provide a step-by-step guide for training a custom model, using[ YOLO (v8, v9)](https://ecosystem.supervisely.com/apps/yolov8/train) as an example. Supervisely offers a no-code solution for training, deploying and predicting with [YOLO (v8, v9)](https://ecosystem.supervisely.com/apps/yolov8/train) models directly in your web browser, leveraging user-friendly interfaces and integrated tools. ## Step 1. Prepare training data You have several options for preparing your training data: * [Upload](/getting-started/how-to-import) your images, [label](/getting-started/how-to-annotate) them, and then train a custom neural network model. {% hint style="info" %} We recommend starting experiments with several hundred images. Continuously improve your object detection neural network by adding new images, especially those where the model's accuracy is lower. {% endhint %} * Import your [existing training dataset](https://ecosystem.supervisely.com/import) (such as [COCO](https://ecosystem.supervisely.com/apps/import-coco) and [YOLOv5](https://ecosystem.supervisely.com/apps/convert-yolov5-to-supervisely-format)) and try to build neural network model directly on your custom data. * Pick [ready-to-use data](/import-and-export/import/import-sample-dataset) we prepared for you and reproduce this tutorial from start to end. ## Step 2. Deploy an agent Before you start training or running neural networks, you need to connect your PC or a cloud server with a GPU to Supervisely by running a simple command in your terminal. This connection allows you to train neural networks and run inference directly from the Supervisely web interface. You can find detailed instructions on how to do this [here](/agents/connect-your-computer). {% hint style="warning" %} Ensure no network configuration is needed, and the connection is secure and private. {% endhint %} {% embed url="" %} ## Step 3. Train a model 1. Open the training app from your labeled data project, click the `[⫶]` button → **Neural Networks** → YOLO → [Train YOLO (v8, v9)](https://ecosystem.supervisely.com/apps/yolov8/train).

How to run the YOLOv8 training App from your custom training dataset

2. Follow the wizard to configure the main training settings, similar to those allowed by the original repository. You can: * Choose all or a subset of classes for training. * Define training and validation splits. * Select one of the available model architectures. * Configure training hyperparameters, including augmentations. 3. Press the `Train` button and monitor logs, charts and visualizations in real-time.
4. The training process generates artifacts, including model weights (checkpoints), logs, charts, additional visualizations of training batches, predictions on validation data, precision-recall curves, confusion matrices and so on. These artifacts will be automatically saved to your **Team Files**.
## Step 4. Deploy a trained model Once the model is trained, you probably want to try it on your data and evaluate its performance. 1. Use the [Serve YOLO (v8, v9)](https://ecosystem.supervisely.com/apps/yolov8/serve) app to deploy your model as a REST API service so it can receive images and return predictions in response. 2. Provide the checkpoint (model weights file in `.pt` format) and follow the app's instructions.
{% hint style="info" %} In Supervisely you can quickly deploy custom or pretrained neural network models weights on your GPU using the [Serve Supervisely Applications](https://app.supervisely.com/nn/apps) in just a few clicks. {% endhint %} ## Step 5. Get predictions #### Option 1. Integrate model in Labeling Interface Use the [NN Image Labeling ](https://ecosystem.supervisely.com/apps/nn-image-labeling/annotation-tool)app to apply your model to images or regions of interest during annotation, configure inference settings like confidence thresholds or select all or several model classes. This approach gives you the ability to automatically pre-label images and then just manually correct model mistakes
#### **Option 2. Apply model to all images at once** Use the [Apply NN to Images Project](https://ecosystem.supervisely.com/apps/nn-image-labeling/project-dataset) app to pre-label all images in a project. Follow the wizard to configure settings and run batch inference (connect to the model, select model classes, configure inference settings, and preview predictions). The app will iterate over all images in your project, apply your model in a batch manner, and save all predicted labels to a new project.

Apply the custom model to all images in your project in a few clicks

## Step 6. Export weights The trained model can be easily exported and used outside the platform. Go to the directory with training artifacts in your **Team Files** and download the model weights in PyTorch (`.pt`) format for external use.

Just download the trained model and use it outside the Supevisley platform

Now you can follow the [YOLOv8 documentation](https://docs.ultralytics.com/modes/predict/) to get predictions on images. Here is a Python example of inference: ```python from ultralytics import YOLO # Load your model model = YOLO("my_checkpoint.pt") # Predict on an image results = model("/a/b/c/my_image.jpg") # Process results list for result in results: boxes = result.boxes # Boxes object for bbox outputs masks = result.masks # Masks object for segmentation masks outputs keypoints = result.keypoints # Keypoints object for pose outputs ``` You can check the main section of the documentation on neural networks: {% content-ref url="/pages/-M5Jh8BK8CuqzBET\_pWc" %} [Overview](/neural-networks/overview) {% endcontent-ref %} # Import As you already know, we have a concept of [projects and datasets](/data-organization/overview). Now, how can you import your dataset to Supervisely? Dataset formats are very different (coco, cityscapes and others), and there are many more modalities (images, videos and others), and even just with images, there are many variations of file formats. We don't want you to convert anything yourself, so, to deal with that, here at Supervisely we've put together a number of [Supervisely Apps](https://ecosystem.supervisely.com/import) for every format out there (and if we don't have one, you can write an [app import yourself](https://supervisely.readthedocs.io/en/latest/sdk_packages.html) or use the [API](https://api.docs.supervisely.com/)). Using Supervisely Apps or API, you can turn your images, videos and annotations into Supervisely projects and datasets: they will be stored in the [Supervisely Format](https://github.com/supervisely/docs/blob/master/data-organization/supervisely-format.md) and at any time you can [download](/import-and-export/export) them in this or another format. Supervisely has three ways how to store your assets: **Store files locally** The default strategy is to place all uploaded and generated assets to the storage on the same server where Supervisely is installed, for example, on a hard drive. **Store files remotely** The instance administrator can configure the Supervisely platform to upload and store all generated assets to a remote cloud storage provider, such as S3. This can affect performance, since files need to be transferred over the network, but it is a more reliable method of data storage. This is only available on Enterprise Edition. **Store individual files remotely (“Import by Link”)** The hybrid approach that takes the best of both worlds. In this scenario, you don't store files locally (so that generated or uploaded manually files end up on a hard drive), but use [special Supervisely Apps](/import-and-export/import/import-from-cloud) or API methods to add images or videos to datasets “by link”. That means, that instead of uploading the actual content of the file, you provide a resource url (such as “https\://…” or “s3://…”) and Supervisely will not store the content on the platform - instead, it will load it from your remote provider on-fly. This method allows you to import huge datasets almost instantly, but you will have to manage the remote storage yourself. {% hint style="info" %} Supervisely calculate file hashes when you upload your assets: because of that, we do not store duplicates. {% endhint %}
Import using Web UIThe most simple and straightforward method of importing is uploading your data using one of our Supervisely Apps./pages/u5q5rsb3n3M5M1BHRuRQ
Import sample datasetSave valuable time by starting with already prepared datasets. We provide access to a variety of ready-made data to speed up your start./pages/50RO94cBianJmndXyY9u
Import into an existing datasetIt is possible to add more assets such as images to the existing project or dataset./pages/gliCuTC2aAq8BLJaHUmb
Import using Team Filesyou can just select the appropriate Supervisely App from the context menu of a folder in your Team Files - and enjoy./pages/EDLlzaF1QvjE1EyCI1YR
Import from CloudWant to contribute to Supervisely? Start with our GitHub page here./pages/rObdqA3dJE7gifdhhUGG
Import using API & SDKSave valuable time by starting with already prepared datasets./pages/IvpUcWiH7PlF5ST80PHW
# Import using Web UI {% hint style="info" %} Check our 5-minute tutorial on [how to import your first images to Supervisely](/getting-started/how-to-import). {% endhint %} The easiest and most straightforward import method is to load data using **Quick Import**. To get started, click the `Import Data` button (if you don't have any projects), `+ New` button or the interactive tile on the Project page.

3 quick ways to create a project and import data

Next, follow these step-by-step guide: 1. **Name and describe the project.** Enter a unique name for the project. Ensure the name is unique in the workspace and note that it is case-sensitive. (Optional) Add a description to provide additional information about the project or to track updates. 2. If you aren't a new user, you can click the `Create from template` and choose the source project from which to copy the classes and tags. Thus you can select projects from any of your team workspaces. 3. **Define project type.** Select the content modality for the project: images, videos, point clouds, or DICOM 3D volumes. {% hint style="warning" %} **Note**: You can't mix multiple content types in a single project, and this setting can't be changed later. {% endhint %} 4. **Choose labeling interface.** Select one of the available interfaces for labeling your data. These interfaces are cover different industries and annotation scenarios. 5. Click **Create** to finish the project creation and proceed to uploading data.
6. **Drag & drop** one or more images of supported formats into the modal window: .jpg, .jpeg, .mpo, .bmp, .png, .webp, .tiff, .tif, .nrrd, .jfif, .avif, .heic, NIfTI, DICOM. 7. You can view supported annotation formats. Check format you are interested in by clicking on its title.
You will be redirected to the **Tasks** page where you can monitor the upload progress. To check application logs, click the **three dots (⋮)** icon next to the task.
Once the import is finished, you will see the link to your new project in the `Output` column of the table (or find it at the **Projects** page).
### **How to import images with applications.** 1. Click the `More Features` tab. Navigate to the **Import** page in the `Categories` section. Locate the **Import Images** application.
2. Move the cursor over the application and click the `Run Application` button. You can always have a look at the description on the application page and follow the instructions.
3. In the modal window **drag & drop** a folder with images or images itself. Enter the name of the future project and click `Run` button.
3. You will be redirected to the **Tasks** page where you can watch import progress. When it is done, you will see the link to your new project (or find it at the Projects page).
# Supported annotation formats # Images # Supervisely JSON ## Overview {% hint style="success" %} Easiest way to import your images with annotations is to use the Supervisely format. Check out the [Supervisely JSON format](/customization-and-integration/00_ann_format_navi) documentation for more details. {% endhint %} The Supervisely json-based annotation format supports such figures: `rectangle`, `line (polyline)`, `polygon`, `multipolygon`, `point`, `bitmap` (`mask`), `graph` (`keypoints`), `alpha mask`, `2D cuboid`. It is a universal format for various task types and is used in the Supervisely platform. Enterprise users have access to "Import as links" option, which supports import of this format with annotations. This option might be beneficial in many cases, as it allows data import to Supervisely platform without re-uploading, maintaining a single source and speeding up import process. To step up import speed even further you can compress all annotation files (`.json`'s) into an archive and import it together with the images. (Note: This method is format-dependent and may not apply to all formats.) ## Format description **Supported image formats:** `.jpg`, `.jpeg`, `.mpo`, `.bmp`, `.png`, `.webp`, `.tiff`, `.tif`, `.jfif`, `.avif`, `.heic`, and `.heif`\ **With annotations:** Yes\ **Supported annotation file extension:** `.json`.\ **Grouped by:** Any structure (will be uploaded as a single dataset)\\ ## Input files structure {% hint style="success" %} Example data: [download ⬇️](https://github.com/supervisely-ecosystem/import-images-in-sly-format/files/12537201/robots_project.zip). {% endhint %} Both directory and archive are supported. **Recommended directory structure:** ``` 📦input_folder ┣ 📂dataset_name_01 ┃ ┣ 📂ann ┃ ┃ ┣ 📄IMG_0748.jpeg.json ┃ ┃ ┗ 📄IMG_8144.jpeg.json ┃ ┣ 📂img ┃ ┃ ┣ 🏞️IMG_0748.jpeg ┃ ┃ ┗ 🏞️IMG_8144.jpeg ┃ ┗ 📂meta (optional) ┃ ┣ 📄IMG_0748.jpeg.json ┃ ┗ 📄IMG_8144.jpeg.json ┗ 📄meta.json ``` Project meta file `meta.json` is recommended to be present in the project directory. It contains classes and tags definitions for the project. If it is not present, app will try to create it from the annotations (if possible). Learn more about the `meta.json` file [here](https://docs.supervisely.com/customization-and-integration/00_ann_format_navi/02_project_classes_and_tags). {% hint style="info" %} **Struggled with the structure?** No worries! If you don't have the recommended structure, don't worry. You can upload images and annotations in any structure. In this case, the app will upload all images and annotations to a single dataset. Just make sure that: * Annotation files are in the `.json` format. * Annotation files have the corresponding file name to the image file name (e.g. `image_1.jpg.json` is for the image `image_1.jpg`). * Annotation files have the correct format (look at the example below). * Image files are in the supported formats (provided above). * Image and annotation files can be placed in any subdirectories or the root directory. {% endhint %} ## Single-Image Annotation JSON For each image, we store the annotations in a separate `json` file named `image_name.image_format.json` with the following file structure: ```json { "description": "food", "name": "tomatoes-eggs-dish.jpg", "size": { "width": 2100, "height": 1500 }, "tags": [], "objects": [] } ``` **Fields definitions:** * `name` - string - image name * `description` - string - (optional) - This field is used to store the text we want to assign to the image. In the labeling intrface it corresponds to the 'data' filed. * `size` - stores image size. Mostly, it is used to get the image size without the actual image reading to speed up some data processing steps. * `width` - image width in pixels * `height` - image height in pixels * `tags` - **list** of strings that will be interpreted as image [tags](https://docs.supervisely.com/customization-and-integration/00_ann_format_navi/03_supervisely_format_tags) * `objects` - **list** of [objects on the image](https://docs.supervisely.com/customization-and-integration/00_ann_format_navi/04_supervisely_format_objects) ### Image annotation example with objects and tags ![Image annotation example](/files/0il4WOUUATMSPYLrnxHu) Example: ```json { "description": "", "tags": [ { "id": 86458971, "tagId": 28283797, "name": "like", "value": null, "labelerLogin": "alexxx", "createdAt": "2020-08-26T09:12:51.155Z", "updatedAt": "2020-08-26T09:12:51.155Z" }, { "id": 86458968, "tagId": 28283798, "name": "situated", "value": "outside", "labelerLogin": "alexxx", "createdAt": "2020-08-26T09:07:26.408Z", "updatedAt": "2020-08-26T09:07:26.408Z" } ], "size": { "height": 952, "width": 1200 }, "objects": [ { "id": 497521359, "classId": 1661571, "description": "", "geometryType": "bitmap", "labelerLogin": "alexxx", "createdAt": "2020-08-07T11:09:51.054Z", "updatedAt": "2020-08-07T11:09:51.054Z", "tags": [], "classTitle": "person", "bitmap": { "data": "eJwBgQd++IlQTkcNChoKAAAADUlIRF", "origin": [535, 66] } }, { "id": 497521358, "classId": 1661574, "description": "", "geometryType": "rectangle", "labelerLogin": "alexxx", "createdAt": "2020-08-07T11:09:51.054Z", "updatedAt": "2020-08-07T11:09:51.054Z", "tags": [], "classTitle": "bike", "points": { "exterior": [ [0, 236], [582, 872] ], "interior": [] } } ] } ``` ## Useful links * [Supervisely Annotation Format](https://developer.supervisely.com/getting-started/supervisely-annotation-format) * [Supervisely Image Annotation](https://developer.supervisely.com/getting-started/supervisely-annotation-format/images) * [\[SDK CLI\] Upload projects in Supervisely format](https://developer.supervisely.com/getting-started/command-line-interface/sdk-cli#upload-a-project) * [\[CLI Tool Beta\] Upload projects in Supervisely format](https://developer.supervisely.com/getting-started/command-line-interface/cli-tool/workflow-automation#upload-projects-in-supervisely-format) * [\[Supervisely Ecosystem\] Import images in Supervisely format](https://ecosystem.supervisely.com/apps/import-images-in-sly-format) ![](https://i.imgur.com/Y6RcQPT.png) # Supervisely Blob ## Overview When dealing with large quantities of small images (e.g., thousands of images under 100KB each), importing them individually is inefficient. The blob approach combines multiple images into a single archive file, making transfer and storage more efficient. ## Annotations with Blob format The key advantage of the blob format is that it optimizes storage and transfer of image data without changing how annotations work. When using blob format: * **Annotations remain in the standard Supervisely JSON format** exactly as described in the [Supervisely JSON](/import-and-export/import/supported-annotation-formats/images/supervisely) documentation * Each annotation file still corresponds to a specific image by name * All annotation features (rectangles, polygons, masks, etc.) work exactly the same way * The only difference is how the image data itself is stored and accessed {% hint style="info" %} This approach gives you the best of both worlds: efficient storage and transfer of image data while maintaining the flexible and powerful Supervisely annotation system you're already familiar with. {% endhint %} ### What is a Blob File? A blob file in Supervisely is essentially a `.tar` archive that contains multiple images bundled together. Instead of storing and transferring each image as a separate file, these images are packed into a single large file (the blob). This approach: * Reduces the number of network requests needed for transfers * Minimizes filesystem overhead when dealing with many small files ### What is an Offset File? An offset file `.pkl` is a companion file to the blob archive that contains metadata about where each image is located within the blob file. Specifically: * It maps each image filename to its exact byte position (start and end offsets) in the blob file * Allows direct extraction of specific images without scanning the entire archive * Stored as a Python pickle file containing batches of dictionaries with image names as keys and offset positions as values These two files work together to provide efficient storage and random access to large collections of small images. Benefits include: * Faster import and export speeds * Reduced server load * More efficient storage on disk ### Offset Representation The `BlobImageInfo` class of [Supervisely Python SDK](https://supervisely.readthedocs.io/en/latest/index.html) represents image metadata within a blob storage file. It contains information about where the image data is located in the blob file, defined by byte offsets. This class provides methods to manipulate and convert blob image information to formats suitable for storage and API interactions. {% hint style="success" %} Once blob files are uploaded to Team Files, you can reuse them for multiple projects without re-uploading the images. {% endhint %} This approach helps optimize the import process for multiple projects since you don't need to re-upload the original images each time. By simply creating and uploading different offset files, you can import different subsets of images from the same blob archive. ### Recommended Project Structure A typical blob-based project structure looks like this: ``` 📂 project-name ┣ 📂 blob ┃ ┗ 📦 small_images.tar ┣ 📂 dataset-name-001 ┃ ┣ 📄 small_images_offsets.pkl ┃ ┣ 📂 ann ┃ ┃ ┣ 📄 small-image-0000001.png.json ┃ ┃ ┣ ... ┃ ┃ ┗ 📄 small-image-0999999.png.json ┗ 📄 meta.json ``` For detailed information about blob project structure, refer to the extended [Project Structure documentation](/customization-and-integration/00_ann_format_navi/01_project_structure_new#extended-project-structure). ## Performance Comparison Information A blob project with 30000 small images (\~4KB each) can be: * Uploaded `~2x` faster than standard uploads, `~x14` especially using fast methods `coming soon in apps` * Downloaded `~4x` faster than standard downloads, `~22x` especially using fast methods ## Useful links * [Supervisely Annotation Format](https://developer.supervisely.com/getting-started/supervisely-annotation-format) * [Supervisely Image Annotation](https://developer.supervisely.com/getting-started/supervisely-annotation-format/images) * [\[SDK CLI\] Upload projects in Supervisely format](https://developer.supervisely.com/getting-started/command-line-interface/sdk-cli#upload-a-project) * [\[CLI Tool Beta\] Upload projects in Supervisely format](https://developer.supervisely.com/getting-started/command-line-interface/cli-tool/workflow-automation#upload-projects-in-supervisely-format) * [\[Supervisely Ecosystem\] Import images in Supervisely format](https://ecosystem.supervisely.com/apps/import-images-in-sly-format) # COCO ## Overview This converter allows to import images with annotations in [COCO](https://cocodataset.org/#home) format. COCO format has all annotations in one `.json` file. "Auto Import" app supports the following COCO annotation types: **instances**, **keypoints**, **captions**. ![Result of the import](/files/MyTaDnYhlvuDW15AQFV1) Enterprise users have access to "Import as links" option, which supports import of this format with annotations. This option might be beneficial in many cases, as it allows data import to Supervisely platform without re-uploading, maintaining a single source and speeding up import process. To step up import speed even further you can compress all annotation files (`.json`'s) into an archive and import it together with the images. (Note: This method is format-dependent and may not apply to all formats.) ## Format description **Supported image formats:** `.jpg`, `.jpeg`, `.mpo`, `.bmp`, `.png`, `.webp`, `.tiff`, `.tif`, `.jfif`, `.avif`, `.heic`, and `.heif`\ **With annotations:** yes\ **Supported annotation file extension:** `.json`.\ **Grouped by:** any structure (will be uploaded as a single dataset)\\ ## Default option: Import images and annotations together Use this option **if you have images and annotations in COCO format** and you want to upload them together to Supervisely. {% hint style="success" %} We prepared sample datasets in COCO format for you to try the import process: * instances: [download ⬇️](https://github.com/supervisely-ecosystem/import-wizard-docs/files/14918161/sample_coco.zip) * keypoints: [download ⬇️](https://github.com/supervisely-ecosystem/import-wizard-docs/files/14918389/sample_coco_keypoints.zip) {% endhint %} Recommended directory structure: ``` 📦project name ┗ 📂dataset ┣ 📂annotations ┃ ┗ 📜instances.json ┗ 📂images ┣ 🖼️0001.png ┣ 🖼️0002.png ┣ 🖼️0003.png ┣ 🖼️0004.png ┗ 🖼️0005.png ``` ## Advanced option: how to speed up the import process 🏋️‍♂️ Use this option **if you have a large dataset already uploaded to Supervisely** and you don't want to upload images again (for example, you have a dataset with images and you want to upload annotations only). All you need to do is upload the JSON file with annotations in COCO format. The application will match the annotations with the images by their names and upload the annotations to the existing dataset. Key points: * **Press `+ Import data` button inside the dataset**: you need to run the import process from the dataset that contains the images. * **Image names**: the application will match the annotations with the images by their names. So, make sure that the names of the images in the dataset match the names of the images in the COCO annotations file. * **Impact on existing annotations**: new annotations will be merged with the existing ones. If you want to keep the original annotations, clone the dataset before importing new annotations. ## COCO Annotation COCO format is a complex format that can contain multiple types of annotations. Supervisely import supports only `instances`, `keypoints`, and `captions`. The COCO dataset is formatted in `.json` and is a dictionary of keys `info`, `licenses`, `images`, `annotations`, and `categories` (in most cases). * `info` - contains high-level information about the dataset * `licenses` - contains a list of image licenses that apply to images in the dataset. * `images` - contains the complete list of images in your dataset. Note that image ids need to be unique among other images. * `annotations` - contains a list of every individual object annotation from every image in the dataset. * `categories` - contains a list of categories (e.g. dog, boat) and each of those belongs to a supercategory (e.g. animal, vehicle). The original COCO dataset contains 90 categories. You can use the existing COCO categories or create an entirely new list of your own. Each category ID must be unique among the rest of the categories. ### Instances Regions of interest indicated by these annotations are specified by `segmentations`, which are usually a list of polygon vertices around the object, but can also be a run-length-encoded (RLE) bit mask. Typically, RLE is used for groups of objects (like a large stack of books). Example annotation for instances for one image in COCO format:
📜 instances.json ```json { "info": { "description": "", "url": "None", "version": "1.0", "year": 2023, "contributor": "Supervisely", "date_created": "2023-08-22T09:33:23.811Z" }, "licenses": [ { "url": "None", "id": 0, "name": "None" } ], "images": [ { "license": "None", "file_name": "IMG_1836.jpeg", "url": "None", "height": 800, "width": 1067, "date_captured": "2023-08-22T09:33:23.890Z", "id": 22027400 } ], "annotations": [ { "segmentation": [[759.0, 429.0, ..., 765.0, 423.0]], "area": 29889.5, "iscrowd": 0, "image_id": 22027400, "bbox": [752.0, 421.0, 257.0, 167.0], "category_id": 2, "id": 1 }, { "segmentation": [[665.0, 128.0, ..., 673.0, 132.0]], "area": 15603.5, "iscrowd": 0, "image_id": 22027400, "bbox": [569.0, 122.0, 137.0, 151.0], "category_id": 1, "id": 2 }, { "segmentation": [[563.0, 542.0, ..., 572.0, 549.0]], "area": 15740.5, "iscrowd": 0, "image_id": 22027400, "bbox": [464.0, 539.0, 131.0, 151.0], "category_id": 1, "id": 3 } ], "categories": [ { "supercategory": "kiwi", "id": 1, "name": "kiwi" }, { "supercategory": "lemon", "id": 2, "name": "lemon" } ] } ```
### Keypoints Annotations for keypoints are just like in Object Detection (Segmentation) above, except a number of keypoints is specified in sets of 3, (x, y, v). * **x** and **y** indicate pixel positions in the image. * **v** indicates visibility - v=0: not labeled (in which case x=y=0), v=1: labeled but not visible (behind an object for example), and v=2: labeled and visible. Only keypoints with v=2 will be uploaded to the project. Example of the annotation file with keypoints:
📜 instances.json with keypoints ```json { "info": { "description": "", "url": "None", "version": "1.0", "year": 2023, "contributor": "Supervisely User", "date_created": "2023-09-15T16:36:43.593Z" }, "licenses": [ { "url": "None", "id": 0, "name": "None" } ], "images": [ { "license": "None", "file_name": "pexels-photo-175706.png", "url": "None", "height": 800, "width": 1292, "date_captured": "2023-09-15T16:36:43.742Z", "id": 23364344 } ], "annotations": [ { "segmentation": [], "area": 608998, "iscrowd": 0, "image_id": 23364344, "bbox": [617.0, 279.0, 152.0, 517.0], "category_id": 1, "id": 1, "keypoints": [727, 295, 2, ..., 758 ,794 ,2], "num_keypoints": 17 } ], "categories": [ { "supercategory": "person", "id": 1, "name": "person", "keypoints": [ "nose", "left_eye", "right_eye", "left_ear", "right_ear", "left_shoulder", "right_shoulder", "left_elbow", "right_elbow", "left_wrist", "right_wrist", "left_hip", "right_hip", "left_knee", "right_knee", "left_ankle", "right_ankle" ], "skeleton": [ [16,14], [14,12], [17,15], [15,13], [12,13], [6,12], [7,13], [6,7], [6,8], [7,9], [8,10], [9,11], [2,3], [1,2], [1,3], [2,4], [3,5], [4,6], [5,7] ] } ] } ```
### Captions Image caption annotations are pretty simple. There are no categories in this `.json` file, just annotations with caption descriptions.
📜 instances.json with captions ```json { "info": { "description": "", "url": "None", "version": "1.0", "year": 2023, "contributor": "Supervisely", "date_created": "2023-08-22T09:33:23.811Z" }, "licenses": [ { "url": "None", "id": 0, "name": "None" } ], "images": [ { "license": "None", "file_name": "IMG_1836.jpeg", "url": "None", "height": 800, "width": 1067, "date_captured": "2023-08-22T09:33:23.890Z", "id": 22027400 } ], "annotations": [ { "image_id": 22027400, "id": 1, "caption": "An image of 2 pieces of kiwi and 1 lemon." } ] } ```
## Useful links * [COCO dataset](https://cocodataset.org/#home) * [COCO annotation structure](https://www.immersivelimit.com/tutorials/create-coco-annotations-from-scratch) * [\[Supervisely Ecosystem\] Import COCO](https://ecosystem.supervisely.com/apps/import-coco) # Yolo ## Overview This converter allows to import images with annotations in [YOLO](https://docs.ultralytics.com/datasets/detect/) format for **segmentation**, **detection** and **pose estimation** tasks. Each image should have a corresponding `.txt` file with the same name, which contains information about objects in the image. * Segmentation labels will be converted to polygons. Labels format: ` ... ` * Detection labels will be converted to rectangles. Labels format: ` ` * Pose estimation labels will be converted to keypoints. Labels format: ` ... ` for Dim=2 and ` ` for Dim=3. YOLO format data should have a specific configuration file that contains information about classes and datasets, usually named `data_config.yaml`. ⚠️ **Note:** If the input data does not contain `data_config.yaml` file, it will use default COCO class names. ![Result of the import](/files/xfnjmhqbFVfeiaR3Xjv2) Enterprise users have access to "Import as links" option, which supports import of this format with annotations. This option might be beneficial in many cases, as it allows data import to Supervisely platform without re-uploading, maintaining a single source and speeding up import process. To step up import speed even further you can compress all annotation files (`.txt`'s) into an archive and import it together with the images. (Note: This method is format-dependent and may not apply to all formats.)
Default COCO class names ``` names: [ "person", "bicycle", "car", "motorcycle", "airplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "couch", "potted plant", "bed", "dining table", "toilet", "tv", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush", ] ```
## Format description **Supported image formats:** `.jpg`, `.jpeg`, `.mpo`, `.bmp`, `.png`, `.webp`, `.tiff`, `.tif`, `.jfif`, `.avif`, `.heic`, and `.heif`\ **With annotations:** Yes\ **Supported annotation file extension:** `.txt`.\ **Grouped by:** Any structure (will be uploaded as a single dataset)\\ ## Input files structure {% hint style="success" %} Example data: [download ⬇️](https://github.com/supervisely-ecosystem/import-wizard-docs/files/14919196/sample_yolo.zip)\\ {% endhint %} Recommended directory structure: ``` 📂project name ┣ 📂images ┃ ┣ 📂train ┃ ┃ ┣ 🖼️IMG_0748.jpeg ┃ ┃ ┣ 🖼️IMG_1836.jpeg ┃ ┃ ┣ 🖼️IMG_2084.jpeg ┃ ┃ ┗ 🖼️IMG_3861.jpeg ┃ ┗ 📂val ┃ ┣ 🖼️IMG_4451.jpeg ┃ ┗ 🖼️IMG_8144.jpeg ┣ 📂labels ┃ ┣ 📂train ┃ ┃ ┣ 📜IMG_0748.txt ┃ ┃ ┣ 📜IMG_1836.txt ┃ ┃ ┣ 📜IMG_2084.txt ┃ ┃ ┗ 📜IMG_3861.txt ┃ ┗ 📂val ┃ ┣ 📜IMG_4451.txt ┃ ┗ 📜IMG_8144.txt ┗ 📜data_config.yaml ``` ## Format Config File File `data_config.yaml` should contain the following keys: * `names` - a list of class names * `colors` - a list of class colors in RGB format * `nc` - the number of classes * `train` - the path to the train images * `val` - the path to the validation images
📜data_config.yaml ```yaml names: [kiwi, lemon] # class names colors: [[255, 1, 1], [1, 255, 1]] # class colors nc: 2 # number of classes train: ../lemons/images/train # path to train imgs (or "images/train") val: ../lemons/images/val # path to val imgs (or "images/val") # Keypoints (for pose estimation) kpt_shape: [17, 3] # number of keypoints, number of dims (2 for x,y or 3 for x,y,visible) ```
## Single-Image Annotation Annotation files are in `.txt` format and should contain object labels on each line: * Class numbers that correspond to the class names in the `data_config.yaml` file. * Label coordinates must be in normalized format (from 0 to 1). **1. Segmentation** Labels should be formatted with one row per object in: ``` ... ``` **2. Detection:** Labels should be formatted with one row per object in: ``` ``` If your boxes are in pixels, you should divide x\_center and width by image width, and y\_center and height by image height. **3. Pose Estimation:** Labels should be formatted with one row per object. For Dim=2: ``` ... ``` For Dim=3: ``` ... ``` **Yolo coordinates explanation:** The label file corresponding to the below image contains 2 persons (class 0) and a tie (class 27) from original COCO classes. 📜zidan.txt: ``` 0 0.481719 0.634028 0.690625 0.713278 0 0.741094 0.524306 0.314750 0.933389 27 0.364844 0.795833 0.078125 0.400000 ``` ![Yolo coordinates explanation](/files/i5iqUASJACnLmEDLmjZf) ## Useful links * [YOLO format](https://docs.ultralytics.com/datasets/detect/) * [\[Supervisely Ecosystem\] Convert YOLO v5 to Supervisely format](https://ecosystem.supervisely.com/apps/convert-yolov5-to-supervisely-format) # Pascal VOC ## Overview The Pascal VOC (Visual Object Classes) format stands as one of the benchmarks established relatively early for object classification, segmentation and detection. It furnishes a standardized dataset for identifying object classes, utilizing an XML-based export format that enjoys widespread adoption in computer vision tasks. This converter converts Pascal VOC format to Supervisely format. Learn more how to prepare data in Pascal VOC format and how to import the original Pascal VOC dataset in the [Import Pascal VOC](https://ecosystem.supervisely.com/apps/import-pascal-voc) app. ## Format description **Supported image formats:** `.jpg`, `.jpeg`, `.mpo`, `.bmp`, `.png`, `.webp`, `.tiff`, `.tif`, `.jfif`, `.avif`, `.heic`, and `.heif`\ **With annotations:** Yes\ **Supported annotation files extensions:** `.xml` (bounding boxes), `.png` (segmentation masks)\ **Grouped by:** Any structure (will be uploaded as a single dataset)
## Input files structure {% hint style="success" %} Example data: [download ⬇️](https://github.com/supervisely-ecosystem/import-pascal-voc/files/12600118/sample_project.zip) {% endhint %} Pascal VOC archive or directory must have the following structure: ``` 📦custom_pascal.tar 📂custom_pascal_project_dir ┗ 📂VOCdevkit ┗ 📂VOCdevkit ┗ 📂VOC or VOC2012 ┗ 📂VOC or VOC2012 ┣ 📂Annotations ┣ 📂Annotations ┣ 📂ImageSets ┣ 📂ImageSets ┃ ┣ 📂Mainn ┃ ┣ 📂Main ┃ ┗ 📂Segmentation ┃ ┗ 📂Segmentation ┣ 📂JPEGImage ┣ 📂JPEGImages ┣ 📂SegmentationClasss ┣ 📂SegmentationClass ┣ 📂SegmentationObject ┣ 📂SegmentationObject ┗ 📜colors.txt ┗ 📜colors.txt ``` **`colors.txt`** file is custom, and not provided in the original Pascal VOC Dataset. File contains information about instance mask colors associated with classes in Pascal VOC format. This file is required by this app, if you are uploading custom dataset. Each line of `colors.txt` file starts with `class_name` and ends with `RGB` values that represent class color. **`colors.txt`** example: ```txt neutral 224 224 192 kiwi 255 0 0 lemon 81 198 170 ``` {% hint style="info" %} Action and Layout Classification Image Sets are not supported by import application. {% endhint %} ## Useful links * [The PASCAL Visual Object Classes Homepage](http://host.robots.ox.ac.uk/pascal/VOC/) * [Pascal VOC Ground Truth Annotation](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/htmldoc/devkit_doc.html#SECTION00035000000000000000) * [\[Supervisely Ecosystem\] Import Pascal VOC](https://ecosystem.supervisely.com/apps/import-pascal-voc) ![](https://github.com/supervisely-ecosystem/import-pascal-voc/assets/57998637/147d2ad4-327e-462a-b5b5-bc0887ac3c19) # Cityscapes ## Overview This converter allows to import images with `.json` annotations in [Cityscapes](https://github.com/mcordts/cityscapesScripts) format. ⚠️ **Note:** images must have suffix `_leftImg8bit` and annotations suffix `_gtFine_polygons` and `.json` extension. Check the example of the file structure below. ![Result of the import](/files/xEL1LMfFJzkTthfY4jcl) Enterprise users have access to "Import as links" option, which supports import of this format with annotations. This option might be beneficial in many cases, as it allows data import to Supervisely platform without re-uploading, maintaining a single source and speeding up import process. To step up import speed even further you can compress all annotation files (`.json`'s) into an archive and import it together with the images. (Note: This method is format-dependent and may not apply to all formats.) ## Format description **Supported image formats:** `.jpg`, `.jpeg`, `.mpo`, `.bmp`, `.png`, `.webp`, `.tiff`, `.tif`, `.jfif`, `.avif`, `.heic`, and `.heif`\ **With annotations:** yes\ **Supported annotation file extension:** `.json`.\ **Grouped by:** any structure (uploaded to a single dataset)\\ ## Input files structure {% hint style="success" %} Example data: [download ⬇️](https://github.com/supervisely-ecosystem/import-wizard-docs/files/14908276/sample_cityscapes.zip)\\ {% endhint %} Recommended directory structure: ``` 📦project name ┣ 📂gtFine ┃ ┣ 📂test ┃ ┃ ┗ 📂ds1 ┃ ┃ ┃ ┗ 📜IMG_8144_gtFine_polygons.json ┃ ┣ 📂train ┃ ┃ ┗ 📂ds1 ┃ ┃ ┃ ┣ 📜IMG_1836_gtFine_polygons.json ┃ ┃ ┃ ┣ 📜IMG_2084_gtFine_polygons.json ┃ ┃ ┃ ┣ 📜IMG_3861_gtFine_polygons.json ┃ ┃ ┃ ┗ 📜IMG_4451_gtFine_polygons.json ┃ ┗ 📂val ┃ ┃ ┗ 📂ds1 ┃ ┃ ┃ ┗ 📜IMG_0748_gtFine_polygons.json ┣ 📂leftImg8bit ┃ ┣ 📂test ┃ ┃ ┗ 📂ds1 ┃ ┃ ┃ ┗ 🖼️IMG_8144_leftImg8bit.png ┃ ┣ 📂train ┃ ┃ ┗ 📂ds1 ┃ ┃ ┃ ┣ 🖼️IMG_1836_leftImg8bit.png ┃ ┃ ┃ ┣ 🖼️IMG_2084_leftImg8bit.png ┃ ┃ ┃ ┣ 🖼️IMG_3861_leftImg8bit.png ┃ ┃ ┃ ┗ 🖼️IMG_4451_leftImg8bit.png ┃ ┗ 📂val ┃ ┃ ┗ 📂ds1 ┃ ┃ ┃ ┗ 🖼️IMG_0748_leftImg8bit.png ┗ 📜class_to_id.json ``` ## Format Config File In order to import custom annotations for the images, you need to provide a `class_to_id.json` file. This file should contain a list with dictionaries. Each dictionary should contain information about the class with the following fields: * `name` - the name of the class. It should be unique. * `id` - the ID of the class. From 1 to N-1, where N is the number of classes. * `color` - the color of the class in RGB format. If not specified, the color will be generated randomly
📜class_to_id.json ```json [ { "name": "kiwi", "id": 1, "color": [255, 0, 0] }, { "name": "lemon", "id": 2, "color": [81, 198, 170] } ] ```
## Single-Image Annotation JSON Annotation file should contain the following fields: * `imgHeight` - the height of the image * `imgWidth` - the width of the image * `objects` - a list of dictionaries, each containing information about the object * `label` - the name of the class * `polygon` - a list of points that form the polygon of the object Example of the annotation file from provided sample data:
📜IMG_1836_gtFine_polygons.json ```json { "imgHeight": 800, "imgWidth": 1067, "objects": [ { "label": "lemon", "polygon": [ [772, 421], [771, 422], ... [785, 422], [784, 421] ] }, { "label": "kiwi", "polygon": [ [637, 122], [636, 123], ... [645, 123], [644, 122] ] }, { "label": "kiwi", "polygon": [ [543, 539], [542, 540], ... [548, 540], [547, 539] ] } ] } ```
## Useful links * [Cityscapes format](https://github.com/mcordts/cityscapesScripts) * [\[Supervisely Ecosystem\] Import Cityscapes](https://ecosystem.supervisely.com/apps/import-cityscapes) # Images with PNG masks ## Overview Allows to upload images with annotations in the format of PNG masks. Masks are 3-(1-)channel images containing only pixels that have the same values in all channels, to map pixel masks with the appropriate class app requires `obj_class_to_machine_color.json` file to match classes and colors, otherwise app won't start. The converter supports both semantic and instance segmentation masks. All data will be uploaded to a single dataset. ## Format description **Supported image formats:** `.jpg`, `.jpeg`, `.mpo`, `.bmp`, `.png`, `.webp`, `.tiff`, `.tif`, `.jfif`, `.avif`, `.heic`, and `.heif`\ **With annotations:** yes\ **Supported annotation format:** `.png`.\\ ## Input files structure {% hint style="success" %} Example data: [download ⬇️](https://github.com/user-attachments/files/17330052/masks_sample.zip) {% endhint %} Images should be in the folder `"img"` and masks should be in one (or more) of the following folders below: * `obj_class_to_machine_color.json` - contains class to color mapping. * `masks_machine` - contains semantic segmentation masks. Masks for semantic segmentation should have the same name as the original images (but may have a different extension e.g original image name: `cats_1.jpg` -> mask name `cats_1.png`). * `masks_instance` - contains for instance segmentation masks. Masks for instance segmentation must be placed in the subdirectories that have the same name as the original images (but without extension e.g original image name: `cats_1.jpg` -> subdirectory name `cats_1`). **Example of `obj_class_to_machine_color.json`** ```json { "dog": [50, 50, 50], "cat": [100, 100, 100] } ``` **Input data structure example:** ``` 📦Drag & Drop ┣ 📜obj_class_to_machine_color.json ┣ 📂img ┃ ┣ 🖼️image_1.jpg ┃ ┗ 🖼️image_2.jpg ┣ 📂masks_instances ┃ ┣ 📂image_1 ┃ ┃ ┣ 🖼️dog_1.png # <- `dog` class instance mask ┃ ┃ ┗ 🖼️dog_2.png ┃ ┗ 📂image_2 ┃ ┣ 🖼️dog_1.png ┃ ┗ 🖼️dog_2.png ┗ 📂masks_machine ┣ 🖼️image_1.png # <- class name for each pixel > 0 must be in the obj_class_to_machine_color.json ┗ 🖼️image_2.png ``` **`obj_class_to_machine_color.json`** example: ```json { "Lemon": 170, "Kiwi": 85 } ``` **Semantic (machine) masks example** In this configuration example, all pixels in the mask with value **equal to 170** will be combined in one Bitmap figure and will be assigned to the class **"Lemon"** and **equal to 85** will be assigned to the class **"Kiwi"**. ![](https://i.imgur.com/a5cVpAB.png) **Instance masks example** For example we have an image with 2 cats on it placed in `img/**cats_1.jpg**` directory, and we have instance masks for them placed in `mask_instances/**cats_1**/cat_1.png` and `mask_instances/**cats_1**/cat_2.png`. Subdirectories in `mask_instances` folder define to which original image these masks belong to. Masks names inside these subdirectories define a names of the class. As a result, we will have an image `cats_1.jpg` with 2 labels `cat` and `cat`.
**⚠️ Notice**: If you just want to import semantic segmentation masks, just drag & drop original images, semantic segmentation masks and `obj_class_to_machine_color.json` file. Same for instance segmentation masks, you don't have to create all directories if this is unnessecary. # Links from CSV, TXT and TSV ## Overview You can import Images into Supervisely project using a `.csv`, `.tsv` or `.txt` file. This import converter is designed to help you quickly upload images to Supervisely from a file containing image paths from **Team Files** or URLs from cloud storage or any accessible internet link (✨ **Available only in Enterprise Edition**). Additionally, you can assign tags to each image by providing a tag column in the input file. This feature is optional, and you can choose to import images without any tags. ## Format description **Supported file formats:** `.csv`, `.tsv`, and `.txt`.\ **With annotations:** yes (optional)\ **Supported annotation types:** tags\ **Grouped by:** Any structure (will be uploaded as a single dataset)\\ ### Key Features * Import Images from **Team Files** * Import Images by **URLs** from cloud storage or any accessible internet link (✨ **Available only in Enterprise Edition**) * Supported file formats: `.csv`, `.tsv` or `.txt` * Automatically **assign Tags** to each Image (*optional*) ### How to Use All images will be uploaded to a single dataset, so you don't have to worry about the full project structure in Supervisely format. All you need is to prepare a file with URLs or paths and drop this file in quick import. ### Input files structure {% hint style="success" %} Example data: [download ⬇️](https://github.com/supervisely-ecosystem/import-wizard-docs/files/14934860/sample_csv.zip) {% endhint %} In your input file, the first column is crucial as it contains either the paths or URLs to the images you want to import. This column is mandatory for the importer to function correctly. The second column, which contains the tags, is optional. If provided, these tags will be automatically assigned to the corresponding images upon import. If this column is omitted, the images will be imported without any tags. Yes, tags are optional for each image. If you choose not to assign tags to certain images, simply leave the tag column blank for those images in your input file. The importer will still process these images and upload them without any tags. #### Delimiters for File Formats In the context of this importer, we use specific delimiters for different purposes in the `.csv` file: **Column Delimiter** * `.csv` - A semicolon (`;`) is used to separate different columns. Each column represents a different attribute, such as the image path/url or the image tag. * `.tsv` - A tab () is used as a delimiter to separate different columns. Similar to the `.csv` format, each column represents a different attribute. * `.txt` - Multiple spaces ( ), tabs (), or a semicolon (`;`) can be used to separate different columns in a `.txt` file. Each column represents a different attribute, similar to the `.csv` and `.tsv` formats. Please note, ⚠️ a single space ( ) cannot be used as a delimiter. **Tag Delimiter** A comma (`,`) is used to separate different tags assigned to the same image for all file formats. This allows you to assign multiple tags to a single image. #### Examples Regardless of the file format you choose (`.csv`, `.tsv`, or `.txt`), you can specify either a path or a URL for each image, but not both in the same file. This means that each file should contain either paths to local images or URLs to internet images, but not a mix of both. 1. **Team Files**: Create a `.csv` file with columns for the relative path to the image and the image tag. ```csv path;tag /dogs/img_01.jpeg;dog /cats/img_02.jpeg;cat /horses/img_01.jpeg;horse ``` 2. **URLs**: {% hint style="info" %} ✨ Importing images by URLs is available only in [the Enterprise Edition](https://supervisely.com/enterprise/). {% endhint %} * Create a `.txt` file with columns for the full URL-link to the image and the image tag. In this example, tab () delimiters are used. ``` url tag https://images.io/image_example_1.png tag1,tag2 https://images.io/image_example_2.png tag3 https://images.io/image_example_3.png ``` * Cloud storage link example: link structure: `:///` ```csv url;tag s3://remote-img-test/08. images YOLO masks, bboxes (mix)/ds1_IMG_0748.jpeg;1 azure://supervisely-test/TEST-NEW-IMPORT/01. images SLY (from export)/ds1/img/IMG_1836.jpeg google://sly-dev-test/test_img_new/berries-02.jpeg;3 ``` ## Useful links * [\[Supervisely Ecosystem\] Import Images from CSV](https://ecosystem.supervisely.com/apps/import-images-from-csv) ![](https://imgur.com/Cqe7fjv.png) # PDF files to images ## Overview This converter allows to import `.PDF` files as images in `.PNG` format. Each page of the `.PDF` file will be converted to a separate image. The images will have a suffix added to their names to indicate the page number. ## Format description **Supported image formats:** `.pdf`\ **With annotations:** No\ **Supported annotation file extension:** Not applicable\ **Grouped by:** Any structure (will be uploaded as a single dataset)
![PDF import results](https://github.com/supervisely-ecosystem/import-wizard-docs/assets/48913536/488fec72-f2fe-4078-a4b3-3105a06e1b8a) ## Input files structure {% hint style="success" %} Example data: [download ⬇️](https://github.com/supervisely-ecosystem/import-wizard-docs/files/14905329/Sample_PDF.zip)
{% endhint %} Recommended directory structure: ``` 📦project name ┣ 📜Demo_1.pdf ┣ 📜Demo_2.pdf ┣ 📜Demo_3.pdf ┣ 📜Demo_4.pdf ┗ 📜Demo_5.pdf ``` ## Useful links * [\[Supervisely Ecosystem\] Import PDF as Images](https://ecosystem.supervisely.com/apps/import-pdf-as-images) # Multiview images ## Overview Multiview mode is a feature that allows you to view and annotate multiple images simultaneously. It is especially useful when you need to label objects from different perspectives, 3D reconstruction images, Autonomous vehicle camera views or depth estimation task images. Labeling in multiview mode can significantly increase the speed of the labeling process (for example, you don't need to switch between images and select a desired class to label the same object) Just organize images into groups and drop them to the import. The app will do the rest: it will detect groups, tag images, and activate grouping and multiview modes in the project settings. {% hint style="info" %} Note: To use the multiview import feature, you need to create a project with the `Multiview image annotation` setting enabled. You can also enable this setting in the project settings after the import. Here is an illustration of how to upload multiview images: {% endhint %} ![Import Multiview images](https://github.com/supervisely-ecosystem/import-wizard-docs/assets/79905215/81e7c8d1-dc38-4baf-bcef-165521a33c2a) Enterprise users have access to "Import as links" option, which supports import of this format with annotations. This option might be beneficial in many cases, as it allows data import to Supervisely platform without re-uploading, maintaining a single source and speeding up import process. To step up import speed even further you can compress all annotation files (`.json`'s) into an archive and import it together with the images. (Note: This method is format-dependent and may not apply to all formats.) ## Format description **Supported image formats:** `.jpg`, `.jpeg`, `.mpo`, `.bmp`, `.png`, `.webp`, `.tiff`, `.tif`, `.jfif`, `.avif`, `.heic`, and `.heif`\ **With annotations:** Yes\ **Annotation types:** Tags in Supervisely format\ **Grouped by:** Folders (corresponding tags will be assigned to images)\\ ### Key Features * 🏷️ NEW: Upload multiview project with grouped labels * All images in groups in the created project will be tagged * `Images Grouping` option will be turned on by default in the created project * Images will be grouped by tag's value * Tag value is defined by the group directory name * Works with `.nrrd` image format (2D only) ### Supported directory structures * **Archive** `zip`, `tar`, `tar.xz`, `tar.gz` ``` 📦 my_project.zip ┗ 📂 cars catalog ┗ 📂 used cars ┣ 📂 105 ┃ ┣ 🏞️ car_105_front.jpg ┃ ┗ 🏞️ car_105_top.jpg ┣ 📂 202 ┃ ┣ 🏞️ car_202_front.jpg ┃ ┗ 🏞️ car_202_top.jpg ┣ 📂 357 ┃ ┣ 🏞️ car_357_front.jpg ┃ ┗ 🏞️ car_357_top.jpg ┣ 🏞️ car_401_front.jpg ┣ 🏞️ car_401_top.jpg ┗ 🏞️ car_401_side.jpg ``` * **Folder** ``` 📂 cars catalog ┗ 📂 used cars ┣ 📂 car_id_105 ┃ ┣ 🏞️ car_105_front.jpg ┃ ┗ 🏞️ car_105_top.jpg ┣ 📂 car_id_202 ┃ ┣ 🏞️ car_202_front.jpg ┃ ┗ 🏞️ car_202_top.jpg ┣ 📂 car_id_357 ┃ ┣ 🏞️ car_357_front.jpg ┃ ┗ 🏞️ car_357_top.jpg ┣ 🏞️ car_401_front.jpg ┣ 🏞️ car_401_top.jpg ┗ 🏞️ car_401_side.jpg ``` **Structure explained:** * An archive must contain only 1 project directory. * Inside the project directory must be 1 dataset directory. * Group directories must be populated with images and placed inside the dataset directory. All images inside the group will be tagged with folder name value. * All images in the root dataset directory will be uploaded as regular images and will not be tagged. * 🏷️ **NEW: Supervisely Project Folder or Archive with label groups** This type of structure will work only if you have the required data in the files: * All images must be tagged with a group tag of the same value. * All necessary labels must be tagged with a label group tag of the same value. * The project settings in `meta.json` must contain a `multiView` section with the correct data. **Recommended structure** ``` 📦archive ┗📂project folder ┣ 📂dataset_name_01 ┃ ┣ 📂ann ┃ ┃ ┣ 📄106_1.jpeg.json ┃ ┃ ┣ 📄106_2.jpeg.json ┃ ┃ ┣ 📄106_3.jpeg.json ┃ ┃ ┣ 📄107_1.jpeg.json ┃ ┃ ┗ ... ┃ ┣ 📂img ┃ ┃ ┣ 🏞️106_1.jpg ┃ ┃ ┣ 🏞️106_2.jpg ┃ ┃ ┣ 🏞️106_3.jpg ┃ ┃ ┣ 🏞️107_1.jpg ┃ ┃ ┗ ... ┃ ┗ 📂meta (optional) ┃ ┣ 📄106_1.jpeg.json ┃ ┣ 📄106_2.jpeg.json ┃ ┣ 📄106_3.jpeg.json ┃ ┣ 📄107_1.jpeg.json ┃ ┗ ... ┗ 📄meta.json ``` **Annotation explained:** Below is the annotation for each image, for example `106_1.jpeg.json`. Only the lines of interest are shown; other lines are omitted, but the structure is preserved. ```json { "tags": [ { "name": "multiview", "value": "106" } ], "objects": [ { "classTitle": "Head Light", "tags": [ { "name": "@label-group-id", "value": "head-light" } ] } ] } ``` For an image group, an image must be tagged with the `multiview` tag (in our example) and assigned a value that represents the group, such as `106`. For a label group, an object (label) must be tagged with the `@label-group-id` tag and assigned a value that represents the group. For example `head-light` **Meta explained** Required setting for the project to import as multiview. Also shown only lines of interest. ```json { "projectSettings": { "multiView": { "enabled": true, "tagName": "multiview" } } } ``` **Image Labeling Tool Interface** * 1 Multiview group * 2 Labeling group ![Labeling Tool Interface](https://github.com/user-attachments/assets/5283e0b7-eb22-48ce-ae3b-e991191857da) {% hint style="success" %} We prepared sample datasets for you to try the import process: * images: [download ⬇️](https://github.com/supervisely-ecosystem/import-images-groups/releases/download/v0.0.1/cars.catalog.zip) * NRRD: [download ⬇️](https://github.com/supervisely-ecosystem/import-images-groups/releases/download/v0.0.1/research.zip) {% endhint %} * To display single images switch off `Images Grouping` setting. ![Switch off multiview mode](/files/h7kA7LikfEgnRzoLvRXq) * If you want to disable images grouping for the whole project, go to `Project` → `Settings` → `Visuals` and uncheck ![Disable multiview in project settings](/files/wWcAtOG9XHUNOo8yyg08) * Windowing tool is available when working with `.nrrd` files. It helps to filter pixels to see bones, air, liquids etc. ![Nrrd windowing tool](/files/gYkKFH3J5KILFgkEWPHd) * Images view synchronization | Synchronization OFF ![](/files/330W2TfsJbNR6fRuvzNo) | Synchronization ON ![](/files/raZM4qD7vCJFfNNhIw6o) | | ---------------------------------------------------- | --------------------------------------------------- | ## Useful links * [\[Supervisely Ecosystem\] Group Images for Multiview Labeling](https://ecosystem.supervisely.com/apps/group-images-for-multiview-labeling) ![](https://github.com/supervisely-ecosystem/group-images-for-multiview-labeling/assets/57998637/823cf901-8d8c-4a64-b884-c59f5ff83e93) * [\[Supervisely Ecosystem\] Import Multiview Image Groups](https://ecosystem.supervisely.com/apps/import-images-groups) ![](https://github.com/user-attachments/assets/1788313e-8ef3-4f42-9277-38e32aa9dfa6) ## Easy integration for Python developers Automate processes with multiview images using Supervisely Python SDK. ```bash pip install supervisely ``` You can learn more about it in our [Developer Portal](https://developer.supervisely.com/getting-started/python-sdk-tutorials/images/multiview-images), but here we'll just give you a quick examples of how you can get started with multiview images. ### Upload multiview images The following code snippets demonstrate how you can upload your multiview images with just a few lines of code. ```python # enable multiview display in project settings api.project.set_multiview_settings(project_id) images_paths = ['path/to/audi_01.png', 'path/to/audi_02.png'] # upload group of images api.image.upload_multiview_images(dataset_id, "audi", images_paths) ``` In the example above we uploaded two groups of multiview images. Before or after uploading images, we also need to enable image grouping in the project settings.\\ ### Group existing images for multiview {% hint style="info" %} Available starting from version `v6.73.236` of the Supervisely Python SDK. {% endhint %} If you already have images in your project and you want to group them for multiview, you can group them by your own logic. By default, images will be grouped by the value of the `multiview` tag. You can change the tag name by passing the `multiview_tag_name` argument. Here is an example of how you can do it with just a few lines of code. ```python images_1 = [2389126, 2389127, 2389128, 2389129, 2389130] group_name_1 = 'audi' multiview_tag_name = 'cars' # optional api.image.group_images_for_multiview(images_1, group_name_1, multiview_tag_name) ``` {% hint style="success" %} * If the tag does not exist, it will be created automatically. * If multiview mode is not enabled in the project settings, it will be enabled automatically. {% endhint %} Let's consider a more complex example. For instance, you have a project with several datasets containing images and you want to group them by dataset name. Here is an example of how you can do it. ```python GROUP_SIZE = 6 # number of images in one group project_id = 111111 meta_json = api.project.get_meta(project_id, with_settings=True) meta = sly.ProjectMeta.from_json(meta_json) datasets = api.dataset.get_list(project_id, recursive=True) with sly.ApiContext(api, project_id=project_id, project_meta=meta): for dataset in datasets: images = api.image.get_list(dataset.id, force_metadata_for_links=False) image_ids = [image_info.id for image_info in images] for idx, ids in enumerate(sly.batched(image_ids, batch_size=GROUP_SIZE)): group_name = f"{dataset.name}_{idx}" api.image.group_images_for_multiview(ids, group_name) ``` {% hint style="info" %} We recommend grouping images in batches of 6-12 images (depending on the size of your display). {% endhint %} ### How to upload label groups {% hint style="info" %} Available starting from version `v6.73.293` of the Supervisely Python SDK. {% endhint %} There are many cases when you need to group labels together. For example, if you have some labels captured from different perspectives that represent one object on different images and you want to analyze the object as a whole and not as separate instances, you can join them into a single group. **Label group** - is a simple group of objects, that displays the relationship between objects and helps you to quickly locate the object on different images and to avoid labeling the same object multiple times. ![label group example](https://github.com/user-attachments/assets/2552ce5c-76b3-41fd-be12-80d1dd6e834d) Using the `api.annotation.append_labels_group` method, you can upload labels as a group to images. Let's group it all together and upload local images and labels to Supervisely using this method. Our sample data directory structure: ``` 📂 data ┣ 📂 images ┃ ┣ 🏞️ car_01.jpeg ┃ ┣ 🏞️ car_02.jpeg ┃ ┗ 🏞️ car_03.jpeg ┗ 📂 masks ┣ 🏞️ car_01.png ┣ 🏞️ car_02.png ┗ 🏞️ car_03.png ``` ![data sample](https://github.com/user-attachments/assets/746eff69-e3c3-43f8-8094-8b5839dee61f) ⬇️ You can download this sample here: [data.zip](https://github.com/supervisely/developer-portal/releases/download/untagged-6cc0d25bd610d680cc0d/data.zip) Follow the code below to upload images and labels to Supervisely. ```python project_id = 56 dataset_id = 196 # GET PROJECT META meta = sly.ProjectMeta.from_json(api.project.get_meta(project_id, with_settings=True)) # GET OBJ CLASS FROM META BY NAME obj_cls = meta.get_obj_class("car") # OR CREATE NEW OBJ CLASS IF NOT EXISTS # obj_cls = sly.ObjClass(name="car", geometry_type=sly.Rectangle, color=[255, 0, 0]) # UPDATE PROJECT META IF CREATING NEW OBJ CLASS # meta = meta.add_obj_classes([obj_cls]) # api.project.update_meta(project_id, meta.to_json()) # SET MULTIVIEW SETTINGS api.project.set_multiview_settings(project_id) # GET IMAGES AND MASKS PATHS image_dir = os.path.join("data", "images") mask_dir = os.path.join("data", "masks") # SORT PATHS FOR CORRECT LABELS ORDER image_paths = sorted([os.path.join(image_dir, path) for path in os.listdir(image_dir)]) mask_paths = sorted([os.path.join(mask_dir, path) for path in os.listdir(mask_dir)]) # CREATE LABELS labels = [] for image_path, mask_path in zip(image_paths, mask_paths): # READ MASK bitmap = sly.Bitmap.from_path(mask_path) # CREATE LABEL label = sly.Label(geometry=bitmap, obj_class=obj_cls) labels.append(label) # UPLOAD IMAGES image_infos = api.image.upload_multiview_images(dataset_id, "white_car", image_paths) images_ids = [image_info.id for image_info in image_infos] # APPEND LABELS TO IMAGES api.annotation.append_labels_group( dataset_id=dataset_id, image_ids=images_ids, labels=labels, project_meta=meta, ) ``` **Result:** ![result](https://github.com/user-attachments/assets/6a89c945-529a-4125-98c3-6d0582ce05dd) # Multispectral images ## Multispectral images ### Overview This converter allows to import of multispectral images as channels or as separate images without annotations.\ Images will be grouped by directories, files from the "split" directory will be split into separate images by channels and the files from the "images" directory will be uploaded as they are.\\ {% hint style="info" %} Note: To use the multispectral import feature, you need to create a project with the `Multispectral image annotation` setting enabled. You can also enable this setting in the project settings after the import. Here is a illustration of how to upload multispectral images: {% endhint %} ![Import Multispectral images](https://github.com/supervisely-ecosystem/import-wizard-docs/assets/79905215/5571a96b-9c2f-42cd-abed-904acec3d625) Result of the import: ![Result of the import](/files/SdNoH1YRjtUf9EeCjTNj) ### Format description **Supported image formats:** `.jpg`, `.jpeg`, `.mpo`, `.bmp`, `.png`, `.webp`, `.tiff`, `.tif`, `.jfif`, `.avif`, `.heic`, and `.heif`\ **With annotations:** No\ **Supported annotation file extension:** Not applicable\ **Grouped by:** Folders (corresponding tags will be assigned to images)\\ ### Input files structure {% hint style="success" %} Example data: [download ⬇️](https://github.com/supervisely-ecosystem/import-multispectral-images/files/13487269/demo_data.zip)\\ {% endhint %} Recommended directory structure: ``` 📦project_name ┣ 📂group_name_1 ┃ ┗ 📂split ┃ ┗ 🏞️demo1.png ┣ 📂group_name_2 ┃ ┣ 📂images ┃ ┃ ┣ 🏞️demo4-rgb.png ┃ ┃ ┗ 🏞️demo4-thermal.png ┃ ┗ 📂split ┃ ┗ 🏞️demo4-thermal copy.png ┗ 📂group_name_3 ┗ 📂images ┣ 🏞️demo8-mri1.png ┣ 🏞️demo8-mri2.png ┗ 🏞️demo8-rgb.png ``` In this example, we have 3 groups with images. In the first group, we have one image, which should be split. In the second group, we have one image, which should be split and two images, which should be uploaded as is. In the third group, we have three images, which should be uploaded as is.\\ ## Enable "Windowing" tool In some cases, you may need to filter out certain pixels to better visualize specific elements like bones, air, or liquids. The Windowing tool allows you to adjust the visible range and filter pixels by their values, making it easier to see important details. ![Nrrd windowing tool](https://i.imgur.com/gW37Tyn.png) The Windowing tool is available when working with `.nrrd` files. If you have 16-bit `.tiff` files or `.nrrd` files with pixel values greater than 8 bits, they will be automatically converted to `.nrrd` format during import, enabling the Windowing tool. ### Useful links * [\[Supervisely Developer Portal\] Multispectral Images](https://developer.supervisely.com/getting-started/python-sdk-tutorials/images/multispectral-images) * [\[Supervisely Blog\] How to Annotate Multispectral Images for Computer Vision Models](https://supervisely.com/blog/labeling-multispectral-images/) * [\[Supervisely Ecosystem\] Import Multispectral Images](https://ecosystem.supervisely.com/apps/import-multispectral-images) ### Easy integration for Python developers Automate processes with multiview images using Supervisely Python SDK. ```bash pip install supervisely ``` You can learn more about it in our [Developer Portal](https://developer.supervisely.com/getting-started/python-sdk-tutorials/images/multispectral-images), but here we'll just show how you can upload your multispectral images with just a few lines of code. ```python # Setting multispectral settings for the project. api.project.set_multispectral_settings(project.id) # Preparing images for upload. image_name = "demo7.png" images = ["demo_data/demo7-rgb.png", "demo_data/demo7-thermal.png"] # Reading thermal image and extracting its channels as 2d numpy arrays. image = cv2.imread(images[1]) channels = [image[:, :, i] for i in range(image.shape[2])] # Uploading images. image_infos = api.image.upload_multispectral(dataset.id, image_name, channels, images) ``` In the example above we uploaded two images as they are and also split a thermal image into separate channels\\ # Medical 2D images ## Overview Medical 2D Converter allows to import 2D files with `.nrrd`, `.dcm`, `.nii` and `.nii.gz` extensions. Files with extensions different from `.nrrd` will be converted to `.nrrd` While this converter primarily supports 2D medical images, it is possible to import 3D files. However, please note that 3D files will be transposed and sliced along the axial plane. This process converts the 3D image into a series of 2D slices, which can then be viewed and analyzed individually. ## Format description **Supported image formats:** `.nrrd`, `.dcm`, `.nii` and `.nii.gz`\ **With annotations:** No\ **Grouped by:** Any structure (will be uploaded as a single dataset)
## Input files structure {% hint style="info" %} Example data: [download ⬇️](https://github.com/supervisely-ecosystem/import-wizard-docs/files/14934438/sample_medical2d.zip)
{% endhint %} Recommended directory structure: ``` 📦project name ┣ 📜Image_1.dcm ┣ 📜Image_2.dcm ┣ 📜Image_3.dcm ┣ 📜Image_4.dcm ┣ 📜Image_5.DCM ┣ 📜Image_6.nrrd ┗ 📜Image_7.nii ``` ### 2D Medical Image Formats Explained * **DCM** `DCM` file is an image following Digital Imaging and Communications in Medicine (DICOM) format. Format is used to store various medical images like CT scans, MRIs, PET, ultrasound, etc. Uses `.dcm` and `.DICOM` extensions If your DICOM data contains one of the following metadata fields, it will be used as `group` tag in the project: * `StudyInstanceUID` * `StudyID` * `SeriesInstanceUID` * `TreatmentSessionUID` * `Manufacturer` * `ManufacturerModelName` * `Modality` Moreover, all other DICOM metadata will be saved as image metadata and can be viewed in the Labeling interface. * **NRRD** `NRRD` file is a medical imaging format. It is used to store 2D and 3D images along with metadata. It is commonly used in medical imaging. Uses `.nrrd` extension. * **NII** `NII` format is commonly used to store magnetic resonance imaging (MRI) data. Uses `.nii` and `.nii.gz` extensions. ![Medical data import results](/files/B6nAw7rp4mo4g3PR7494) ## Useful links * [\[Supervisely Ecosystem\] Import DICOM studies](https://ecosystem.supervisely.com/apps/import-dicom-studies) * [Nearly Raw Raster Data (NRRD): Format, Examples etc.](https://teem.sourceforge.net/nrrd/) * [Overview of The Content of The DICOM Standard](https://dicom.nema.org/medical/dicom/current/output/html/part01.html#chapter_6) * [Neuroimaging Informatics Technology Initiative (NIfTI)](https://nifti.nimh.nih.gov/) # LabelMe ## Overview This converter allows to import images with `.json` annotations in [LabelMe](https://github.com/labelmeai/labelme?tab=readme-ov-file) format. Supported LabelMe format geometry types: `polygon`, `rectangle`, `circle`, `point`, `linestring`, `mask`, `line`. ![Result of the import](/files/lrpYZBvWuTvx5ZbXRkaM) ## Format description **Supported image formats:** `.jpg`, `.jpeg`, `.mpo`, `.bmp`, `.png`, `.webp`, `.tiff`, `.tif`, `.jfif`, `.avif`, `.heic`, and `.heif`\ **With annotations:** yes\ **Supported annotation file extension:** `.json`.\ **Grouped by:** Any structure (will be uploaded as a single dataset)\\ ## Input files structure {% hint style="success" %} Example data: [download ⬇️](https://github.com/user-attachments/files/16179633/labelme_demo.zip) {% endhint %} Example directory structure: ``` 📦input_folder ┣ 📂ann ┃ ┣ 📄IMG_0748.json ┃ ┗ 📄IMG_8144.json ┗ 📂img ┣ 🏞️IMG_0748.jpeg ┗ 🏞️IMG_8144.jpeg ``` ## Single-Image Annotation JSON An annotation file should contain the following fields: * `shapes` - a list of dictionaries, each containing information about the object * `label` - the name of the class * `points` - a list of points of the object * `mask` - a base64 encoded mask of the object (for `mask` shape type) * `shape_type` - the type of the object (one of the following: `polygon`, `rectangle`, `circle`, `point`, `linestring`, `mask`, `line`) * `imageHeight` - the height of the image * `imageWidth` - the width of the image Example of the annotation file:
📄IMG_0748.json ```json { "version": "5.5.0", "flags": {}, "shapes": [ { "label": "cat_polygon", "points": [ [1038.0000000000002, 91.00000000000023], [2363.0, 1311.0000000000002], [2373.0, 3236.0] ], "group_id": null, "description": "", "shape_type": "polygon", "flags": {}, "mask": null }, { "label": "cat_rectangle", "points": [ [1033.0000000000002, 76.00000000000023], [2368.0, 1311.0000000000002] ], "group_id": null, "description": "", "shape_type": "rectangle", "flags": {}, "mask": null }, { "label": "cat_circle", "points": [ [1123.0000000000002, 361.0000000000002], [1123.0000000000002, 631.0000000000002] ], "group_id": null, "description": "", "shape_type": "circle", "flags": {}, "mask": null }, { "label": "cat_line", "points": [ [1043.0000000000002, 106.00000000000023], [1153.0000000000002, 3251.0] ], "group_id": null, "description": "", "shape_type": "line", "flags": {}, "mask": null }, { "label": "cat_point", "points": [[1038.0000000000002, 101.00000000000023]], "group_id": null, "description": "", "shape_type": "point", "flags": {}, "mask": null }, { "label": "cat_polyline", "points": [ [1053.0000000000002, 96.00000000000023], [2373.0, 1291.0000000000002], [1148.0000000000002, 2171.0], [2393.0, 3246.0], [2393.0, 3246.0], [2393.0, 3246.0] ], "group_id": null, "description": "", "shape_type": "linestrip", "flags": {}, "mask": null }, { "label": "cat_ai_mask", "points": [ [946.0, 847.0], [1665.0, 1346.0] ], "group_id": null, "description": "", "shape_type": "mask", "flags": {}, "mask": "iVBORw0KGgoAAAANSU ... ElFTkSuQmCC" } ], "imagePath": "IMG_5853 2.jpg", "imageData": "/9j/4AAQSkZJRgAB ...dg+9FFFIo//Z", "imageHeight": 3382, "imageWidth": 2536 } ```
## Useful links * [LabelMe page](https://github.com/labelmeai/labelme?tab=readme-ov-file) # LabelStudio ## Overview This converter allows to import images with `.json` annotations in [LabelStudio](https://labelstud.io/guide/export#Label-Studio-JSON-format-of-annotated-tasks) format. Supported LabelStuidio format geometry types: `polygonlabels` (`polygon`), `rectanglelabels` (`rectangle`), `brushlabels` (RLE masks), and `choices` (`tags`) ![Result of the import](/files/mq09PWZgrRq9RHd6dWRE) ## Format description **Supported image formats:** `.jpg`, `.jpeg`, `.mpo`, `.bmp`, `.png`, `.webp`, `.tiff`, `.tif`, `.jfif`, `.avif`, `.heic`, and `.heif`\ **With annotations:** yes\ **Supported annotation file extension:** `.json`.\ **Grouped by:** Any structure (will be uploaded as a single dataset)\\ ## Input files structure {% hint style="success" %} Example data: [download ⬇️](https://github.com/user-attachments/files/16183688/label_studio_demo.zip) {% endhint %} ⚠️ **Note:** image names should correspond to the names in the annotation files (`data` > `image` field in the JSON file). Example directory structure: ``` 📦input_folder ┣ 📂ann ┃ ┣ 📄annotation_1.json ┃ ┗ 📄annotation_4.json ┗ 📂img ┣ 🏞️IMG_0748.jpeg ┗ 🏞️IMG_8144.jpeg ``` ## Single-Image Annotation JSON An annotation file should contain the following fields: * `annotations` or `predictions` - a list of dictionaries, each containing annotation for single image * `result` - list of dictionaries, each containing information about the objects * `original_width` - the width of the original image * `original_height` - the height of the original image * `value` - a dictionary containing information about the object * `polygonlabels`/`rectanglelabels`/`brushlabels`/`choices` - field with the object class name * `points` - a list of points of the object (for `polygonlabels` and `rectanglelabels` shape types) * `rle` and `format` - a base64 encoded mask of the object (for `mask` shape type) * `type` - the type of the object (one of the following: `polygonlabels`, `rectanglelabels`, `brushlabels`, `choices`, `relation`) * `data` - a dictionary containing information about the image * `image` - the path to the image Example of the annotation file:
📄 annotation_1.json ```json [ { "id": 13, "annotations": [ { "id": 7, "completed_by": 1, "result": [ { "original_width": 1280, "original_height": 853, "image_rotation": 0, "value": { "x": 14.107390372983872, "y": 15.524193548387096, "width": 61.535093245967744, "height": 70.36290322580646, "rotation": 0, "rectanglelabels": ["Airplane"] }, "id": "eGEJdycmv3", "from_name": "label", "to_name": "image", "type": "rectanglelabels", "origin": "manual" } ], "was_cancelled": false, "ground_truth": false, "created_at": "2024-07-05T13:18:24.130642Z", "updated_at": "2024-07-05T13:18:24.130665Z", "draft_created_at": null, "lead_time": 7.935, "prediction": {}, "result_count": 0, "unique_id": "f527f9c8-affe-469b-991a-70ec6fd79e54", "import_id": null, "last_action": null, "task": 13, "project": 8, "updated_by": 1, "parent_prediction": null, "parent_annotation": null, "last_created_by": null } ], "file_upload": "airplane.jpg", "drafts": [], "predictions": [], "data": { "image": "/data/upload/8/airplane.jpg" }, "meta": {}, "created_at": "2024-07-05T13:18:14.329289Z", "updated_at": "2024-07-05T13:18:24.152845Z", "inner_id": 1, "total_annotations": 1, "cancelled_annotations": 0, "total_predictions": 0, "comment_count": 0, "unresolved_comment_count": 0, "last_comment_updated_at": null, "project": 8, "updated_by": 1, "comment_authors": [] } ] ```
## Useful links * [LabelStudio GitHub page](https://github.com/HumanSignal/label-studio?tab=readme-ov-file#try-out-label-studio) * [LabelStudio website](https://labelstud.io/) # Fisheye ## Overview Annotate fisheye images with ease using the fisheye labeling interface in Supervisely. {% embed url="" %} Fisheye labeling interface {% endembed %} To use the fisheye labeling interface, you need to: 1. create a project with the `Fisheye` labeling interface enabled. 2. prepare calibration files with parameters for fisheye images (metadata files). Check out the [Fisheye Lens Metadata](#fisheye-lens-metadata) section for more details. 3. import fisheye images with the calibration parameters and annotations (optional). {% hint style="info" %} To correctly import fisheye images, all metadata files should be placed **in the directory with the `meta` name**. {% endhint %} The `Fisheye` labeling interface in Supervisely provides the new labeling tool 2D Cuboid for annotating objects like cars, pedestrians, and other objects in fisheye images. Learn more about the 2D Cuboid annotation format in the [documentation](/customization-and-integration/00_ann_format_navi/04_supervisely_format_objects#cuboids-2d-annotation). ## Labeling Toolbox After import, fisheye images are annotated in the Image Labeling Toolbox with the **Fisheye labeling interface**. It uses the calibration metadata to project annotations onto the distorted image and includes a special `Cuboid 3D` tool — a 3D cuboid projected onto the 2D fisheye image. {% hint style="success" %} Check out the [Fisheye labeling interface](/labeling/labeling-toolbox/fisheye) page to learn how to annotate fisheye images in Supervisely. {% endhint %} ## Description **Supported image formats:** `.jpg`, `.jpeg`, `.mpo`, `.bmp`, `.png`, `.webp`, `.tiff`, `.tif`, `.jfif`, `.avif`, `.heic`, and `.heif`\ **With annotations:** supported\ **Supported projection:** cylindrical ## Data structure * **Folder** or **Archive** (`zip`, `tar`) ``` 📦 my_project.zip or 📂 my_project └── 📂 dataset_01 ├── 📂 img │ ├── 🏞️ car_105.jpg │ └── 🏞️ car_202.jpg ├── 📂 meta │ ├── 📄 car_105.jpg.json ⬅️ calibration parameters │ └── 📄 car_202.jpg.json ⬅️ calibration parameters └── 📂 ann ├── 📄 car_105.jpg.json ⬅️ annotation files (optional) └── 📄 car_202.jpg.json ⬅️ annotation files (optional) ``` The project may contain one or more dataset folders (e.g., `dataset_01`), each with its own `img`, `meta`, and `ann` folders. The `meta` folder contains metadata files for fisheye images with the calibration parameters. Each metadata file should have the same name as the corresponding image file + `.json` extension. The `ann` folder contains annotation files in the Supervisely format. Each annotation file should have the same name as the corresponding image file + `.json` extension. Annotation files are optional. ## Fisheye Lens Metadata It is essential to provide calibration data for fisheye images to allow the fisheye labeling interface to correct the distortion. The calibration data is stored in metadata files in JSON format.
Metadata example ``` { "calibration": { "extrinsic": { "quaternion": [ 0.39492483984846793, -0.5928584556321699, -0.5854007522749839, 0.3871164962798451 ], "translation": [ -3.819498356, -0.070724798, 0.730674159 ] }, "intrinsic": { "cx": 968, "cy": 776, "fx": 500, "fy": 500, "cameraModel": "cylindrical_equidist" } } } ``` The `extrinsic` block is optional — the minimal valid metadata file contains only the `intrinsic` block.
Here are the key points and fields descriptions: * `cameraModel` - the camera projection model. Currently, only `cylindrical_equidist` is supported; support for Equirectangular, Cubemap, Rectilinear (Perspective), and Stereographic projections is in active development. * `fx`, `fy` - the focal lengths of the camera in pixels. * `cx`, `cy` - the coordinates of the principal point (optical image center) in pixels, e.g., half of the image width and height for a centered principal point. * Optionally, the `extrinsic` block with the `quaternion` (rotation) and `translation` (in meters) fields can be provided. It describes the coordinate transformation from the camera coordinate system to the vehicle coordinate system. * The vehicle coordinate system, which follows the ISO 8855 convention, is anchored to the ground below the midpoint of the rear axle. The X-axis points in the driving direction, the Y-axis points to the left side of the vehicle and the Z-axis points up from the ground. * The camera sensor's coordinate system is based on OpenCV. The X axis points to the right along the horizontal sensor axis, the Y axis points downwards along the vertical sensor axis and the Z-axis points in viewing direction along the optical axis to maintain the right-handed system. # High Color Depth ## High Color Depth Images ## Overview This option allows you to upload images with high color depth to the platform without annotations. All images from the input directory and its subdirectories will be uploaded to a single dataset. All 3 dimensional images will be converted 2D grayscale images in `.nrrd` format by using `cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)` which uses a weighted sum of the color channels, giving more importance to green, followed by red, and the least importance to blue. This weighted conversion better represents human perception of brightness, as the green channel contributes the most. After uploading high color depth images, you will be able to use windowing feature to adjust the brightness and contrast of the images. ![windowing](/files/32RxH1G7tRmXVA1jH4qP) ## Format description **Supported image formats:** `.exr`, `.hdr` **With annotations:** No\ **Supported annotation format:** Not applicable.\ **Grouped by:** Any structure (will be uploaded as a single dataset).
## Input files structure Example data: [download ⬇️](https://github.com/user-attachments/files/17310984/high-color-depth-sample.zip)
Recommended directory structure: ``` 📦project name ┣ 🖼️IMG_01.exr ┣ 🖼️IMG_02.exr ┣ 🖼️IMG_03.exr ┣ 🖼️IMG_04.exr ┣ 🖼️IMG_05.exr ┣ 🖼️IMG_06.hdr ┣ 🖼️IMG_07.hdr ┣ 🖼️IMG_08.hdr ┣ 🖼️IMG_09.hdr ┗ 🖼️IMG_10.hdr ``` # Videos # Supervisely {% hint style="success" %} Easily import your videos with annotations in the Supervisely format. The Supervisely json-based annotation format supports such figures: `rectangle`, `line (polyline)`, `polygon`, `point`, `bitmap` (`mask`), `graph` (`keypoints`). It is a universal format that supports various types of annotations and is used in the Supervisely platform. {% endhint %} {% hint style="info" %} All information about the Supervisely JSON format can be found [here](https://docs.supervisely.com/data-organization/00_ann_format_navi) {% endhint %} Enterprise users have access to "Import as links" option, which supports import of this format with annotations. This option might be beneficial in many cases, as it allows data import to Supervisely platform without re-uploading, maintaining a single source and speeding up import process. To step up import speed even further you can compress all annotation files (`.json`'s) into an archive and import it together with the images. (Note: This method is format-dependent and may not apply to all formats.) ## Format description **Supported video formats:** `.avi`, `.mp4`, `.3gp`, `.flv`, `.webm`, `.wmv`, `.mov`, and `.mkv`\ **With annotations:** yes\ **Supported annotation format:** `.json`.\ **Data structure:** Information is provided below. ## Input files structure {% hint style="success" %} Example data: [download ⬇️](https://github.com/supervisely-ecosystem/import-videos-in-sly-format/files/12546490/my_videos_project.zip). {% endhint %} Both directory and archive are supported. **Recommended directory structure:** ``` 📦input_folder ┣ 📂dataset_name_01 ┃ ┣ 📂ann ┃ ┃ ┣ 📄vid_0748.mp4.json ┃ ┃ ┗ 📄vid_8144.mp4.json ┃ ┣ 📂video ┃ ┃ ┣ 🎬vid_0748.mp4 ┃ ┃ ┗ 🎬vid_8144.mp4 ┃ ┗ 📂meta (optional) ┃ ┣ 📄vid_0748.mp4.json ┃ ┗ 📄vid_8144.mp4.json ┣ 📄meta.json ┗ 📄key_id_map.json file (optional) ``` **Struggled with the structure?** No worries! All videos will be uploaded to a single dataset, so you don't have to worry about the full project structure in Supervisely format. All you need is to prepare videos with annotations and `meta.json` file (recommended). Items even can be placed in any subdirectories or the root directory. Just make sure that the annotation file names match the video file names (e.g. annotaion file `video_1.jpg.json` is for the video `video_1.jpg`) and that the annotation file format is correct (we will provide an example in the next section). The application will do the rest. Project meta file `meta.json` is recommended to be present in the project directory. It contains classes and tags definitions for the project. If it is not present, it will try to create it from the annotations. Learn more about the `meta.json` file [here](https://docs.supervisely.com/customization-and-integration/00_ann_format_navi/02_project_classes_and_tags). ## Single-Video Annotation JSON For each video, we store the annotations in a separate `json` file named `video_name.video_format.json` with the following file structure: ![](/files/J9SW5c7xmXN6wjenXHe6) ```json { "size": { "height": 1080, "width": 1920 }, "description": "", "key": "c8168b43ae1b45c38930f456df9d0f2b", "tags": [], "objects": [ { "key": "198f727d40c749eebcacc4aed299b39a", "classTitle": "rect", "tags": [], "labelerLogin": "alexxx", "updatedAt": "2020-08-23T12:06:11.963Z", "createdAt": "2020-08-23T12:06:11.963Z" } ], "frames": [ { "index": 0, "figures": [ { "key": "65f21690780e43b49863c3cbd07eab3a", "objectKey": "198f727d40c749eebcacc4aed299b39a", "geometryType": "rectangle", "geometry": { "points": { "exterior": [ [266, 420], [847, 845] ], "interior": [] } }, "labelerLogin": "alexxx", "updatedAt": "2020-08-23T12:06:13.544Z", "createdAt": "2020-08-23T12:06:13.544Z" } ] } ], "framesCount": 375 } ``` **Fields definitions:** * `size` - string - is equal to image(frame) size * `description` - string - (optional) - this field is used to store the text we want to assign to the video. In the labeling intrface it corresponds to the 'data' filed. * `tags` - **list** of strings that will be interpreted as video [tags](https://docs.supervisely.com/customization-and-integration/00_ann_format_navi/03_supervisely_format_tags) * `key` - string, unique key for a given video (used in key\_id\_map.json to get the video ID) * `objects` - **list** of [objects on the video](https://docs.supervisely.com/customization-and-integration/00_ann_format_navi/04_supervisely_format_objects) * `frames` - **list** of frames of which the video consists. List contains only frames with an object from the 'objects' field * `index` - integer - number of the current frame * `figures` - integer - list of objects which the current frame contains * `framesCount` - integer - total number of frames in the video * `objectKey` - string - unique key for a given object (used in key\_id\_map.json) * `labelerLogin` - string - the name of a user who created the current figure * `geometryType` - "cuboid\_3d" - class shape * `geometry` - a dictionary containing indicators of location, rotation and dimensions of cuboids **Fields definitions for objects field:** * `key` - string, a unique key for the given object (used in key\_id\_map.json to get the object ID) * `classTitle` - string - the title of a class. It's used to identify the class shape from the `meta.json` file * `tags` - list of strings that will be interpreted as object tags * `labelerLogin` - string - the name of the user that added this figure to the project **Fields description for figures field:** * `key` - string, a unique key for the given figure (used in key\_id\_map.json to get the figure ID) * `objectKey` - string, a unique key for the given object (used in key\_id\_map.json to get the object ID). * `geometryType` - "rectangle" -class shape * `geometry` - geometry of the object * `classTitle` - string - the title of a class. It's used to identify the class shape from the `meta.json` file * `labelerLogin` - string - the name of the user that added this figure to the current frame ### Key id map file Key\_id\_map.json file is optional. It is created when annotating the video inside Supervisely interface and sets the correspondence between the unique identifiers of the video, object and the frame on which the object is located. If you annotate manually, you do not need to create this file. This will not affect the work being done. Json format of key\_id\_map.json: ```json { "tags": {}, "objects": { "198f727d40c749eebcacc4aed299b39a": 20520 }, "figures": { "65f21690780e43b49863c3cbd07eab3a": 503130811 }, "videos": { "c8168b43ae1b45c38930f456df9d0f2b": 157876296 } } ``` Fields definitions: * `objects` - dictionary, where the key is a unique string, generated inside Supervisely environment to set correspondence of current object in annotation, and values are unique integer ID corresponding to the current object * `figures` - dictionary, where the key is a unique string, generated inside Supervisely environment to set correspondence of object on current frame in annotation, and values are unique integer ID corresponding to the current frame * `videos` - dictionary, where the key is unique string, generated inside Supervisely environment to set correspondence of video in annotation, and value is a unique integer ID corresponding to the current video * `tags` - dictionary, where the keys are unique strings, generated inside Supervisely environment to set correspondence of tag on current frame in annotation, and values are a unique integer ID corresponding to the current tag ## Useful links * [Supervisely Annotation Format](https://docs.supervisely.com/customization-and-integration/00_ann_format_navi) * [Supervisely Video Annotation](https://docs.supervisely.com/customization-and-integration/00_ann_format_navi/06_supervisely_format_videos) * [\[SDK CLI\] Upload projects in Supervisely format](https://developer.supervisely.com/getting-started/command-line-interface/sdk-cli#upload-a-project) * [\[CLI Tool Beta\] Upload projects in Supervisely format](https://developer.supervisely.com/getting-started/command-line-interface/cli-tool/workflow-automation#upload-projects-in-supervisely-format) * [\[Supervisely Ecosystem\] Import video in Supervisely format](https://ecosystem.supervisely.com/apps/import-videos-in-sly-format) # Multiview ## Overview Import format for multiview video projects in Supervisely. Videos are grouped by datasets - all videos within one dataset form a synchronized multiview group. ![](/files/x5JDy6CwYXgwwRyGdvRQ) ## Format description **Supported video formats:** `.mp4`, `.avi`, `.mov`, `.webm`, `.wmv`, `.3gp`, `.flv`, `.mkv`, `.mpeg`, `.mpg`\ **With annotations:** Yes\ **Grouped by:** Datasets (each dataset = one multiview group)
### When to Use Multiview Multiview mode is particularly useful when you have multiple videos of the same scene captured from different angles or cameras, featuring a specific object of interest. **Unified object across all videos:** If you have a common object appearing in different videos, you can annotate it as a single Supervisely object (with a shared ID). When you export and re-import the project, this object will be recreated as a unified entity - a cross-video object spanning all videos in the dataset (multiview set). **Tags are video-specific:** Unlike objects, tags apply only within a specific video. When you tag a figure, frame, or video on a particular video, that tag will be associated only with that video and displayed only on it. ### Key Features * Videos are grouped by datasets * Synchronized playback of multiple video streams * Unified object annotations across all videos in the multiview group * Video-specific tags * Optional annotations in Supervisely video format * Optional metadata files for keeping video frame offset information ### How to Use **Prepare structure:** Example data: [download ⬇️](https://github.com/user-attachments/files/23659139/multiview-videos-sample.zip)
* **Archive** `zip`, `tar`, `tar.xz`, `tar.gz` ``` 📦 archive.zip ┗ 📂 project_name ┣ 📂 dataset_01 ┃ ┣ 📂 video ┃ ┃ ┣ 🎥 camera_front.mp4 ┃ ┃ ┣ 🎥 camera_left.mp4 ┃ ┃ ┗ 🎥 camera_right.mp4 ┃ ┣ 📂 ann (optional) ┃ ┃ ┣ 📄 camera_front.mp4.json ┃ ┃ ┣ 📄 camera_left.mp4.json ┃ ┃ ┗ 📄 camera_right.mp4.json ┃ ┗ 📂 metadata (optional) ┃ ┣ 📄 camera_front.mp4.meta.json ┃ ┣ 📄 camera_left.mp4.meta.json ┃ ┗ 📄 camera_right.mp4.meta.json ┣ 📂 dataset_02 ┃ ┣ 📂 video ┃ ┃ ┗ ... ┃ ┣ 📂 ann (optional) ┃ ┃ ┗ ... ┃ ┗ 📂 metadata (optional) ┃ ┗ ... ┗ 📄 meta.json (optional) ``` * **Folder** ``` 📂 project_name ┣ 📂 dataset_01 ┃ ┣ 📂 video ┃ ┃ ┣ 🎥 video_001.mp4 ┃ ┃ ┣ 🎥 video_002.mp4 ┃ ┃ ┗ 🎥 video_003.mp4 ┃ ┣ 📂 ann (optional) ┃ ┃ ┣ 📄 video_001.mp4.json ┃ ┃ ┣ 📄 video_002.mp4.json ┃ ┃ ┗ 📄 video_003.mp4.json ┃ ┗ 📂 metadata (optional) ┃ ┣ 📄 video_001.mp4.meta.json ┃ ┣ 📄 video_002.mp4.meta.json ┃ ┗ 📄 video_003.mp4.meta.json ┗ 📄 meta.json ``` **Structure explained:** * Inside project directory can be one or multiple dataset directories * **Each dataset = one multiview group:** All videos within the same dataset will be displayed together * Each dataset directory can contain: * `video/` - directory with video files * `ann/` - (optional) directory with annotations in Supervisely format * `metadata/` - (optional) directory with video metadata files * Annotation file names pattern: `{video_name}.{video_ext}.json` * Metadata file names pattern: `{video_name}.{video_ext}.meta.json` **Meta explained** Required setting for the project to import as multiview. Also shown only lines of interest. ```json { "projectSettings": { "multiView": { "enabled": true, "tagName": null, "tagId": null, "isSynced": false }, "labelingInterface": "multi_view" } } ``` **Annotation explained** This format uses the standard Supervisely video annotation format. Check [the documentation](https://docs.supervisely.com/customization-and-integration/00_ann_format_navi/06_supervisely_format_videos) for more details. ```json { "objects": [ { "key": "object_key_1", "classTitle": "Car", "tags": [] } ] } ``` **Metadata explained** Optional JSON file with custom video information: ```json { "offsetType": "frame", "offsetValue": 5, "videoStreamIndex": 0 } ``` `offsetType` - type of offset, can be `frame` or `time (ms)`\ `offsetValue` - offset value in frames or milliseconds\ `videoStreamIndex` - index of the video stream in the multiview group (starting from 0)
Can contain offset value, video stream index and offset type. ## Useful links * [\[Supervisely Documentation\] Video Annotation Format](https://docs.supervisely.com/data-organization/00_ann_format_navi/04_supervisely_format_videos) * [\[Supervisely Ecosystem\] Export Videos in Supervisely Format](https://ecosystem.supervisely.com/apps/export-videos-in-supervisely-format) # Pointclouds # Supervisely ## Overview {% hint style="success" %} Easily import your pointclouds with annotations in the Supervisely format. The Supervisely json-based annotation format supports `cuboid_3d` shape figures. It is a universal format that supports various types of annotations and is used in the Supervisely platform. {% endhint %} {% hint style="info" %} All information about the Supervisely JSON format can be found [here](https://docs.supervisely.com/data-organization/00_ann_format_navi) {% endhint %} ## Format description **Supported point cloud format:** `.pcd`\ **With annotations:** yes\ **Supported annotation format:** `.json`.\ **Data structure:** Information is provided below. ## Input files structure {% hint style="success" %} Example data: [download ⬇️](https://github.com/supervisely-ecosystem/demo-pointcloud-project-annotated/releases/download/v0.0.1/demo_pointcloud_project_annotated.zip). {% endhint %} Both directory and archive are supported. **Recommended directory structure:** ``` 📦pointcloud_project (folder or .tar/.zip archive) ├──📄key_id_map.json (optional) ├──📄meta.json ├──📂dataset1 │ ├──📂pointcloud │ │ ├──📄scene_1.pcd │ │ ├──📄scene_2.pcd │ │ └──📄... │ ├──📂related_images │ │ ├──📂scene_1_pcd │ │ │ ├──🏞️scene_1_cam0.png │ │ │ ├──📄scene_1_cam0.png.json │ │ │ ├──🏞️scene_1_cam1.png │ │ │ ├──📄scene_1_cam1.png.json │ │ │ └──📄... │ │ ├──📂scene_2_pcd │ │ │ ├──🏞️scene_2_cam0.png │ │ │ ├──📄scene_2_cam0.png.json │ │ │ ├──🏞️scene_2_cam1.png │ │ │ ├──📄scene_2_cam1.png.json │ │ │ └──📄... │ │ └──📂... │ └──📂ann │ ├──📄scene_1.pcd.json │ ├──📄scene_2.pcd.json │ └──📄... └──📂dataset... ``` Every `.pcd` file in a sequence has to be stored inside a `pointcloud` folder of datasets. | Key | Value | | --- | --------------------------------------------------------- | | x | The x coordinate of the point. | | y | The y coordinate of the point. | | z | The z coordinate of the point. | | r | The red color channel component. An 8-bit value (0-255). | | g | The green color channel component. An 8-bit value (0-255) | | b | The blue color channel component. An 8-bit value (0-255) | All the positional coordinates (x, y, z) are in meters. Supervisely supports all PCD encoding: ASCII, binary, binary\_compressed. The PCD file format description can be found [here](https://pointclouds.org/documentation/tutorials/pcd_file_format.html) `related_images` optional folder, contains photo-context data: - Frame folder, each named according to pointcloud (`/related_images/scene_1_pcd/`), which contains: - image files (`.png \ .jpg`) - photo context image annotation file (`.json`) - json files, named according to image name (`1.png -> 1.png.json`). Read more in the "Photo context image annotation file" section below. ## Format of Annotations Point cloud Annotations refer to each point cloud and contains information about labels on the point clouds in the datasets. A dataset has a list of `objects` that can be shared between some point clouds. The list of `objects` is defined for the entire dataset, even if the object's figure occurs in only one point cloud. `Figures` represents individual labels, attached to one single frame and its object. ```json { "description": "", "key": "e9f0a3ae21be41d08eec166d454562be", "tags": [], "objects": [ { "key": "ecb975d70735486b90fe4fdd2be77e3b", "classTitle": "Car", "tags": [], "labelerLogin": "admin", "updatedAt": "2022-05-04T00:32:30.872Z", "createdAt": "2022-05-04T00:32:30.872Z" } ], "figures": [ { "key": "abbaec8785c1468585f6210c62bb2374", "objectKey": "ecb975d70735486b90fe4fdd2be77e3b", "geometryType": "cuboid_3d", "geometry": { "position": { "x": 58.756710052490234, "y": 4.623323917388916, "z": -0.4174150526523591 }, "rotation": { "x": 0, "y": 0, "z": -1.77 }, "dimensions": { "x": 1.59, "y": 4.28, "z": 1.45 } }, "labelerLogin": "admin", "updatedAt": "2022-05-04T00:32:37.432Z", "createdAt": "2022-05-04T00:32:37.432Z" } ] } ``` **Optional fields and loading** These fields are optional and are not needed when loading the project. The server can automatically fill in these fields while project is loading. * `id` - unique identifier of the current object * `classId` - unique class identifier of the current object * `labelerLogin` - string - the name of user who created the current figure * `createdAt` - string - date and time of figure creation * `updatedAt` - string - date and time of the last figure update Main idea of `key` fields and `id` you can see below in "Key id map" file section. **Fields definitions:** * `description` - string - (optional) - this field is used to store the text to assign to the sequence. * `key` - string, unique key for a given sequence (used in key\_id\_map.json to get the sequence ID) * `tags` - list of strings that will be interpreted as point cloud tags * `objects` - list of objects that may be present on the dataset * `geometryType` - "cuboid\_3d" or other 3D geometry - class shape **Fields definitions for `objects` field:** * `key` - string - unique key for a given object (used in key\_id\_map.json) * `classTitle` - string - the title of a class. It's used to identify the class shape from the `meta.json` file * `tags` - list of strings that will be interpreted as object tags (can be empty) **Fields description for `figures` field:** * `key` - string - unique key for a given figure (used in key\_id\_map.json) * `objectKey` - string - unique key to link figure to object (used in key\_id\_map.json) * `geometryType` - "cuboid\_3d" or other 3D geometry -class shape * `geometry` - geometry of the object **Description for `geometry` field (cuboid\_3d):** * `position` 3D vector of box center coordinates: * **x** - forward in the direction of the object * **y** - left * **z** - up * `dimensions` is a 3D vector that scales a cuboid from its local center along x, y, z: * **x** - width * **y** - length * **z** - height * `rotation` is a 3D Vector that rotates a cuboid along an axis in world space: * **x** - pitch * **y** - roll * **z** - yaw (direction) Rotation values bound inside \[**-pi** ; **pi**] When `yaw = 0` box direction will be strict `+y` Read more about the `key_id_map.json` file and photo context annotations in the documentation [here](https://developer.supervisely.com/getting-started/supervisely-annotation-format/point-clouds#key-id-map-file). ### Photo context image annotation file ```json { "name": "host-a005_cam4_1231201437716091006.jpeg" "entityId": 2359620, "meta": { "deviceId": "CAM_BACK_LEFT", "timestamp": "2019-01-11T03:23:57.802Z", "sensorsData": { "extrinsicMatrix": [ -0.8448329028461443, -0.5350302199120708, 0.00017334762588639086, -0.012363736761232369, -0.0035124448582330757, 0.005222293412494302, -0.9999801949951969, -0.16621728572112304, 0.5350187183638307, -0.8448167798004226, -0.006291229448121315, -0.3527897896721229 ], "intrinsicMatrix": [ 882.42699274, 0, 602.047851885, 0, 882.42699274, 527.99972239, 0, 0, 1 ] } } } ``` **Fields description:** * name - string - Name of image file * Id - (OPTIONAL) - integer >= 1 ID of the photo in the system. It is not required when upload and is filled in automatically when the project is loaded. * entityId (OPTIONAL) - integer >= 1 ID of the Point Cloud in the system, that photo attached to. Doesn't required while uploading. * deviceId - string- Device ID or name. * timestamp - string - Time when the frame occurred in ISO 8601 format * sensorsData - Sensors data such as Pinhole camera model parameters. See wiki: [Pinhole camera model](https://en.wikipedia.org/wiki/Pinhole_camera_model) and [OpenCV docs for 3D reconstruction](https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html). * intrinsicMatrix - Array of number - 3x3 flatten matrix (dropped last zeros column) of intrinsic parameters in row-major order, also called camera matrix. It's used to denote camera calibration parameters. See [Intrinsic parameters](https://en.wikipedia.org/wiki/Camera_resectioning#Intrinsic_parameters). * extrinsicMatrix - Array of number - 4x3 flatten matrix (dropped last zeros column) of extrinsic parameters in row-major order, also called joint rotation-translation matrix. It's used to denote the coordinate system transformations from 3D world coordinates to 3D camera coordinates. See [Extrinsic\_parameters](https://en.wikipedia.org/wiki/Camera_resectioning#Extrinsic_parameters). ## Useful links * [Supervisely Annotation Format](https://docs.supervisely.com/customization-and-integration/00_ann_format_navi) * [Supervisely Pointcloud Annotation](https://developer.supervisely.com/getting-started/supervisely-annotation-format/point-clouds) * [\[CLI Tool Beta\] Upload projects in Supervisely format](https://developer.supervisely.com/getting-started/command-line-interface/cli-tool/workflow-automation#upload-projects-in-supervisely-format) * [\[SDK CLI\] Upload projects in Supervisely format](https://developer.supervisely.com/getting-started/command-line-interface/sdk-cli#upload-a-project) * [Import Point Cloud Project](https://ecosystem.supervisely.com/apps/import-pointcloud-project) app. * [Export pointclouds project in Supervisely format](https://ecosystem.supervisely.com/apps/export-pointclouds-project-in-supervisely-format) app. * [Demo pointcloud project with labels](https://ecosystem.supervisely.com/projects/demo-pointcloud-project-annotated) # .PCD, .PLY, .LAS, .LAZ pointclouds ## Overview This option allows you to upload point clouds to the platform without any annotations. All items from the input directory and its subdirectories will be uploaded to a single dataset. Supported formats are `.pcd`, `.ply`, `.las`, and `.laz`, and they are imported natively without conversion. If you need to preserve the directory structure, you can use the [Import Pointclouds PCD](https://ecosystem.supervisely.com/apps/import-pointcloud-pcd) application from the Supervisely Ecosystem. ## Format description **Supported point cloud formats:** `.pcd`, `.ply`, `.las`, `.laz`.\ **With annotations:** No\ **Supported annotation format:** Not applicable.\ **Grouped by:** Any structure (will be uploaded to a single dataset).
## Why native LAS/LAZ/PLY import matters Import Wizard loads `.las`, `.laz`, and `.ply` files natively. `PCD` is a simpler point cloud format. `LAS` / `LAZ` and many `PLY` variants can store richer source data, so native import helps preserve more of the information already present in your files. Practical advantages: * **More source attributes preserved**: `LAS` / `LAZ` can contain intensity, return information, classification, GPS time, and other LiDAR-specific fields. `PLY` can contain RGB, normals, and custom per-point properties. With native import, these attributes do not need to be flattened into a simpler intermediate representation. * **No data loss from PCD conversion**: when point clouds are converted to `PCD`, some source-specific fields may be dropped, remapped, or normalized depending on the conversion path. Native import avoids that lossy step. * **Smaller files can speed up the workflow**: some source formats, especially compressed `LAZ`, can be much smaller than converted `PCD`. That reduces upload volume, download volume in the labeling tool, and time spent opening dense scenes. * **Faster time to first annotation**: skipping conversion removes an entire preprocessing stage before import. * **Immediate benefit from rendering improvements**: once the data is loaded, the WebGPU pipeline can use preserved attributes such as RGB and Intensity while keeping interaction smooth on dense scenes. ## Input files structure {% hint style="success" %} Example data: [download ⬇️](https://github.com/supervisely-ecosystem/import-wizard-docs/files/15025187/sample_pcd.zip) {% endhint %} Recommended directory structure: ``` 📦 folder ├── 📄 item_01.pcd ├── 📄 item_02.ply ├── 📄 item_03.las ├── 📄 item_04.laz ├── 📄 item_05.pcd ├── 📄 item_06.ply ├── 📄 item_07.las ├── 📄 item_08.laz ├── 📄 item_09.pcd └── 📄 item_10.ply ``` ## Format LAS/LAZ In Import Wizard, LAS and LAZ files are supported natively, so no conversion to PCD is required. ### Coordinate handling In Import Wizard, Supervisely preserves source coordinates as provided in the input file. Coordinate shift is applicable only when using the [Import LAS format](https://ecosystem.supervisely.com/apps/import-las-format) app, where LAS/LAZ files are converted to PCD. This app is optional and is not required for Auto Import Wizard. Example log entry: ``` Applied coordinate shift for filename: X=1234567.89, Y=9876543.21, Z=123.45 ``` ## Format PLY In Import Wizard, PLY files are supported natively, so no conversion is required. If you use the [Import Pointcloud PLY](http://ecosystem.supervisely.com/apps/import-pointcloud-ply) app, files are converted from PLY to PCD during import. This app is optional and is not required for Auto Import Wizard. ## Useful links * [\[Supervisely Ecosystem\] Import Pointclouds PCD](https://ecosystem.supervisely.com/apps/import-pointcloud-pcd) * [\[Supervisely Ecosystem\] Import LAS format](https://ecosystem.supervisely.com/apps/import-las-format) * [\[Supervisely Ecosystem\] Import Pointcloud PLY](http://ecosystem.supervisely.com/apps/import-pointcloud-ply) # Lyft ## Overview {% hint style="success" %} Easily import your pointclouds with annotations in the LYFT format. LYFT is an annotation format used in the well-regarded `Lyft Level 5 Prediction` dataset. {% endhint %} Originaly, the dataset is suited to be imported as Pointcloud Episodes, even though it is available in Pointcloud format as well. ## Format description **Supported point cloud format:** `.bin`\ **With annotations:** yes\ **Supported annotation format:** `.json`\ **Data structure:** Information is provided below. ## Input files structure {% hint style="success" %} Example data: [download ⬇️](https://github.com/supervisely-ecosystem/import-wizard-docs/releases/download/v0.0.2/lyft-sample.zip). {% endhint %} Both directory and archive are supported. **Format directory structure:** ``` 📦pointcloud_project (folder or .tar/.zip archive) ├──📂dataset1 │ ├──📂data │ │ ├──📄attribute.json │ │ ├──📄calibrated_sensor.json │ │ ├──📄category.json │ │ ├──📄ego_pose.json │ │ ├──📄instance.json │ │ ├──📄log.json │ │ ├──📄map.json │ │ ├──📄sample.json │ │ ├──📄sample_annotation.json │ │ ├──📄sample_data.json │ │ ├──📄scene.json │ │ ├──📄sensor.json │ │ └──📄visibility.json │ ├──📂images │ │ ├──🏞️host-a101_cam0_0000000000000000001.jpeg │ │ ├──🏞️host-a101_cam0_0000000000000000002.jpeg │ │ ├──🏞️host-a101_cam0_0000000000000000003.jpeg │ │ ├──🏞️host-a101_cam0_0000000000000000004.jpeg │ │ ├──🏞️host-a101_cam0_0000000000000000005.jpeg │ │ └──🏞️... │ ├──📂lidar │ │ ├──📄host-a101_lidar1_1241893239302414726.bin │ │ ├──📄host-a101_lidar1_1241893239302414727.bin │ │ ├──📄host-a101_lidar1_1241893239302414728.bin │ │ ├──📄host-a101_lidar1_1241893239302414729.bin │ │ ├──📄host-a101_lidar1_1241893239302414730.bin │ │ └──📄... │ └──📂maps └── └──🏞️map_raster_palo_alto.png ``` Every `.bin` file in a sequence has to be stored inside a `lidar` folder of dataset. | Key | Value | | --- | ------------------------------- | | x | The x coordinate of the point. | | y | The y coordinate of the point. | | z | The z coordinate of the point. | | i | Intensity of the return signal. | | r | Ring index. | The LYFT format description can be found [here](https://mmdetection3d.readthedocs.io/en/stable/advanced_guides/datasets/lyft.html) ## LYFT Annotation format The original LYFT dataset's annotations are token-based. The dataset is composed of dozens of scenes, each consisting of 126 point cloud frames, named samples. Samples are linked to annotation data through the use of unique identifiers, known as tokens. These tokens ensure that each point cloud frame is accurately associated with its corresponding annotations, maintaining the integrity and structure of the dataset. #### `attribute.json` The `attribute.json` file contains metadata about the attributes of objects in the dataset. Each entry includes a unique token, a name, and a description of the attribute. Content example: ```json { "description": "", "token": "f5081f1e5aa941f9d9f727ad186c8db67b916336f975a2f5d65d14ea01ed098f", "name": "object_action_lane_change_right" }, { "token": "1b388c1f5e5149ae173ad3d674e7ad7f1847e213173d14ce5ecf431ad697ca17", "description": "", "name": "object_action_running" }, { "description": "", "token": "17d61007ee69782e0ad8ffa5f8cd4c075f18b4b09e11f0e966bc27026c7929ea", "name": "object_action_lane_change_left" } ``` #### `calibrated_sensor.json` The `calibrated_sensor.json` file contains calibration data for each sensor used in the dataset. This includes information about the sensor's position, orientation, and intrinsic parameters. Each entry in the file is associated with a unique sensor token. Content example: ```json { "sensor_token": "172a55e2b50f18a6b6d545369a457003c2f3b438d0180b2b4c7819ca29b3f6ab", "rotation": [ -0.4944743277529699, 0.5037402715904608, 0.5076908453442883, -0.49395433344067297 ], "camera_intrinsic": [ [ 1112.8384901, 0, 958.488205774 ], [ 0, 1112.8384901, 539.540735426 ], [ 0, 0, 1 ] ], "translation": [ 0.8201327486252449, -0.002437920474569259, 1.6531257722973938 ], "token": "59155106c0ac5abe83cb6558ad8ce98400e3c3abf51234734bc89bc9d613470a" } ``` #### `category.json` The `category.json` file contains metadata about the categories of objects in the dataset. Each entry includes a unique token, a name, and a description of the category. Contents example: ```json { "description": "", "token": "8eccddb83fa7f8f992b2500f2ad658f65c9095588f3bc0ae338d97aff2dbcb9c", "name": "car" }, { "description": "", "token": "73e8de69959eb9f5b4cd2859e74bec4b5491417336cad63f27e8edb8530ffbf8", "name": "pedestrian" }, { "description": "", "token": "f81f51e1897311b55c0c6247c3db825466733e08df687c0ea830b026316a1c12", "name": "animal" } ``` #### `ego_pose.json` The `ego_pose.json` file contains the position and orientation of the ego vehicle at each timestamp. Each entry includes a unique token, rotation, translation, and timestamp. Entry example: ```json { "rotation": [ -0.6004078747001649, -0.0008682874404776075, 0.0018651459228554213, 0.7996912850004297 ], "translation": [ 1007.2332778546739, 1725.4217301399474, -24.580000733806127 ], "token": "0c257254dad346c9d90f7970ce2c0b8142f7c6e6a90716f4c0538cd2d2ef77d5", "timestamp": 1557858039302414.8 } ``` #### `instance.json` The `instance.json` file contains metadata about each instance in the dataset. Each entry includes a unique token, category token, and tokens linking to the first and last annotations of the instance. The file also includes the number of annotations associated with each instance. Content example: ```json { "last_annotation_token": "36a6954dcf400dcca8a25353939f106c09d43fcd40a3ba18add54bd149e8afb5", "category_token": "8eccddb83fa7f8f992b2500f2ad658f65c9095588f3bc0ae338d97aff2dbcb9c", "token": "695097711d9ec763b55f24d04ae9eb51289575645968f00723cee666a9f68c27", "first_annotation_token": "2a03c42173cde85f5829995c5851cc81158351e276db493b96946882059a5875", "nbr_annotations": 92 } ``` #### `log.json` The `log.json` file contains metadata about the logging information for each scene in the dataset. Each entry includes a unique token, the date the data was captured, the location, the vehicle used, and a token linking to the corresponding map. ```json { "date_captured": "2019-05-14", "location": "Palo Alto", "token": "da4ed9e02f64c544f4f1f10c6738216dcb0e6b0d50952e158e5589854af9f100", "vehicle": "a101", "logfile": "", "map_token": "53992ee3023e5494b90c316c183be829" } ``` #### `map.json` The `map.json` file contains metadata about the maps used in the dataset. Each entry represents a single map and includes a unique token, the filename of the map image, the category of the map, and a list of tokens linking to the logs associated with the map itself. Content example: ```json { "log_tokens": [ "da4ed9e02f64c544f4f1f10c6738216dcb0e6b0d50952e158e5589854af9f100", "0a6839d6ee6804113bb5591ed99cc70ad883d0cff396e3aec5e76e718771b30e", "a939e6edc494777d058c3b1eafb91a7236f6b4ff5e98c9abb1216046a0b3a45f", "25721546712bf8ba8729d88992ba2a72ed04918ffda3081cbefe3e1a6c02173f", "71dfb15d2f88bf2aab2c5d4800c0d10a76c279b9fda98720781a406cbacc583b", "5fbada2dc96f8ae4485af83535d63c24a3116262ae94edd69c4185166d4c5e0e", "47ce1d46dc84a54949a350ff4e0cc855fb0cd8ab02246951a10a95ed6b0f863e", "b28b767dac46db1659065a51d6453129eeb82bf397fdbb779de95414090d6842", "d23bc414dd4c48a55e24d80d86cc54f393ea20649f381af567b53644b761b40a" ], "token": "53992ee3023e5494b90c316c183be829", "filename": "maps/map_raster_palo_alto.png", "category": "semantic_prior" } ``` #### `sample.json` The `sample.json` file contains metadata about each sample in the dataset. Each sample represents a single point cloud frame and includes tokens linking to related data such as annotations and sensor data. The file also includes information about the sample's position in the sequence and the associated scene. Single entry example: ```json { "next": "", "prev": "", "token": "24b0962e44420e6322de3f25d9e4e5cc3c7a348ec00bfa69db21517e4ca92cc8", "timestamp": 1557858039302414.8, "scene_token": "da4ed9e02f64c544f4f1f10c6738216dcb0e6b0d50952e158e5589854af9f100", "data": { "CAM_BACK": "542a9e44f2e26221a6aa767c2a9b90a9f692c3aee2edb7145256b61e666633a4", "CAM_FRONT_ZOOMED": "9c9bc711d93d728666f5d7499703624249919dd1b290a477fcfa39f41b26259e", "LIDAR_FRONT_RIGHT": "8cfae06bc3d5d7f9be081f66157909ff18c9f332cc173d962460239990c7a4ff", "CAM_FRONT": "fb40b3b5b9d289cd0e763bec34e327d3317a7b416f787feac0d387363b4d00f0", "CAM_FRONT_LEFT": "f47a5d143bcebb24efc269b1a40ecb09440003df2c381a69e67cd2a726b27a0c", "CAM_FRONT_RIGHT": "5dc54375a9e14e8398a538ff97fbbee7543b6f5df082c60fc4477c919ba83a40", "CAM_BACK_RIGHT": "ae8754c733560aa2506166cfaf559aeba670407631badadb065a9ffe7c337a7d", "CAM_BACK_LEFT": "01c0eecd4b56668e949143e02a117b5683025766d186920099d1e918c23c8b4b", "LIDAR_TOP": "ec9950f7b5d4ae85ae48d07786e09cebbf4ee771d054353f1e24a95700b4c4af", "LIDAR_FRONT_LEFT": "5c3d79e1cf8c8182b2ceefa33af96cbebfc71f92e18bf64eb8d4e0bf162e01d4" }, "anns": [ "2a03c42173cde85f5829995c5851cc81158351e276db493b96946882059a5875", "c3c663ed5e7b6456ab27f09175743a551b0b31676dae71fbeef3420dfc6c7b09", "4193e4bf217c8a0ff598d792bdd9d049b496677d5172e38c1ed22394f20274fb", "f543b455c7066d7054ed5218670b4432a3fc7a0f57cadb0cb59aa96e8dd1a135", "6ef405e1f47a4a234f0bfeeb559d61c70031af6c47bf2e499a31002f845cd403", "db7413e79e7df6de53d801e96e26c494caf1d5b709da85aa7bf04e42c76609f5", "40eb44f0b586f2b41fb962adca2c1de351b70f3577a382798db2735564566a33", "1ab21da1d6f6d88107a5b067ba1f40929a47ae414b72e8bc17d774f281776cb2", "3d7bdcb0c99a5ba50cc74b9b9194fe4f2fdfa82a078c8afa561eec9afad052f9", "4e00f88a86a5ae6a015d0a58027a280db2aebf13d4626875c877d23cca553d8d", "64e6ea3476f64420c76db21b2fa4ec589def2c05ad97d2574ea63f8f0f7f22c9" ] } ``` #### `sample_annotation.json` The `sample_annotation.json` file contains detailed information about each annotation in the dataset. This includes the size, position, and orientation of the annotated objects, as well as tokens linking to related data such as attributes and instances. Each entry in the file represents an instance annotation. Single entry example: ```json { "token": "2a03c42173cde85f5829995c5851cc81158351e276db493b96946882059a5875", "num_lidar_pts": -1, "size": [ 1.997, 5.284, 1.725 ], "sample_token": "24b0962e44420e6322de3f25d9e4e5cc3c7a348ec00bfa69db21517e4ca92cc8", "rotation": [ 0.1539744509331139, 0, 0, 0.9880748293827983 ], "prev": "", "translation": [ 1048.155950230245, 1691.8102354006162, -23.304943447792454 ], "num_radar_pts": 0, "attribute_tokens": [ "1ba8c9a8bda54423fa710b0af1441d849ecca8ed7b7f9393ba1794afe4aa6aa2", "daf16a3f6499553cc5e1df4a456de5ee46e2e6b06544686d918dfb1ddb088f6f" ], "next": "9986dac1bcecb560153ab58ae7560028caeed3c1e067b37503cf50932e983afc", "instance_token": "695097711d9ec763b55f24d04ae9eb51289575645968f00723cee666a9f68c27", "visibility_token": "", "category_name": "car" } ``` #### `sample_data.json` The `sample_data.json` file contains metadata about each data sample in the dataset. This includes information about the sensor used to capture the data, the file format, and the associated tokens for calibration and ego pose. Each entry in the file represents a single data sample, linking it to the corresponding point cloud or image file. Single entry example: ```json { "width": 1920, "height": 1080, "calibrated_sensor_token": "59155106c0ac5abe83cb6558ad8ce98400e3c3abf51234734bc89bc9d613470a", "token": "542a9e44f2e26221a6aa767c2a9b90a9f692c3aee2edb7145256b61e666633a4", "sample_token": "24b0962e44420e6322de3f25d9e4e5cc3c7a348ec00bfa69db21517e4ca92cc8", "is_key_frame": true, "prev": "", "fileformat": "jpeg", "ego_pose_token": "0c257254dad346c9d90f7970ce2c0b8142f7c6e6a90716f4c0538cd2d2ef77d5", "timestamp": 1557858039200000, "next": "8d614daa8d1d48d3af4a0c817b676da1cb3e68f1432296eb52cfc428d0ff4d6d", "filename": "images/host-a101_cam3_1241893239200000006.jpeg", "sensor_modality": "camera", "channel": "CAM_BACK" } ``` #### `scene.json` The `scene.json` file contains metadata about each scene in the dataset. Each scene is a sequence of point cloud frames captured over a period of time. The file includes tokens linking to the first and last samples in the scene, the number of samples, and a description. Entry example: ```json { "log_token": "da4ed9e02f64c544f4f1f10c6738216dcb0e6b0d50952e158e5589854af9f100", "first_sample_token": "24b0962e44420e6322de3f25d9e4e5cc3c7a348ec00bfa69db21517e4ca92cc8", "name": "host-a101-lidar0-1241893239199111666-1241893264098084346", "description": "", "last_sample_token": "2346756c83f6ae8c4d1adec62b4d0d31b62116d2e1819e96e9512667d15e7cec", "nbr_samples": 126, "token": "da4ed9e02f64c544f4f1f10c6738216dcb0e6b0d50952e158e5589854af9f100" } ``` #### `sensor.json` The `sensor.json` file contains metadata about each sensor used in the dataset. This includes information about the sensor's modality (e.g., camera, lidar), the channel it corresponds to, and a unique token for each sensor. Example entries: ```json { "modality": "camera", "channel": "CAM_BACK", "token": "172a55e2b50f18a6b6d545369a457003c2f3b438d0180b2b4c7819ca29b3f6ab" }, { "modality": "camera", "channel": "CAM_FRONT_ZOOMED", "token": "286718e1fbc8c8f0ca441969d91c36a8a809666e049e54ce121636100b520946" }, { "modality": "lidar", "channel": "LIDAR_FRONT_RIGHT", "token": "953faed96fd3d2fae3ec03cd2838b312b8c1a9bb7a0629481982870cb28acb67" } ``` #### `visibility.json` The `visibility.json` file contains information about the visibility levels of objects in the dataset. Each entry specifies a visibility level, a description of what that level represents, and a unique token. Original dataset's file contents: ```json [ { "level": "v60-80", "description": "visibility of whole object is between 60 and 80%", "token": "3" }, { "level": "v0-40", "description": "visibility of whole object is between 0 and 40%", "token": "1" }, { "level": "v40-60", "description": "visibility of whole object is between 40 and 60%", "token": "2" }, { "level": "v80-100", "description": "visibility of whole object is between 80 and 100%", "token": "4" } ] ``` ## Useful links * [LYFT Dataset Documentation](https://mmdetection3d.readthedocs.io/en/stable/advanced_guides/datasets/lyft.html) * [Papers With Code's paper on LYFT](https://paperswithcode.com/dataset/lyft-level-5-prediction) * [Lyft SDK GitHub Page](https://github.com/lyft/nuscenes-devkit) * ["Lyft 3D Object Detection for Autonomous Vehicles" on Kaggle](https://www.kaggle.com/competitions/3d-object-detection-for-autonomous-vehicles) # nuScenes ## Overview {% hint style="success" %} Easily import your pointclouds with annotations in the nuScenes format. {% endhint %} The original nuScenes dataset is a comprehensive, large-scale public dataset designed for autonomous driving research, developed by Motional. It includes 1,000 diverse 20-second driving scenes from Boston and Singapore, featuring dense traffic and challenging conditions. With 1.4M camera images, 390k LIDAR sweeps, and 1.4M RADAR sweeps, it provides extensive multimodal sensor data, including annotations for 23 object classes with 3D bounding boxes at 2Hz and object-level attributes like visibility and activity. Please note that the original dataset's format is best suited for point cloud episodes modality. That said, it is available to be imported as point cloud. ## Format description **Supported point cloud format:** `.bin`\ **With annotations:** yes\ **Supported annotation format:** `.json`\ **Data structure:** Information is provided below. ## Input files structure {% hint style="success" %} Example data: [download ⬇️](https://github.com/supervisely-ecosystem/import-wizard-docs/releases/download/v0.0.2/nuscenes-mini-sample.zip). {% endhint %} Both directory and archive are supported. **Format directory structure:** ``` 📦pointcloud_project (folder or .tar/.zip archive) ├──📂dataset1 │ ├──📂maps │ │ ├──🏞️36092f0b03a857c6a3403e25b4b7aab3.png │ │ └──🏞️... │ ├──📂samples │ │ ├──📂CAM_BACK │ │ │ ├──🏞️n008-2018-08-01-15-16-36-0400__CAM_BACK__1533151603537558.jpg │ │ │ └──🏞️... │ │ ├──📂CAM_BACK_LEFT │ │ │ ├──🏞️n008-2018-08-01-15-16-36-0400__CAM_BACK_LEFT__1533151603547405.jpg │ │ │ └──🏞️... │ │ ├──📂CAM_BACK_RIGHT │ │ │ ├──🏞️n008-2018-08-01-15-16-36-0400__CAM_BACK_RIGHT__1533151605028113.jpg │ │ │ └──🏞️... │ │ ├──📂CAM_FRONT │ │ │ ├──🏞️n008-2018-08-01-15-16-36-0400__CAM_FRONT__1533151615912404.jpg │ │ │ └──🏞️... │ │ ├──📂CAM_FRONT_LEFT │ │ │ ├──🏞️n008-2018-08-01-15-16-36-0400__CAM_FRONT_LEFT__1533151614904799.jpg │ │ │ └──🏞️... │ │ ├──📂CAM_FRONT_RIGHT │ │ │ ├──🏞️n008-2018-08-01-15-16-36-0400__CAM_FRONT_RIGHT__1533151611420482.jpg │ │ │ └──🏞️... │ │ ├──📂LIDAR_TOP │ │ │ ├──📄n008-2018-08-01-15-16-36-0400__LIDAR_TOP__1533151611896734.pcd.bin │ │ │ └──📄... │ │ ├──📂RADAR_BACK_LEFT │ │ │ ├──📄n008-2018-08-01-15-16-36-0400__RADAR_BACK_LEFT__1533151610450182.pcd │ │ │ └──📄... │ │ ├──📂RADAR_BACK_RIGHT │ │ │ ├──📄n008-2018-08-01-15-16-36-0400__RADAR_BACK_RIGHT__1533151607538446.pcd │ │ │ └──📄... │ │ ├──📂RADAR_FRONT │ │ │ ├──📄n008-2018-08-01-15-16-36-0400__RADAR_FRONT__1533151610426071.pcd │ │ │ └──📄... │ │ ├──📂RADAR_FRONT_LEFT │ │ │ ├──📄n008-2018-08-01-15-16-36-0400__RADAR_FRONT_LEFT__1533151616430028.pcd │ │ │ └──📄... │ │ ├──📂RADAR_FRONT_RIGHT │ │ │ ├──📄n008-2018-08-01-15-16-36-0400__RADAR_FRONT_RIGHT__1533151609574740.pcd │ │ └─└──📄... │ ├──📂sweeps │ │ ├──📂CAM_BACK │ │ │ └──🏞️... │ │ ├──📂CAM_BACK_LEFT │ │ │ └──🏞️... │ │ ├──📂CAM_BACK_RIGHT │ │ │ └──🏞️... │ │ ├──📂CAM_FRONT │ │ │ └──🏞️... │ │ ├──📂CAM_FRONT_LEFT │ │ │ └──🏞️... │ │ ├──📂CAM_FRONT_RIGHT │ │ │ └──🏞️... │ │ ├──📂LIDAR_TOP │ │ │ └──📄... │ │ ├──📂RADAR_BACK_LEFT │ │ │ └──📄... │ │ ├──📂RADAR_BACK_RIGHT │ │ │ └──📄... │ │ ├──📂RADAR_FRONT │ │ │ └──📄... │ │ ├──📂RADAR_FRONT_LEFT │ │ │ └──📄... │ │ ├──📂RADAR_FRONT_RIGHT │ │ └─└──📄... │ ├──📂v1.0-mini │ │ ├──📄attribute.json │ │ ├──📄calibrated_sensor.json │ │ ├──📄category.json │ │ ├──📄ego_pose.json │ │ ├──📄instance.json │ │ ├──📄log.json │ │ ├──📄map.json │ │ ├──📄sample.json │ │ ├──📄sample_annotation.json │ │ ├──📄sample_data.json │ │ ├──📄scene.json │ │ ├──📄sensor.json └── └──📄visibility.json ``` Every `.bin` file in a sequence has to be stored inside a `LIDAR_TOP` folder of 'sample' and 'sweeps' folders of dataset. | Key | Value | | --- | ------------------------------- | | x | The x coordinate of the point. | | y | The y coordinate of the point. | | z | The z coordinate of the point. | | i | Intensity of the return signal. | | r | Ring index. | {% hint style="info" %} The nuScenes format description can be found [here](https://www.nuscenes.org/nuscenes) {% endhint %} ## nuScenes' Annotation format The nuScenes annotations are token-based. The dataset is composed of dozens of scenes, each containing a certain amount of samples (pointclouds). Samples are linked to annotation data through the use of unique identifiers. These tokens ensure that each point cloud frame is accurately associated with its corresponding annotations, camera data, log information, maps, etc. maintaining the integrity and structure of the dataset. #### `attribute.json` The `attribute.json` file contains metadata about the attributes of objects in the dataset. An attribute is a property of an instance that can change while the category remains the same. Example: a vehicle being parked/stopped/moving, and whether or not a bicycle has a rider. ```json { "token": -- Unique record identifier. "name": -- Attribute name. "description": -- Attribute description. } ``` #### `calibrated_sensor.json` The `calibrated_sensor.json` file contains calibration data for each sensor used in the dataset. Definition of a particular sensor (lidar/radar/camera) as calibrated on a particular vehicle. All extrinsic parameters are given with respect to the ego vehicle body frame. All camera images come undistorted and rectified. ```json { "token": -- Unique record identifier. "sensor_token": -- Foreign key pointing to the sensor type. "translation": [3] -- Coordinate system origin in meters: x, y, z. "rotation": [4] -- Coordinate system orientation as quaternion: w, x, y, z. "camera_intrinsic": [3, 3] -- Intrinsic camera calibration. Empty for sensors that are not cameras. } ``` #### `category.json` The `category.json` file contains metadata about the categories of objects in the dataset. Taxonomy of object categories (e.g. vehicle, human). Subcategories are delineated by a period (e.g. human.pedestrian.adult). ```json { "token": -- Unique record identifier. "name": -- Category name. Subcategories indicated by period. "description": -- Category description. "index": -- The index of the label used for efficiency reasons in the .bin label files of nuScenes-lidarseg. This field did not exist previously. } ``` #### `ego_pose.json` Ego vehicle pose at a particular timestamp. Given with respect to global coordinate system of the log's map. The ego\_pose is the output of a lidar map-based localization algorithm described in our paper. The localization is 2-dimensional in the x-y plane. ```json { "token": -- Unique record identifier. "translation": [3] -- Coordinate system origin in meters: x, y, z. Note that z is always 0. "rotation": [4] -- Coordinate system orientation as quaternion: w, x, y, z. "timestamp": -- Unix time stamp. } ``` #### `instance.json` The `instance.json` file contains metadata about each instance in the dataset. An object instance, e.g. particular vehicle. This table is an enumeration of all object instances we observed. Note that instances are not tracked across scenes, only inside a given scene. ```json { "token": -- Unique record identifier. "category_token": -- Foreign key pointing to the object category. "nbr_annotations": -- Number of annotations of this instance. "first_annotation_token": -- Foreign key. Points to the first annotation of this instance. "last_annotation_token": -- Foreign key. Points to the last annotation of this instance. } ``` #### `log.json` The `log.json` file contains information about the log from which the data was extracted. ```json { "token": -- Unique record identifier. "logfile": -- Log file name. "vehicle": -- Vehicle name. "date_captured": -- Date (YYYY-MM-DD). "location": -- Area where log was captured, e.g. singapore-onenorth. } ``` #### `map.json` The `map.json` file contains metadata about the map data that is stored as binary semantic masks from a top-down view. ```json { "token": -- Unique record identifier. "log_tokens": [n] -- Foreign keys. "category": -- Map category, currently only semantic_prior for drivable surface and sidewalk. "filename": -- Relative path to the file with the map mask. } ``` #### `sample.json` The `sample.json` file contains metadata about each sample in the dataset. A sample is an annotated keyframe at 2 Hz. The data is collected at (approximately) the same timestamp as part of a single LIDAR sweep. ```json { "token": -- Unique record identifier. "timestamp": -- Unix time stamp. "scene_token": -- Foreign key pointing to the scene. "next": -- Foreign key. Sample that follows this in time. Empty if end of scene. "prev": -- Foreign key. Sample that precedes this in time. Empty if start of scene. } ``` #### `sample_annotation.json` The `sample_annotation.json` file contains detailed information about each annotation in the dataset. A bounding box defining the position of an object seen in a sample. All location data is given with respect to the global coordinate system. ```json { "token": -- Unique record identifier. "sample_token": -- Foreign key. NOTE: this points to a sample NOT a sample_data since annotations are done on the sample level taking all relevant sample_data into account. "instance_token": -- Foreign key. Which object instance is this annotating. An instance can have multiple annotations over time. "attribute_tokens": [n] -- Foreign keys. List of attributes for this annotation. Attributes can change over time, so they belong here, not in the instance table. "visibility_token": -- Foreign key. Visibility may also change over time. If no visibility is annotated, the token is an empty string. "translation": [3] -- Bounding box location in meters as center_x, center_y, center_z. "size": [3] -- Bounding box size in meters as width, length, height. "rotation": [4] -- Bounding box orientation as quaternion: w, x, y, z. "num_lidar_pts": -- Number of lidar points in this box. Points are counted during the lidar sweep identified with this sample. "num_radar_pts": -- Number of radar points in this box. Points are counted during the radar sweep identified with this sample. This number is summed across all radar sensors without any invalid point filtering. "next": -- Foreign key. Sample annotation from the same object instance that follows this in time. Empty if this is the last annotation for this object. "prev": -- Foreign key. Sample annotation from the same object instance that precedes this in time. Empty if this is the first annotation for this object. } ``` #### `sample_data.json` The `sample_data.json` file contains metadata about each data sample in the dataset. A sensor data e.g. image, point cloud or radar return. For sample\_data with is\_key\_frame=True, the time-stamps should be very close to the sample it points to. For non key-frames the sample\_data points to the sample that follows closest in time. ```json { "token": -- Unique record identifier. "sample_token": -- Foreign key. Sample to which this sample_data is associated. "ego_pose_token": -- Foreign key. "calibrated_sensor_token": -- Foreign key. "filename": -- Relative path to data-blob on disk. "fileformat": -- Data file format. "width": -- If the sample data is an image, this is the image width in pixels. "height": -- If the sample data is an image, this is the image height in pixels. "timestamp": -- Unix time stamp. "is_key_frame": -- True if sample_data is part of key_frame, else False. "next": -- Foreign key. Sample data from the same sensor that follows this in time. Empty if end of scene. "prev": -- Foreign key. Sample data from the same sensor that precedes this in time. Empty if start of scene. } ``` #### `scene.json` The `scene.json` file contains metadata about each scene in the dataset. A scene is a 20s long sequence of consecutive frames extracted from a log. Multiple scenes can come from the same log. Note that object identities (instance tokens) are not preserved across scenes. ```json { "token": -- Unique record identifier. "name": -- Short string identifier. "description": -- Longer description of the scene. "log_token": -- Foreign key. Points to log from where the data was extracted. "nbr_samples": -- Number of samples in this scene. "first_sample_token": -- Foreign key. Points to the first sample in scene. "last_sample_token": -- Foreign key. Points to the last sample in scene. } ``` #### `sensor.json` The `sensor.json` file specifies sensor types. ```json { "token": -- Unique record identifier. "channel": -- Sensor channel name. "modality": {camera, lidar, radar} -- Sensor modality. Supports category(ies) in brackets. } ``` #### `visibility.json` The visibility of an instance is the fraction of annotation visible in all 6 images. Binned into 4 bins 0-40%, 40-60%, 60-80% and 80-100%. ```json { "token": -- Unique record identifier. "level": -- Visibility level. "description": -- Description of visibility level. } ``` ## Useful links * [nuScenes homepage](https://www.nuscenes.org/nuscenes) * [MMDetection3D Documentation on nuScenes dataset](https://mmdetection3d.readthedocs.io/en/v0.17.1/datasets/nuscenes_det.html) * [nuScenes devkit GitHub page](https://github.com/nutonomy/nuscenes-devkit) # KITTI 3D ## Overview {% hint style="success" %} Easily import your pointclouds with annotations in the KITTI 3D format. {% endhint %} The KITTI dataset is a widely used computer vision dataset for training and evaluating algorithms for tasks like object detection, 3D object tracking, and scene understanding. ## Format description **Supported point cloud format:** `.bin`\ **With annotations:** yes\ **Supported annotation format:** `.txt`\ **Data structure:** Information is provided below. ## Input files structure {% hint style="success" %} Example data: [download ⬇️](https://github.com/user-attachments/files/18378632/kitti3d-sample.zip). {% endhint %} Both directory and archive are supported. **Format directory structure:** ``` 📦kitti3d_project (folder or .tar/.zip archive) ├──📂calib │   ├──📄000000.txt │   ├──📄000001.txt │   ├──📄000002.txt │   └──📄... ├──📂image_2 │   ├──🏞️000000.png │   ├──🏞️000001.png │   ├──🏞️000002.png │   └──🏞️... ├──📂label_2 │   ├──📄000000.txt │   ├──📄000001.txt │   ├──📄000002.txt │   └──📄... └──📂velodyne ├──📄000000.bin ├──📄000001.bin ├──📄000002.bin └──📄... ``` **The KITTI3D sub-folders are structured as follows:** * `image_02/` - contains the left color camera images (png) * `label_02/` - contains the left color camera label files (plain text files) * `calib/` - contains the calibration for all four cameras (plain text file) * `velodyne/` - contains KITTI LIDAR point cloud binary files {% hint style="info" %} The KITTI 3D format description can be found [here](https://github.com/yanii/kitti-pcl/blob/master/KITTI_README.TXT) {% endhint %} ## KITTI 3D Annotation format Dataset annotations are stored in plain text files with the `.txt` extension. Each annotation file corresponds to a single image in the dataset and contains annotations for objects in the scene. #### `label.txt` The label file in the KITTI dataset provides annotations for objects in the scene, such as cars, pedestrians, and cyclists. This information is crucial for training and evaluating object detection and tracking algorithms. The label file is a plain text file associated with each image in the dataset. Each label file contains a set of lines, with each line representing the annotation for a single object in the corresponding image. **The format of each line is as follows:** ``` object_type truncation occlusion alpha left top right bottom height width length x y z rotation_y ``` **Example:** ``` Car -1.00 -1 1.90 434.56 225.91 592.44 319.73 1.44 1.64 3.78 -3.03 1.57 13.30 1.68 1.00 ``` **Fields:** * `object_type`: The type of the annotated object. This can be one of the following: 'Car', 'Van', 'Truck', 'Pedestrian', 'Person\_sitting', 'Cyclist', 'Tram', 'Misc', or 'DontCare'. 'DontCare' is used for objects that are present but ignored for evaluation. * `truncation`: The fraction of the object that is visible. It is a float value in the range \[0.0, 1.0]. A value of 0.0 means the object is fully visible, and 1.0 means the object is completely outside the image frame. * `occlusion`: The level of occlusion of the object. It is an integer value indicating the degree of occlusion, where 0 means fully visible, and higher values indicate increasing levels of occlusion. * `alpha`: The observation angle of the object in radians, relative to the camera. It is the angle between the object's heading direction and the positive x-axis of the camera. * `left`, `top`, `right`, `bottom`: The 2D bounding box coordinates of the object in the image. They represent the pixel locations of the top-left and bottom-right corners of the bounding box. * `height`, `width`, `length`: The 3D dimensions of the object (height, width, and length) in meters. * `x`, `y`, `z`: The 3D location of the object's centroid in the camera coordinate system (in meters). * `rotation_y`: The rotation of the object around the y-axis in camera coordinates, in radians. #### `calib.txt` The calib.txt file in the KITTI dataset contains calibration information for the sensors used in data collection, such as cameras and LiDAR. This calibration data is essential for projecting 3D points into the image plane, transforming between coordinate systems, and performing accurate object detection and localization. The file is a plain text file that contains key-value pairs, with each key representing a calibration parameter and its corresponding value being a matrix or vector. **The format of each line is as follows:** ``` P0: [3x4 matrix] P1: [3x4 matrix] P2: [3x4 matrix] P3: [3x4 matrix] R0_rect: [3x3 matrix] Tr_velo_to_cam: [3x4 matrix] Tr_imu_to_velo: [3x4 matrix] ``` **Example:** ``` P0: 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 P1: 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 P2: 721.5377197265625 0.0 609.559326171875 0.0 0.0 721.5377197265625 172.85400390625 0.0 0.0 0.0 1.0 0.0 P3: 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 R0_rect: 1.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 1.0 Tr_velo_to_cam: 0.00023477392 -0.99994415 -0.010563477 -0.0027968169 0.010449408 0.0105653545 -0.9998896 -0.07510879 0.99994534 0.0001243655 0.010451303 -0.2721328 Tr_imu_to_velo: 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ``` **Fields:** * `P0`, `P1`, `P2`, `P3`: The 3x4 projection matrices for the four cameras (left and right color and grayscale). These matrices project 3D points in the camera coordinate system into the 2D image plane. * `R0_rect`: The 3x3 rectification matrix for aligning the stereo cameras. It rectifies the rotation differences between the cameras to align them for stereo processing. * `Tr_velo_to_cam`: The 3x4 transformation matrix from the Velodyne LiDAR coordinate system to the camera coordinate system. * `Tr_imu_to_velo`: The 3x4 transformation matrix from the IMU coordinate system to the Velodyne LiDAR coordinate system. ## Useful links * [KITTI homepage](https://www.cvlibs.net/datasets/kitti/) * [MMDetection3D Documentation on KITTI 3D dataset](https://mmdetection3d.readthedocs.io/en/v0.17.3/datasets/kitti_det.html) # PCD with labels ## Overview Import point clouds stored in PCD files with semantic segmentation labels stored as a per-point field. This format is intended for PCD files where every point can have a numeric class identifier. Class names are not stored in the PCD file itself, so a `class_mapping.json` file is required. During import, mapped label IDs are converted to Supervisely point cloud segmentation annotations. Label IDs that are not present in `class_mapping.json` are ignored. ## Format description **Supported point cloud format:** `.pcd`\ **With annotations:** yes\ **Supported annotation source:** `classification`, `label`, or `labels` field inside each `.pcd` file\ **Required mapping file:** `class_mapping.json`\ **Photo context:** not supported\ **Data structure:** Information is provided below. ## Input files structure Example data: [download ⬇️](https://github.com/user-attachments/files/28470515/pcd_semantic_labels_demo.zip). Both directory and archive are supported. Recommended directory structure: ``` 📦pcd_with_labels (folder or .tar/.zip archive) ├──📄class_mapping.json ├──📄scene_01.pcd ├──📄scene_02.pcd ├──📄scene_03.pcd └──📄... ``` Nested directories are supported. The project must contain exactly one `class_mapping.json` file. ## PCD requirements Each PCD file must contain one semantic label field. The converter checks the following field names in priority order: * `classification` * `label` * `labels` If none of these fields are present, the PCD file is uploaded without annotation. If more than one of these fields is present in the same PCD file, the converter uses the first available field by this priority order. Example PCD header: ``` # .PCD v0.7 - Point Cloud Data file format VERSION 0.7 FIELDS x y z labels scan_idx timestamp intensity SIZE 4 4 4 1 4 8 1 TYPE F F F U I I U COUNT 1 1 1 1 1 1 1 WIDTH 324681 HEIGHT 1 VIEWPOINT 0 0 0 1 0 0 0 POINTS 324681 DATA binary ``` The semantic label field must contain one scalar numeric value per point. Its `COUNT` must be `1`. If the PCD header does not contain a `COUNT` line, all fields are treated as having `COUNT 1` according to the PCD format. The converter supports PCD `DATA ascii` and `DATA binary`. ## Class mapping The `class_mapping.json` file must be a flat JSON object. Keys are label IDs, and values are Supervisely class names. Example: ```json { "1": "person", "2": "Ground", "3-10": "car" } ``` Keys can be: * A concrete label ID, for example `"2"`. * An inclusive range, for example `"3-10"`. Values must be non-empty strings. The example above is interpreted as: ```json { "1": "person", "2": "Ground", "3": "car", "4": "car", "5": "car", "6": "car", "7": "car", "8": "car", "9": "car", "10": "car" } ``` Overlapping keys are not allowed. For example, the following mapping is invalid because label ID `2` is defined twice: ```json { "2": "Ground", "1-3": "car" } ``` ## Conversion behavior For each PCD file, the converter reads the semantic label value of every point and groups point indices by class name from `class_mapping.json`. Each mapped class becomes a Supervisely object class with `point_cloud` geometry. Each group of point indices becomes one Supervisely figure: ```json { "geometryType": "point_cloud", "geometry": { "indices": [5, 7, 17, 28] } } ``` Unmapped label IDs are ignored and do not create classes, objects, or figures. If a PCD file has no semantic label field or none of its label values are present in `class_mapping.json`, that file is uploaded with an empty annotation. ## Notes PCD does not define an official semantic label field name. The field stores numeric values only. The meaning of those values is dataset-specific and must be provided through `class_mapping.json`. Photo context images and related image annotations are not supported by this format. If you want to use ASPRS LAS classification names as a mapping preset, provide them explicitly in `class_mapping.json`. User-definable or reserved ranges can also be represented with range keys if needed. ## Useful links * [PCD file format](https://pointclouds.org/documentation/tutorials/pcd_file_format.html) * [ASPRS LAS Classification](https://lasformat.org/latest/02.00_definition.html#classification) * [Supervisely Pointcloud Annotation](https://developer.supervisely.com/getting-started/supervisely-annotation-format/point-clouds) # Pointcloud Episodes # Supervisely ## Overview {% hint style="success" %} Easily import your pointcloud episodes with annotations in the Supervisely format. The Supervisely json-based annotation format supports `cuboid_3d` shape figures. It is a universal format that supports various types of annotations and is used in the Supervisely platform. {% endhint %} {% hint style="info" %} All information about the Supervisely JSON format can be found [here](https://docs.supervise.ly/data-organization/00_ann_format_navi) {% endhint %} Enterprise users have access to "Import as links" option, which supports import of this format with annotations. This option might be beneficial in many cases, as it allows data import to Supervisely platform without re-uploading, maintaining a single source and speeding up import process. ## Format Description **Supported point cloud format:** `.pcd`\ **With annotations:** yes\ **Supported annotation format:** `.json`.\ **Data structure:** Information is provided below. ## Input Files Structure {% hint style="success" %} Download example data in Supervisely format - [Demo KITTI pointcloud episodes project (207 MB)](https://github.com/supervisely-ecosystem/demo-kitti-3d-episodes/releases/download/v0.0.1/demo_kitti_pointcloud_episodes.zip). {% endhint %} Both directory and archive are supported. **Recommended directory structure:** Root folder for the project named `project name` * `meta.json` file * `key_id_map.json` file * Dataset folders, that represents single episode. Each named `dataset_name`, which contains: * `annotation.json` - file with whole episode annotation * `frame_pointcloud_map.json` - file with pointcloud to episode frame mapping * `pointcloud` folder, contains source point cloud files, for example `frame1.pcd, frame2.pcd` * `related_images` optional folder, contains photo-context data: * Frame folder, each named according to pointcloud (`/related_images/frame1/`), which contains: * image files (`.png \ .jpg`) * photo context image annotation file (`.json`) - json files, named according to image name (`1.png -> 1.png.json`). Read more in the "Photo context image annotation file" section below. ## Format of Annotations ```json { "description": "", "key": "e9f0a3ae21be41d08eec166d454562be", "tags": [], "objects": [ { "key": "6663ca1d20c74bea83bd48c24568989d", "classTitle": "car", "tags": [] } ], "framesCount": 48, "frames": [ { "index": 0, "figures": [ { "key": "cb8e067dadfc423aa8575a0c4e62de33", "objectKey": "6663ca1d20c74bea83bd48c24568989d", "geometryType": "cuboid_3d", "geometry": { "position": { "x": -10.863547325134277, "y": -93.57706451416016, "z": -4.598618030548096 }, "rotation": { "x": 0, "y": 0, "z": 3.250733629393711 }, "dimensions": { "x": 1.978, "y": 4.607, "z": 1.552 } } } ] }, { "index": 1, "figures": [ { "key": "71e0fe52dc4f4f6aaf059ad095f43c1f", "objectKey": "6663ca1d20c74bea83bd48c24568989d", "labelerLogin": "username", "updatedAt": "2021-11-11T17:19:11.448Z", "createdAt": "2021-11-11T16:53:03.670Z", "geometryType": "cuboid_3d", "geometry": { "position": { "x": -11.10418701171875, "y": -91.33098602294922, "z": -4.5446248054504395 }, "rotation": { "x": 0, "y": 0, "z": 3.24780199600921 }, "dimensions": { "x": 1.978, "y": 4.607, "z": 1.552 } } } ] } ] } ``` **Optional fields** These fields are optional and are not needed when uploading the project. The server can automatically fill in these fields while project is loading. * `id` - unique identifier of the current object * `classId` - unique class identifier of the current object * `labelerLogin` - string - the name of user who created the current figure * `createdAt` - string - date and time of figure creation * `updatedAt` - string - date and time of the last figure update Main idea of `key` fields and `id` you can see below in "Key id map file" section. **Fields definitions:** * `description` - string - (optional) - this field is used to store the text to assign to the sequence. * `key` - string, unique key for a given sequence (used in key\_id\_map.json to get the sequence ID) * `tags` - list of strings that will be interpreted as episode tags * `objects` - list of objects that may be present on the episode * `frames` - list of frames of which the episode consists. List contains only frames with an object from the 'objects' field * `index` - integer - number of the current frame * `figures` - list of figures in the current frame. * `framesCount` - integer - total number of frames in the episode * `geometryType` - "cuboid\_3d" - class shape **Fields definitions for `objects` field:** * `key` - string - unique key for a given object (used in key\_id\_map.json) * `classTitle` - string - the title of a class. It's used to identify the class shape from the `meta.json` file * `tags` - list of strings that will be interpreted as object tags (can be empty) **Fields description for `figures` field:** * `key` - string - unique key for a given figure (used in key\_id\_map.json) * `objectKey` - string - unique key to link figure to object (used in key\_id\_map.json) * `geometryType` - "cuboid\_3d" -class shape * `geometry` - geometry of the object **Description for `geometry` field:** * `position` 3D vector X, Y, Z values matches the axes on world coordinates, defined in global frame of reference as: * **+x** - forward in the direction of travel ego vehicle * **+y** - left * **+z** - up * `dimensions` is 3D vector with: * **x** - width * **y** - length * **z** - height * `rotation`is 3D Vector with: * **x** - pitch * **y** - roll * **z** - yaw (direction) Rotation values bound inside \[**-pi** ; **pi** ] When `yaw = 0` box direction will be strict `+y` ### Key-id-map File You can avoid using key-id-map directly with API and SDK to create your own file structure. The basic idea behind key-id-map is that it maps the unique identifiers of the object to the frame on which the shape is located. The server works with an identifier, but the file system of the loaded project stores these identifiers and object keys on disk, which is necessary for navigation and use of the high-level API and applications. When loading a `dataset` (sequence), the system returns its identifier, after which it is saved to a file on disk, along with the key of the loaded sequence in key-id-map file. When uploading `objects` to the server, a sequence ID is required (to determine which sequence the object belongs to), and it can be read from the key-id-map file by key. The system then returns the IDs of the successfully loaded objects. Then, while `figures` uploading to the server, an object identifier is required (which loaded object the shape belongs to), which can again be read from the key-id-map file. While annotating the episode inside Supervisely interface key-id-map file is created automatically, and will be downloaded with the entire project. Json format of key\_id\_map.json: ```json { "tags": {}, "objects": { "198f727d40c749eebcacc4aed299b39a": 20520 }, "figures": { "65f21690780e43b49863c3cbd07eab3a": 503130811 }, "videos": { "e9f0a3ae21be41d08eec166d454562be": 42656 } } ``` * `objects` - dictionary, where the key is a unique string, generated inside Supervisely environment to set correspondence of current object in annotation, and values are unique integer ID corresponding to the current object * `figures` - dictionary, where the key is a unique string, generated inside Supervisely environment to set correspondence of object on current frame in annotation, and values are unique integer ID corresponding to the current frame * `videos` - dictionary, where the key is unique string, generated inside Supervisely environment to set correspondence of sequence (dataset) in annotation, and value is a unique integer ID corresponding to the current sequence * `tags` - dictionary, where the keys are unique strings, generated inside Supervisely environment to set correspondence of tag on current frame in annotation, and values are a unique integer ID corresponding to the current tag * **Key** - [generated by python3 function `uuid.uuid4().hex`](https://docs.python.org/3/library/uuid.html#uuid.uuid4). The unique string. All key values and id's should be unique inside single project and can not be shared between frames\sequences. * **Value** - returned by server integer identifier while uploading object / figure / sequence / tag ### Format of `frame_pointcloud_map.json` This file set mapping between pointcloud files and annotation frames in the correct order. ```json { "0": "frame1.pcd", "1": "frame2.pcd", "2": "frame3.pcd" } ``` **Keys** - frame order number\ **Values** - point cloud name (with extension) ### Photo Context Image Annotation File ```json { "name": "host-a005_cam4_1231201437716091006.jpeg" "entityId": 2359620, "meta": { "deviceId": "CAM_BACK_LEFT", "timestamp": "2019-01-11T03:23:57.802Z", "sensorsData": { "extrinsicMatrix": [ -0.8448329028461443, -0.5350302199120708, 0.00017334762588639086, -0.012363736761232369, -0.0035124448582330757, 0.005222293412494302, -0.9999801949951969, -0.16621728572112304, 0.5350187183638307, -0.8448167798004226, -0.006291229448121315, -0.3527897896721229 ], "intrinsicMatrix": [ 882.42699274, 0, 602.047851885, 0, 882.42699274, 527.99972239, 0, 0, 1 ] } } } ``` **Fields description:** * name - string - Name of image file * Id - (OPTIONAL) - integer >= 1 ID of the photo in the system. It is not required when upload and is filled in automatically when the project is loaded. * entityId (OPTIONAL) - integer >= 1 ID of the Point Cloud in the system, that photo attached to. Doesn't required while uploading. * deviceId - string- Device ID or name. * timestamp - string - Time when the frame occurred in ISO 8601 format * sensorsData - Sensors data such as Pinhole camera model parameters. See wiki: [Pinhole camera model](https://en.wikipedia.org/wiki/Pinhole_camera_model) and [OpenCV docs for 3D reconstruction](https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html). * intrinsicMatrix - Array of number - 3x3 flatten matrix (dropped last zeros column) of intrinsic parameters in row-major order, also called camera matrix. It's used to denote camera calibration parameters. See [Intrinsic parameters](https://en.wikipedia.org/wiki/Camera_resectioning#Intrinsic_parameters). * extrinsicMatrix - Array of number - 4x3 flatten matrix (dropped last zeros column) of extrinsic parameters in row-major order, also called joint rotation-translation matrix. It's used to denote the coordinate system transformations from 3D world coordinates to 3D camera coordinates. See [Extrinsic\_parameters](https://en.wikipedia.org/wiki/Camera_resectioning#Extrinsic_parameters). ## Useful Links * [Supervisely Annotation Format](/customization-and-integration/00_ann_format_navi) * [Supervisely Pointcloud Episodes Annotation](/customization-and-integration/00_ann_format_navi/07_supervisely_format_pointcloud_episode) * [\[SDK CLI\] Upload projects in Supervisely format](https://developer.supervisely.com/getting-started/command-line-interface/sdk-cli#upload-a-project) * [\[Supervisely Ecosystem\] Import Pointcloud Episodes in Supervisely](https://developer.supervisely.com/getting-started/command-line-interface/cli-tool/workflow-automation#upload-projects-in-supervisely-format) # .PCD, .PLY, .LAS, .LAZ pointclouds ## Overview This option allows you to upload point clouds as episodes to the platform without any annotations. All items from the input directory and its subdirectories will be uploaded to a single dataset. Supported formats are `.pcd`, `.ply`, `.las`, and `.laz`, and they are imported natively without conversion. If you need to preserve the directory structure, you can use the [Import Pointcloud Episodes](https://ecosystem.supervisely.com/apps/import-pointcloud-episode) application from the Supervisely Ecosystem. ## Format description **Supported point cloud episode formats:** `.pcd`, `.ply`, `.las`, `.laz`.\ **With annotations:** No\ **Supported annotation format:** Not applicable.\ **Grouped by:** Any structure (will be uploaded to a single dataset).
## Why native LAS/LAZ/PLY import matters For point cloud episodes, Import Wizard loads `.las`, `.laz`, and `.ply` frames natively. `PCD` is a simpler point cloud format. `LAS` / `LAZ` and many `PLY` variants can keep more useful source information per frame, so native import helps preserve the original sequence data. Practical advantages for multi-frame projects: * **More source attributes preserved across the sequence**: `LAS` / `LAZ` can keep LiDAR-specific fields such as intensity, return information, and classification. `PLY` can keep RGB, normals, and custom properties. Native import avoids simplifying those frames through an intermediate conversion step. * **No conversion-related data loss**: when episodes are converted to `PCD`, source-specific fields may be lost or reduced. Native import keeps the original frame content intact. * **Smaller files can improve sequence loading**: formats such as `LAZ` can be significantly smaller than converted `PCD`, which reduces transfer size and helps large multi-frame projects open faster. * **Faster project startup**: there is no pre-conversion pipeline before frame mapping and sequence upload. * **Immediate benefit from WebGPU rendering upgrades**: preserved attributes and dense frames can be rendered directly in the optimized pipeline, which is designed for large scenes and high point counts. ## Input files structure {% hint style="success" %} Example data: [download ⬇️](https://github.com/supervisely-ecosystem/import-wizard-docs/files/15025197/sample_pcde.zip) Example data with related images: [download ⬇️](https://github.com/supervisely-ecosystem/import-wizard-docs/files/15025207/sample_pcde_w_rimg.zip) {% endhint %} Recommended directory structure: ``` 📦 folder (with related images) 📦 folder ├── 📂 pointcloud ├── 📂 pointcloud │ ├── 📄 0000000000.pcd │ ├── 📄 0000000000.pcd │ ├── 📄 0000000001.ply │ ├── 📄 0000000001.ply │ ├── 📄 0000000002.las │ ├── 📄 0000000002.las │ └── 📄 0000000003.laz │ └── 📄 0000000003.laz ├── 📂 related_images └── 📜 frame_pointcloud_map.json │ ├── 📂 0000000000_pcd │ │ ├── 🖼️ 0000000000.png │ │ └── 📜 0000000000.png.json │ ├── 📂 0000000001_ply │ │ ├── 🖼️ 0000000001.png │ │ └── 📜 0000000001.png.json │ ├── 📂 0000000002_las │ │ ├── 🖼️ 0000000002.png │ │ └── 📜 0000000002.png.json │ ├── 📂 0000000003_laz │ │ ├── 🖼️ 0000000003.png │ │ └── 📜 0000000003.png.json └── 📜 frame_pointcloud_map.json ``` Frames mapping file structure:
📜 frame_pointcloud_map.json ```json { "0": "0000000000.pcd", "1": "0000000001.ply", "2": "0000000002.las", "3": "0000000003.laz" } ```
## Format LAS/LAZ/PLY In Import Wizard, LAS/LAZ/PLY files are imported natively without conversion. ## Useful links * [\[Supervisely Ecosystem\] Import Pointcloud Episodes](https://ecosystem.supervisely.com/apps/import-pointcloud-episode) # Lyft ## Overview {% hint style="success" %} Easily import your pointclouds with annotations in the LYFT format. LYFT is an annotation format used in the well-regarded `Lyft Level 5 Prediction` dataset. {% endhint %} When uploading as Pointcloud Episodes, pointclouds will be grouped by scene, and objects will be matched in a sequence as well. ## Format description **Supported point cloud format:** `.bin`\ **With annotations:** yes\ **Supported annotation format:** `.json`\ **Data structure:** Information is provided below. ## Input files structure {% hint style="success" %} Example data: [download ⬇️](https://github.com/supervisely-ecosystem/import-wizard-docs/releases/download/v0.0.2/lyft-sample.zip). {% endhint %} Both directory and archive are supported. **Format directory structure:** ``` 📦pointcloud_project (folder or .tar/.zip archive) ├──📂dataset1 │ ├──📂data │ │ ├──📄attribute.json │ │ ├──📄calibrated_sensor.json │ │ ├──📄category.json │ │ ├──📄ego_pose.json │ │ ├──📄instance.json │ │ ├──📄log.json │ │ ├──📄map.json │ │ ├──📄sample.json │ │ ├──📄sample_annotation.json │ │ ├──📄sample_data.json │ │ ├──📄scene.json │ │ ├──📄sensor.json │ │ └──📄visibility.json │ ├──📂images │ │ ├──🏞️host-a101_cam0_0000000000000000001.jpeg │ │ ├──🏞️host-a101_cam0_0000000000000000002.jpeg │ │ ├──🏞️host-a101_cam0_0000000000000000003.jpeg │ │ ├──🏞️host-a101_cam0_0000000000000000004.jpeg │ │ ├──🏞️host-a101_cam0_0000000000000000005.jpeg │ │ └──🏞️... │ ├──📂lidar │ │ ├──📄host-a101_lidar1_1241893239302414726.bin │ │ ├──📄host-a101_lidar1_1241893239302414727.bin │ │ ├──📄host-a101_lidar1_1241893239302414728.bin │ │ ├──📄host-a101_lidar1_1241893239302414729.bin │ │ ├──📄host-a101_lidar1_1241893239302414730.bin │ │ └──📄... │ └──📂maps └── └──🏞️map_raster_palo_alto.png ``` Every `.bin` file in a sequence has to be stored inside a `lidar` folder of dataset. | Key | Value | | --- | ------------------------------- | | x | The x coordinate of the point. | | y | The y coordinate of the point. | | z | The z coordinate of the point. | | i | Intensity of the return signal. | | r | Ring index. | {% hint style="info" %} The LYFT format description can be found [here](https://mmdetection3d.readthedocs.io/en/stable/advanced_guides/datasets/lyft.html) {% endhint %} ## LYFT Annotation format The original LYFT dataset's annotations are token-based. The dataset is composed of dozens of scenes, each consisting of 126 point cloud frames, named samples. Samples are linked to annotation data through the use of unique identifiers, known as tokens. These tokens ensure that each point cloud frame is accurately associated with its corresponding annotations, maintaining the integrity and structure of the dataset. #### `attribute.json` The `attribute.json` file contains metadata about the attributes of objects in the dataset. Each entry includes a unique token, a name, and a description of the attribute. Content example: ```json { "description": "", "token": "f5081f1e5aa941f9d9f727ad186c8db67b916336f975a2f5d65d14ea01ed098f", "name": "object_action_lane_change_right" }, { "token": "1b388c1f5e5149ae173ad3d674e7ad7f1847e213173d14ce5ecf431ad697ca17", "description": "", "name": "object_action_running" }, { "description": "", "token": "17d61007ee69782e0ad8ffa5f8cd4c075f18b4b09e11f0e966bc27026c7929ea", "name": "object_action_lane_change_left" } ``` #### `calibrated_sensor.json` The `calibrated_sensor.json` file contains calibration data for each sensor used in the dataset. This includes information about the sensor's position, orientation, and intrinsic parameters. Each entry in the file is associated with a unique sensor token. Content example: ```json { "sensor_token": "172a55e2b50f18a6b6d545369a457003c2f3b438d0180b2b4c7819ca29b3f6ab", "rotation": [ -0.4944743277529699, 0.5037402715904608, 0.5076908453442883, -0.49395433344067297 ], "camera_intrinsic": [ [ 1112.8384901, 0, 958.488205774 ], [ 0, 1112.8384901, 539.540735426 ], [ 0, 0, 1 ] ], "translation": [ 0.8201327486252449, -0.002437920474569259, 1.6531257722973938 ], "token": "59155106c0ac5abe83cb6558ad8ce98400e3c3abf51234734bc89bc9d613470a" } ``` #### `category.json` The `category.json` file contains metadata about the categories of objects in the dataset. Each entry includes a unique token, a name, and a description of the category. Contents example: ```json { "description": "", "token": "8eccddb83fa7f8f992b2500f2ad658f65c9095588f3bc0ae338d97aff2dbcb9c", "name": "car" }, { "description": "", "token": "73e8de69959eb9f5b4cd2859e74bec4b5491417336cad63f27e8edb8530ffbf8", "name": "pedestrian" }, { "description": "", "token": "f81f51e1897311b55c0c6247c3db825466733e08df687c0ea830b026316a1c12", "name": "animal" } ``` #### `ego_pose.json` The `ego_pose.json` file contains the position and orientation of the ego vehicle at each timestamp. Each entry includes a unique token, rotation, translation, and timestamp. Entry example: ```json { "rotation": [ -0.6004078747001649, -0.0008682874404776075, 0.0018651459228554213, 0.7996912850004297 ], "translation": [ 1007.2332778546739, 1725.4217301399474, -24.580000733806127 ], "token": "0c257254dad346c9d90f7970ce2c0b8142f7c6e6a90716f4c0538cd2d2ef77d5", "timestamp": 1557858039302414.8 } ``` #### `instance.json` The `instance.json` file contains metadata about each instance in the dataset. Each entry includes a unique token, category token, and tokens linking to the first and last annotations of the instance. The file also includes the number of annotations associated with each instance. Content example: ```json { "last_annotation_token": "36a6954dcf400dcca8a25353939f106c09d43fcd40a3ba18add54bd149e8afb5", "category_token": "8eccddb83fa7f8f992b2500f2ad658f65c9095588f3bc0ae338d97aff2dbcb9c", "token": "695097711d9ec763b55f24d04ae9eb51289575645968f00723cee666a9f68c27", "first_annotation_token": "2a03c42173cde85f5829995c5851cc81158351e276db493b96946882059a5875", "nbr_annotations": 92 } ``` #### `log.json` The `log.json` file contains metadata about the logging information for each scene in the dataset. Each entry includes a unique token, the date the data was captured, the location, the vehicle used, and a token linking to the corresponding map. ```json { "date_captured": "2019-05-14", "location": "Palo Alto", "token": "da4ed9e02f64c544f4f1f10c6738216dcb0e6b0d50952e158e5589854af9f100", "vehicle": "a101", "logfile": "", "map_token": "53992ee3023e5494b90c316c183be829" } ``` #### `map.json` The `map.json` file contains metadata about the maps used in the dataset. Each entry represents a single map and includes a unique token, the filename of the map image, the category of the map, and a list of tokens linking to the logs associated with the map itself. Content example: ```json { "log_tokens": [ "da4ed9e02f64c544f4f1f10c6738216dcb0e6b0d50952e158e5589854af9f100", "0a6839d6ee6804113bb5591ed99cc70ad883d0cff396e3aec5e76e718771b30e", "a939e6edc494777d058c3b1eafb91a7236f6b4ff5e98c9abb1216046a0b3a45f", "25721546712bf8ba8729d88992ba2a72ed04918ffda3081cbefe3e1a6c02173f", "71dfb15d2f88bf2aab2c5d4800c0d10a76c279b9fda98720781a406cbacc583b", "5fbada2dc96f8ae4485af83535d63c24a3116262ae94edd69c4185166d4c5e0e", "47ce1d46dc84a54949a350ff4e0cc855fb0cd8ab02246951a10a95ed6b0f863e", "b28b767dac46db1659065a51d6453129eeb82bf397fdbb779de95414090d6842", "d23bc414dd4c48a55e24d80d86cc54f393ea20649f381af567b53644b761b40a" ], "token": "53992ee3023e5494b90c316c183be829", "filename": "maps/map_raster_palo_alto.png", "category": "semantic_prior" } ``` #### `sample.json` The `sample.json` file contains metadata about each sample in the dataset. Each sample represents a single point cloud frame and includes tokens linking to related data such as annotations and sensor data. The file also includes information about the sample's position in the sequence and the associated scene. Single entry example: ```json { "next": "", "prev": "", "token": "24b0962e44420e6322de3f25d9e4e5cc3c7a348ec00bfa69db21517e4ca92cc8", "timestamp": 1557858039302414.8, "scene_token": "da4ed9e02f64c544f4f1f10c6738216dcb0e6b0d50952e158e5589854af9f100", "data": { "CAM_BACK": "542a9e44f2e26221a6aa767c2a9b90a9f692c3aee2edb7145256b61e666633a4", "CAM_FRONT_ZOOMED": "9c9bc711d93d728666f5d7499703624249919dd1b290a477fcfa39f41b26259e", "LIDAR_FRONT_RIGHT": "8cfae06bc3d5d7f9be081f66157909ff18c9f332cc173d962460239990c7a4ff", "CAM_FRONT": "fb40b3b5b9d289cd0e763bec34e327d3317a7b416f787feac0d387363b4d00f0", "CAM_FRONT_LEFT": "f47a5d143bcebb24efc269b1a40ecb09440003df2c381a69e67cd2a726b27a0c", "CAM_FRONT_RIGHT": "5dc54375a9e14e8398a538ff97fbbee7543b6f5df082c60fc4477c919ba83a40", "CAM_BACK_RIGHT": "ae8754c733560aa2506166cfaf559aeba670407631badadb065a9ffe7c337a7d", "CAM_BACK_LEFT": "01c0eecd4b56668e949143e02a117b5683025766d186920099d1e918c23c8b4b", "LIDAR_TOP": "ec9950f7b5d4ae85ae48d07786e09cebbf4ee771d054353f1e24a95700b4c4af", "LIDAR_FRONT_LEFT": "5c3d79e1cf8c8182b2ceefa33af96cbebfc71f92e18bf64eb8d4e0bf162e01d4" }, "anns": [ "2a03c42173cde85f5829995c5851cc81158351e276db493b96946882059a5875", "c3c663ed5e7b6456ab27f09175743a551b0b31676dae71fbeef3420dfc6c7b09", "4193e4bf217c8a0ff598d792bdd9d049b496677d5172e38c1ed22394f20274fb", "f543b455c7066d7054ed5218670b4432a3fc7a0f57cadb0cb59aa96e8dd1a135", "6ef405e1f47a4a234f0bfeeb559d61c70031af6c47bf2e499a31002f845cd403", "db7413e79e7df6de53d801e96e26c494caf1d5b709da85aa7bf04e42c76609f5", "40eb44f0b586f2b41fb962adca2c1de351b70f3577a382798db2735564566a33", "1ab21da1d6f6d88107a5b067ba1f40929a47ae414b72e8bc17d774f281776cb2", "3d7bdcb0c99a5ba50cc74b9b9194fe4f2fdfa82a078c8afa561eec9afad052f9", "4e00f88a86a5ae6a015d0a58027a280db2aebf13d4626875c877d23cca553d8d", "64e6ea3476f64420c76db21b2fa4ec589def2c05ad97d2574ea63f8f0f7f22c9" ] } ``` #### `sample_annotation.json` The `sample_annotation.json` file contains detailed information about each annotation in the dataset. This includes the size, position, and orientation of the annotated objects, as well as tokens linking to related data such as attributes and instances. Each entry in the file represents an instance annotation. Single entry example: ```json { "token": "2a03c42173cde85f5829995c5851cc81158351e276db493b96946882059a5875", "num_lidar_pts": -1, "size": [ 1.997, 5.284, 1.725 ], "sample_token": "24b0962e44420e6322de3f25d9e4e5cc3c7a348ec00bfa69db21517e4ca92cc8", "rotation": [ 0.1539744509331139, 0, 0, 0.9880748293827983 ], "prev": "", "translation": [ 1048.155950230245, 1691.8102354006162, -23.304943447792454 ], "num_radar_pts": 0, "attribute_tokens": [ "1ba8c9a8bda54423fa710b0af1441d849ecca8ed7b7f9393ba1794afe4aa6aa2", "daf16a3f6499553cc5e1df4a456de5ee46e2e6b06544686d918dfb1ddb088f6f" ], "next": "9986dac1bcecb560153ab58ae7560028caeed3c1e067b37503cf50932e983afc", "instance_token": "695097711d9ec763b55f24d04ae9eb51289575645968f00723cee666a9f68c27", "visibility_token": "", "category_name": "car" } ``` #### `sample_data.json` The `sample_data.json` file contains metadata about each data sample in the dataset. This includes information about the sensor used to capture the data, the file format, and the associated tokens for calibration and ego pose. Each entry in the file represents a single data sample, linking it to the corresponding point cloud or image file. Single entry example: ```json { "width": 1920, "height": 1080, "calibrated_sensor_token": "59155106c0ac5abe83cb6558ad8ce98400e3c3abf51234734bc89bc9d613470a", "token": "542a9e44f2e26221a6aa767c2a9b90a9f692c3aee2edb7145256b61e666633a4", "sample_token": "24b0962e44420e6322de3f25d9e4e5cc3c7a348ec00bfa69db21517e4ca92cc8", "is_key_frame": true, "prev": "", "fileformat": "jpeg", "ego_pose_token": "0c257254dad346c9d90f7970ce2c0b8142f7c6e6a90716f4c0538cd2d2ef77d5", "timestamp": 1557858039200000, "next": "8d614daa8d1d48d3af4a0c817b676da1cb3e68f1432296eb52cfc428d0ff4d6d", "filename": "images/host-a101_cam3_1241893239200000006.jpeg", "sensor_modality": "camera", "channel": "CAM_BACK" } ``` #### `scene.json` The `scene.json` file contains metadata about each scene in the dataset. Each scene is a sequence of point cloud frames captured over a period of time. The file includes tokens linking to the first and last samples in the scene, the number of samples, and a description. Entry example: ```json { "log_token": "da4ed9e02f64c544f4f1f10c6738216dcb0e6b0d50952e158e5589854af9f100", "first_sample_token": "24b0962e44420e6322de3f25d9e4e5cc3c7a348ec00bfa69db21517e4ca92cc8", "name": "host-a101-lidar0-1241893239199111666-1241893264098084346", "description": "", "last_sample_token": "2346756c83f6ae8c4d1adec62b4d0d31b62116d2e1819e96e9512667d15e7cec", "nbr_samples": 126, "token": "da4ed9e02f64c544f4f1f10c6738216dcb0e6b0d50952e158e5589854af9f100" } ``` #### `sensor.json` The `sensor.json` file contains metadata about each sensor used in the dataset. This includes information about the sensor's modality (e.g., camera, lidar), the channel it corresponds to, and a unique token for each sensor. Example entries: ```json { "modality": "camera", "channel": "CAM_BACK", "token": "172a55e2b50f18a6b6d545369a457003c2f3b438d0180b2b4c7819ca29b3f6ab" }, { "modality": "camera", "channel": "CAM_FRONT_ZOOMED", "token": "286718e1fbc8c8f0ca441969d91c36a8a809666e049e54ce121636100b520946" }, { "modality": "lidar", "channel": "LIDAR_FRONT_RIGHT", "token": "953faed96fd3d2fae3ec03cd2838b312b8c1a9bb7a0629481982870cb28acb67" } ``` #### `visibility.json` The `visibility.json` file contains information about the visibility levels of objects in the dataset. Each entry specifies a visibility level, a description of what that level represents, and a unique token. Original dataset's file contents: ```json [ { "level": "v60-80", "description": "visibility of whole object is between 60 and 80%", "token": "3" }, { "level": "v0-40", "description": "visibility of whole object is between 0 and 40%", "token": "1" }, { "level": "v40-60", "description": "visibility of whole object is between 40 and 60%", "token": "2" }, { "level": "v80-100", "description": "visibility of whole object is between 80 and 100%", "token": "4" } ] ``` ## Useful links * [LYFT Dataset Documentation](https://mmdetection3d.readthedocs.io/en/stable/advanced_guides/datasets/lyft.html) * [Papers With Code's paper on LYFT](https://paperswithcode.com/dataset/lyft-level-5-prediction) * [Lyft SDK GitHub Page](https://github.com/lyft/nuscenes-devkit) * ["Lyft 3D Object Detection for Autonomous Vehicles" on Kaggle](https://www.kaggle.com/competitions/3d-object-detection-for-autonomous-vehicles) # nuScenes ## Overview {% hint style="success" %} Easily import your pointclouds with annotations in the nuScenes format. {% endhint %} The original nuScenes dataset is a comprehensive, large-scale public dataset designed for autonomous driving research, developed by Motional. It includes 1,000 diverse 20-second driving scenes from Boston and Singapore, featuring dense traffic and challenging conditions. With 1.4M camera images, 390k LIDAR sweeps, and 1.4M RADAR sweeps, it provides extensive multimodal sensor data, including annotations for 23 object classes with 3D bounding boxes at 2Hz and object-level attributes like visibility and activity. When uploading as Pointcloud Episodes, pointclouds will be grouped by scene and objects will be tracked across a scene. ## Format description **Supported point cloud format:** `.bin`\ **With annotations:** yes\ **Supported annotation format:** `.json`\ **Data structure:** Information is provided below. ## Input files structure {% hint style="success" %} Example data: [download ⬇️](https://github.com/supervisely-ecosystem/import-wizard-docs/releases/download/v0.0.2/nuscenes-mini-sample.zip). {% endhint %} Both directory and archive are supported. **Format directory structure:** ``` 📦pointcloud_project (folder or .tar/.zip archive) ├──📂dataset1 │ ├──📂maps │ │ ├──🏞️36092f0b03a857c6a3403e25b4b7aab3.png │ │ └──🏞️... │ ├──📂samples │ │ ├──📂CAM_BACK │ │ │ ├──🏞️n008-2018-08-01-15-16-36-0400__CAM_BACK__1533151603537558.jpg │ │ │ └──🏞️... │ │ ├──📂CAM_BACK_LEFT │ │ │ ├──🏞️n008-2018-08-01-15-16-36-0400__CAM_BACK_LEFT__1533151603547405.jpg │ │ │ └──🏞️... │ │ ├──📂CAM_BACK_RIGHT │ │ │ ├──🏞️n008-2018-08-01-15-16-36-0400__CAM_BACK_RIGHT__1533151605028113.jpg │ │ │ └──🏞️... │ │ ├──📂CAM_FRONT │ │ │ ├──🏞️n008-2018-08-01-15-16-36-0400__CAM_FRONT__1533151615912404.jpg │ │ │ └──🏞️... │ │ ├──📂CAM_FRONT_LEFT │ │ │ ├──🏞️n008-2018-08-01-15-16-36-0400__CAM_FRONT_LEFT__1533151614904799.jpg │ │ │ └──🏞️... │ │ ├──📂CAM_FRONT_RIGHT │ │ │ ├──🏞️n008-2018-08-01-15-16-36-0400__CAM_FRONT_RIGHT__1533151611420482.jpg │ │ │ └──🏞️... │ │ ├──📂LIDAR_TOP │ │ │ ├──📄n008-2018-08-01-15-16-36-0400__LIDAR_TOP__1533151611896734.pcd.bin │ │ │ └──📄... │ │ ├──📂RADAR_BACK_LEFT │ │ │ ├──📄n008-2018-08-01-15-16-36-0400__RADAR_BACK_LEFT__1533151610450182.pcd │ │ │ └──📄... │ │ ├──📂RADAR_BACK_RIGHT │ │ │ ├──📄n008-2018-08-01-15-16-36-0400__RADAR_BACK_RIGHT__1533151607538446.pcd │ │ │ └──📄... │ │ ├──📂RADAR_FRONT │ │ │ ├──📄n008-2018-08-01-15-16-36-0400__RADAR_FRONT__1533151610426071.pcd │ │ │ └──📄... │ │ ├──📂RADAR_FRONT_LEFT │ │ │ ├──📄n008-2018-08-01-15-16-36-0400__RADAR_FRONT_LEFT__1533151616430028.pcd │ │ │ └──📄... │ │ ├──📂RADAR_FRONT_RIGHT │ │ │ ├──📄n008-2018-08-01-15-16-36-0400__RADAR_FRONT_RIGHT__1533151609574740.pcd │ │ └─└──📄... │ ├──📂sweeps │ │ ├──📂CAM_BACK │ │ │ └──🏞️... │ │ ├──📂CAM_BACK_LEFT │ │ │ └──🏞️... │ │ ├──📂CAM_BACK_RIGHT │ │ │ └──🏞️... │ │ ├──📂CAM_FRONT │ │ │ └──🏞️... │ │ ├──📂CAM_FRONT_LEFT │ │ │ └──🏞️... │ │ ├──📂CAM_FRONT_RIGHT │ │ │ └──🏞️... │ │ ├──📂LIDAR_TOP │ │ │ └──📄... │ │ ├──📂RADAR_BACK_LEFT │ │ │ └──📄... │ │ ├──📂RADAR_BACK_RIGHT │ │ │ └──📄... │ │ ├──📂RADAR_FRONT │ │ │ └──📄... │ │ ├──📂RADAR_FRONT_LEFT │ │ │ └──📄... │ │ ├──📂RADAR_FRONT_RIGHT │ │ └─└──📄... │ ├──📂v1.0-mini │ │ ├──📄attribute.json │ │ ├──📄calibrated_sensor.json │ │ ├──📄category.json │ │ ├──📄ego_pose.json │ │ ├──📄instance.json │ │ ├──📄log.json │ │ ├──📄map.json │ │ ├──📄sample.json │ │ ├──📄sample_annotation.json │ │ ├──📄sample_data.json │ │ ├──📄scene.json │ │ ├──📄sensor.json └── └──📄visibility.json ``` Every `.bin` file in a sequence has to be stored inside a `LIDAR_TOP` folder of 'sample' and 'sweeps' folders of dataset. | Key | Value | | --- | ------------------------------- | | x | The x coordinate of the point. | | y | The y coordinate of the point. | | z | The z coordinate of the point. | | i | Intensity of the return signal. | | r | Ring index. | {% hint style="info" %} The nuScenes format description can be found [here](https://www.nuscenes.org/nuscenes) {% endhint %} ## nuScenes' Annotation format The nuScenes annotations are token-based. The dataset is composed of dozens of scenes, each containing a certain amount of samples (pointclouds). Samples are linked to annotation data through the use of unique identifiers. These tokens ensure that each point cloud frame is accurately associated with its corresponding annotations, camera data, log information, maps, etc. maintaining the integrity and structure of the dataset. #### `attribute.json` The `attribute.json` file contains metadata about the attributes of objects in the dataset. An attribute is a property of an instance that can change while the category remains the same. Example: a vehicle being parked/stopped/moving, and whether or not a bicycle has a rider. ```json { "token": -- Unique record identifier. "name": -- Attribute name. "description": -- Attribute description. } ``` #### `calibrated_sensor.json` The `calibrated_sensor.json` file contains calibration data for each sensor used in the dataset. Definition of a particular sensor (lidar/radar/camera) as calibrated on a particular vehicle. All extrinsic parameters are given with respect to the ego vehicle body frame. All camera images come undistorted and rectified. ```json { "token": -- Unique record identifier. "sensor_token": -- Foreign key pointing to the sensor type. "translation": [3] -- Coordinate system origin in meters: x, y, z. "rotation": [4] -- Coordinate system orientation as quaternion: w, x, y, z. "camera_intrinsic": [3, 3] -- Intrinsic camera calibration. Empty for sensors that are not cameras. } ``` #### `category.json` The `category.json` file contains metadata about the categories of objects in the dataset. Taxonomy of object categories (e.g. vehicle, human). Subcategories are delineated by a period (e.g. human.pedestrian.adult). ```json { "token": -- Unique record identifier. "name": -- Category name. Subcategories indicated by period. "description": -- Category description. "index": -- The index of the label used for efficiency reasons in the .bin label files of nuScenes-lidarseg. This field did not exist previously. } ``` #### `ego_pose.json` Ego vehicle pose at a particular timestamp. Given with respect to global coordinate system of the log's map. The ego\_pose is the output of a lidar map-based localization algorithm described in our paper. The localization is 2-dimensional in the x-y plane. ```json { "token": -- Unique record identifier. "translation": [3] -- Coordinate system origin in meters: x, y, z. Note that z is always 0. "rotation": [4] -- Coordinate system orientation as quaternion: w, x, y, z. "timestamp": -- Unix time stamp. } ``` #### `instance.json` The `instance.json` file contains metadata about each instance in the dataset. An object instance, e.g. particular vehicle. This table is an enumeration of all object instances we observed. Note that instances are not tracked across scenes, only inside a given scene. ```json { "token": -- Unique record identifier. "category_token": -- Foreign key pointing to the object category. "nbr_annotations": -- Number of annotations of this instance. "first_annotation_token": -- Foreign key. Points to the first annotation of this instance. "last_annotation_token": -- Foreign key. Points to the last annotation of this instance. } ``` #### `log.json` The `log.json` file contains information about the log from which the data was extracted. ```json { "token": -- Unique record identifier. "logfile": -- Log file name. "vehicle": -- Vehicle name. "date_captured": -- Date (YYYY-MM-DD). "location": -- Area where log was captured, e.g. singapore-onenorth. } ``` #### `map.json` The `map.json` file contains metadata about the map data that is stored as binary semantic masks from a top-down view. ```json { "token": -- Unique record identifier. "log_tokens": [n] -- Foreign keys. "category": -- Map category, currently only semantic_prior for drivable surface and sidewalk. "filename": -- Relative path to the file with the map mask. } ``` #### `sample.json` The `sample.json` file contains metadata about each sample in the dataset. A sample is an annotated keyframe at 2 Hz. The data is collected at (approximately) the same timestamp as part of a single LIDAR sweep. ```json { "token": -- Unique record identifier. "timestamp": -- Unix time stamp. "scene_token": -- Foreign key pointing to the scene. "next": -- Foreign key. Sample that follows this in time. Empty if end of scene. "prev": -- Foreign key. Sample that precedes this in time. Empty if start of scene. } ``` #### `sample_annotation.json` The `sample_annotation.json` file contains detailed information about each annotation in the dataset. A bounding box defining the position of an object seen in a sample. All location data is given with respect to the global coordinate system. ```json { "token": -- Unique record identifier. "sample_token": -- Foreign key. NOTE: this points to a sample NOT a sample_data since annotations are done on the sample level taking all relevant sample_data into account. "instance_token": -- Foreign key. Which object instance is this annotating. An instance can have multiple annotations over time. "attribute_tokens": [n] -- Foreign keys. List of attributes for this annotation. Attributes can change over time, so they belong here, not in the instance table. "visibility_token": -- Foreign key. Visibility may also change over time. If no visibility is annotated, the token is an empty string. "translation": [3] -- Bounding box location in meters as center_x, center_y, center_z. "size": [3] -- Bounding box size in meters as width, length, height. "rotation": [4] -- Bounding box orientation as quaternion: w, x, y, z. "num_lidar_pts": -- Number of lidar points in this box. Points are counted during the lidar sweep identified with this sample. "num_radar_pts": -- Number of radar points in this box. Points are counted during the radar sweep identified with this sample. This number is summed across all radar sensors without any invalid point filtering. "next": -- Foreign key. Sample annotation from the same object instance that follows this in time. Empty if this is the last annotation for this object. "prev": -- Foreign key. Sample annotation from the same object instance that precedes this in time. Empty if this is the first annotation for this object. } ``` #### `sample_data.json` The `sample_data.json` file contains metadata about each data sample in the dataset. A sensor data e.g. image, point cloud or radar return. For sample\_data with is\_key\_frame=True, the time-stamps should be very close to the sample it points to. For non key-frames the sample\_data points to the sample that follows closest in time. ```json { "token": -- Unique record identifier. "sample_token": -- Foreign key. Sample to which this sample_data is associated. "ego_pose_token": -- Foreign key. "calibrated_sensor_token": -- Foreign key. "filename": -- Relative path to data-blob on disk. "fileformat": -- Data file format. "width": -- If the sample data is an image, this is the image width in pixels. "height": -- If the sample data is an image, this is the image height in pixels. "timestamp": -- Unix time stamp. "is_key_frame": -- True if sample_data is part of key_frame, else False. "next": -- Foreign key. Sample data from the same sensor that follows this in time. Empty if end of scene. "prev": -- Foreign key. Sample data from the same sensor that precedes this in time. Empty if start of scene. } ``` #### `scene.json` The `scene.json` file contains metadata about each scene in the dataset. A scene is a 20s long sequence of consecutive frames extracted from a log. Multiple scenes can come from the same log. Note that object identities (instance tokens) are not preserved across scenes. ```json { "token": -- Unique record identifier. "name": -- Short string identifier. "description": -- Longer description of the scene. "log_token": -- Foreign key. Points to log from where the data was extracted. "nbr_samples": -- Number of samples in this scene. "first_sample_token": -- Foreign key. Points to the first sample in scene. "last_sample_token": -- Foreign key. Points to the last sample in scene. } ``` #### `sensor.json` The `sensor.json` file specifies sensor types. ```json { "token": -- Unique record identifier. "channel": -- Sensor channel name. "modality": {camera, lidar, radar} -- Sensor modality. Supports category(ies) in brackets. } ``` #### `visibility.json` The visibility of an instance is the fraction of annotation visible in all 6 images. Binned into 4 bins 0-40%, 40-60%, 60-80% and 80-100%. ```json { "token": -- Unique record identifier. "level": -- Visibility level. "description": -- Description of visibility level. } ``` ## Useful links * [nuScenes homepage](https://www.nuscenes.org/nuscenes) * [MMDetection3D Documentation on nuScenes dataset](https://mmdetection3d.readthedocs.io/en/v0.17.1/datasets/nuscenes_det.html) * [nuScenes devkit GitHub page](https://github.com/nutonomy/nuscenes-devkit) # KITTI 360 ## Overview {% hint style="success" %} Easily import your point clouds with annotations in the KITTI-360 format. This converter is designed for the point cloud episodes modality. For the point cloud modality, please refer to [KITTI-3D](https://docs.supervisely.com/import-and-export/import/supported-annotation-formats/pointclouds/kitti3d). {% endhint %} The original KITTI-360 is a large-scale annotated dataset recorded in the suburbs of Karlsruhe, Germany, covering 73.7 km of driving with over 320k images and 100k laser scans. It provides both 3D point clouds and 2D images with dense semantic and instance annotations across 19 classes, and maintains consistent instance IDs over time. The data, captured with a 360° sensing setup including fisheye and perspective cameras, a stereo camera, Velodyne, and SICK laser scanners, is precisely geolocalized using an IMU/GPS system. ## Format description **Supported point cloud format:** `.bin`\ **With annotations:** yes\ **Supported annotation format:** `.xml`\ **Data structure:** Information is provided below. ## Input files structure {% hint style="success" %} Example data: [download ⬇️](https://github.com/supervisely-ecosystem/import-kitti-360/releases/download/v0.0.1/Example_1.zip)
{% endhint %} **Format directory structure:** ``` 📦KITTI-360 ├──📂calibration │ ├──📄calib_cam_to_velo.txt │ ├──📄perspective.txt │ └──📄... ├──📂data_2d_raw │ ├──📂2013_05_28_drive_{seq:04}_sync │ │ ├──📂image_{00|01} │ │ │ ├──📂data_rect │ │ │ │ ├──🏞️{frame:010}.png │ │ │ │ └──🏞️... │ │ ├──📂image_{02|03} │ │ │ └──📂... ├──📂data_3d_raw │ ├──📂2013_05_28_drive_{seq:04}_sync │ │ ├──📂velodyne_points │ │ │ ├──📂data │ │ │ │ ├──📄{frame:010}.bin │ │ │ │ └──📄... ├──📂data_3d_bboxes │ ├──📂train │ │ ├──📄2013_05_28_drive_{seq:04}_sync.xml │ │ └──📄... ├──📂data_poses │ ├──📂2013_05_28_drive_{seq:04}_sync │ │ ├──📄cam0_to_world.txt └── └── └──📄... ``` {% hint style="info" %} Information on the KITTI-360 dataset structure can be found [here](https://www.cvlibs.net/datasets/kitti-360/documentation.php) {% endhint %} ## Useful links * [KITTI-360 homepage](https://www.cvlibs.net/datasets/kitti-360/index.php) * [KITTI-360 GitHub page](https://github.com/autonomousvision/kitti360Scripts) # SemanticKITTI ## What is the SemanticKITTI Format? {% hint style="success" %} Easily import your **LiDAR point cloud sequences** and **3D semantic annotations** using the SemanticKITTI format into Supervisely. {% endhint %} The **SemanticKITTI** format is a widely used standard designed for semantic scene understanding and **autonomous driving** applications. It provides dense point-wise annotations for 3D point cloud episodes, enabling advanced machine learning tasks like **semantic segmentation**, **instance segmentation**, panoptic segmentation, and 3D scene completion. The format supports processing continuous tracking sequences with diverse semantic classes, covering vehicles, pedestrians, buildings, vegetation, road surfaces, and other urban environment objects. ## Input Files Structure {% hint style="success" %} [Download sample dataset](https://github.com/supervisely-ecosystem/demo-semantic-kitti-pointcloud-episodes-annotated/releases/download/v1.0.0/project-example.zip) in SemanticKITTI format (15 MB) {% endhint %} **Format directory structure:** ``` 📦SemanticKITTI ├──📂sequences │ ├──📂00 │ │ ├──📂velodyne │ │ │ ├──📄000000.bin │ │ │ ├──📄000001.bin │ │ │ ├──📄000002.bin │ │ │ └──📄... │ │ ├──📂labels │ │ │ ├──📄000000.label │ │ │ ├──📄000001.label │ │ │ ├──📄000002.label │ │ │ └──📄... │ │ ├──📄calib.txt │ │ ├──📄poses.txt │ │ └──📄times.txt │ ├──📂01 │ │ ├──📂velodyne │ │ │ └──📄... │ │ ├──📂labels │ │ │ └──📄... │ │ ├──📄calib.txt │ │ ├──📄poses.txt │ │ └──📄times.txt │ └──📂... ``` **The SemanticKITTI structure is organized as follows:** * `sequences/` - contains numbered sequence folders * `XX/` - sequence folder (e.g., 00, 01, 02...) * `velodyne/` - contains LiDAR point cloud files in binary format * `labels/` - contains semantic and instance labels for each scan * `calib.txt` - calibration file containing projection matrices * `poses.txt` - camera poses for each scan * `times.txt` - timestamps for each scan ## SemanticKITTI Annotation format ### Point Cloud Files Filename: `NNNNNN.bin` Point cloud files are stored in binary format with `.bin` extension in `velodyne` folder. Each file contains a list of 3D points with intensity values. **Format:** Each point is represented by 4 float32 values: * `x` - X coordinate (float32) * `y` - Y coordinate (float32) * `z` - Z coordinate (float32) * `intensity` - Reflectance value (float32) ### Label Files Filename: `NNNNNN.label` The label files are stored in binary format with the `.label` extension in `labels` folder. Each label file corresponds to a single point cloud scan and contains semantic and instance annotations for each point. **Format:** Each label is a 32-bit unsigned integer (`uint32_t`) encoding both semantic class and instance ID: * **Lower 16 bits** - semantic label (class ID) * **Upper 16 bits** - instance ID (temporally consistent across the sequence) The instance IDs are consistent over the whole sequence, meaning the same object in different scans gets the same ID. This applies to both moving and static objects. ### Calibration File Filename: `calib.txt` The calibration file contains projection matrices for transforming between coordinate systems. It includes: * `P0`, `P1`, `P2`, `P3` - Camera projection matrices (3x4) * `Tr` - Transformation matrix from Velodyne to camera coordinates (3x4 or 4x4) ### Poses File Filename: `poses.txt` The poses file contains the camera pose (transformation from camera coordinates to world coordinates) for each scan in the sequence. Each line represents a pose as a 3x4 transformation matrix (flattened to 12 values). **Format:** Each line contains 12 float values representing the first three rows of a 4x4 transformation matrix (the last row is \[0, 0, 0, 1]). ### Times File Filename: `times.txt` The times file contains timestamps for each scan in the sequence. Each line contains a single float value representing the timestamp in seconds. ## Export to SemanticKITTI Format You can export your labeled point cloud episodes data to SemanticKITTI format using the [Export to SemanticKITTI](https://ecosystem.supervisely.com/apps/export-to-semantic-kitti) application from the Supervisely Ecosystem. ## License The SemanticKITTI dataset is distributed under the [Creative Commons Attribution-NonCommercial-ShareAlike 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/) license. You are free to share and adapt the data, but you must give appropriate credit and may not use the work for commercial purposes. When using the SemanticKITTI dataset, please cite: ```bibtex @inproceedings{behley2019iccv, author = {J. Behley and M. Garbade and A. Milioto and J. Quenzel and S. Behnke and C. Stachniss and J. Gall}, title = {{SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences}}, booktitle = {Proc. of the IEEE/CVF International Conf.~on Computer Vision (ICCV)}, year = {2019} } ``` And the original KITTI Vision Benchmark: ```bibtex @inproceedings{geiger2012cvpr, author = {A. Geiger and P. Lenz and R. Urtasun}, title = {{Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite}}, booktitle = {Proc.~of the IEEE Conf.~on Computer Vision and Pattern Recognition (CVPR)}, pages = {3354--3361}, year = {2012} } ``` # Volumes # Supervisely ## Overview {% hint style="success" %} Easily import your volumes with annotations in the Supervisely format. The Supervisely json-based annotation format supports such figures: `rectangle`, `line (polyline)`, `polygon`, `point`, `bitmap` (`mask`), `graph` (`keypoints`). It is a universal format that supports various types of annotations and is used in the Supervisely platform. {% endhint %} {% hint style="info" %} All information about the Supervisely JSON format can be found [here](https://docs.supervisely.com/data-organization/00_ann_format_navi) {% endhint %} ## Format description **Supported volume formats:** `.nrrd`, `.dcm`\ **With annotations:** yes\ **Supported annotation format:** `.json`.\ **Data structure:** Information is provided below. ## Input files structure {% hint style="success" %} Example data: [download ⬇️](https://github.com/supervisely-ecosystem/import-volumes-with-anns/releases/download/untagged-d5e038177e15d48a8fb4/Volume_Project.tar). {% endhint %} Both directory and archive are supported. **Recommended directory structure:** Root 📁 `project_name` folder named with the project name * 📄 `meta.json` file * 📄 `key_id_map.json` file (optional) * 📁 `dataset_name` folders, each named with the dataset name and containing: * 📁 `volume` folder, contains source volume files in [NRRD file-format](https://teem.sourceforge.net/nrrd/index.html), for example `CTChest.nrrd` * 📁 `ann` - folder, with annotations for volumes. (named as volume + `.json`) for example `CTChest.nrrd.json` * 📁 `mask` optional folder, created automatically while downloading project. * 📁 folders, named according to volume (`CTChest.nrrd`), which contains an additional data files with geometries for annotation objects of class type `Mask3D` stored in [NRRD file format](https://teem.sourceforge.net/nrrd/index.html), named with hex hash code of objects from key\_id\_map. For example: `daff638a423a4bcfa34eb12e42243a87.nrrd` * 📁 `interpolation` ℹ️ optional folder, created automatically while downloading project. * 📁 folders, named according to volume (`CTChest.nrrd`), which contains an additional data files in [STL file format](https://github.com/supervisely/docs/blob/master/data-organization/import/import/supported-formats-volumes/%3Chttps:/en.wikipedia.org/wiki/STL_\(file_format/README.md)>), named with hex hash code of objects from key\_id\_map. For example: `24a56a26ed784e648d3dd6c5186b46ca.stl` ℹ️ - It is recommended to upload 3D objects as Mask3D and not to use STL. But if you already have a prepared STL file, all STL interpolations will be automatically converter to a Mask3D object during project upload. ## Format of Annotations **Example:** annotation JSON file - `/project_name/dataset_name/ann/CTChest.nrrd.json` ```json { "volumeMeta": { "ACS": "RAS", "intensity": { "max": 3071, "min": -3024 }, "windowWidth": 6095, "rescaleSlope": 1, "windowCenter": 23.5, "channelsCount": 1, "dimensionsIJK": { "x": 512, "y": 512, "z": 139 }, "IJK2WorldMatrix": [ 0.7617189884185793, 0, 0, -194.238403081894, 0, 0.7617189884185793, 0, -217.5384061336518, 0, 0, 2.5, -347.7500000000001, 0, 0, 0, 1 ], "rescaleIntercept": 0 }, "key": "bfed5ee444d849118d7aabc350248cb8", "tags": [], "objects": [ { "key": "f1f495a8e0a64fd7a63efbd78af8ef56", "classTitle": "lung_bitmap", "tags": [], "labelerLogin": "username", "createdAt": "2021-11-13T08:05:28.771Z", "updatedAt": "2021-11-13T08:05:28.771Z" }, { "key": "9a0367647d6c48a6bc104a8b8b276adb", "classTitle": "lung_rectangle", "tags": [] }, { "key": "6c1587f381bf419e9d5c2ebd5967e28f", "classTitle": "lung_mask3d", "tags": [], "labelerLogin": "username", "createdAt": "2021-11-13T08:05:28.771Z", "updatedAt": "2021-11-13T08:05:28.771Z" } ], "planes": [ { "name": "axial", "normal": { "x": 0, "y": 0, "z": 1 }, "slices": [ { "index": 51, "figures": [ { "key": "4c68e29372ef4e3a9c87a233ffabd3dd", "objectKey": "f1f495a8e0a64fd7a63efbd78af8ef56", "geometryType": "bitmap", "geometry": { "bitmap": { "data": "eJwBp ... AADUlIRFIAAACeA==", "origin": [156, 275] } }, "labelerLogin": "username", "createdAt": "2021-11-13T08:05:28.771Z", "updatedAt": "2021-11-13T08:05:28.771Z" } ] }, { "index": 68, "figures": [ { "key": "9bddbbceaa6646cf894e80d3bffd7a55", "objectKey": "9a0367647d6c48a6bc104a8b8b276adb", "description": "", "geometryType": "rectangle", "geometry": { "points": { "exterior": [ [305, 380], [167, 256] ], "interior": [] } }, "customData": { "0-0-1": { "68": { "score": "0.68", "comment": "some comment" } } } } ] } ] } ], "spatialFigures": [ { "key": "daff638a423a4bcfa34eb12e42243a87", "objectKey": "6c1587f381bf419e9d5c2ebd5967e28f", "geometryType": "mask_3d", "geometry": { "mask_3d": { "data": "H4sIAGW9OmUC ... CYAE1Nj5QMACwC", "space_origin": [194, 218, -348] }, "shape": "mask_3d", "geometryType": "mask_3d" }, "labelerLogin": "username", "updatedAt": "2021-11-13T08:05:28.771Z", "createdAt": "2021-11-13T08:05:28.771Z", "customData": { "0-0-1": { // axial plane "51": { "score": "0.11", "comment": "somment" }, "68": { "score": "0.51", "comment": "some comment" } }, "0-1-0": { // coronal plane "51": { "score": "0.97", "comment": "123456" } }, "1-0-0": { // sagittal plane "51": { "comment": "qwerty" } } } } ] } ``` #### Annotation JSON fields definitions: * `volumeMeta` - metadata for 3D reconstruction of volume * `key` - string - a unique identifier of given object represented as `UUID.hex` value (used in `key_id_map.json` to get the object ID) * `tags` - list of strings that will be interpreted as volume tags * `objects` - list of objects that may be present on the volume * `planes` - a list of figures that defined in these planes: [`coronal, sagittal, axial`](https://www.slicer.org/wiki/Coordinate_systems#Anatomical_coordinate_system) * `spatialFigures` - list of 3D figures may be present as the volume annotation **`volumeMeta` fields description:** * `ACS` - string - "RAS" or "LPS" - name of type of [Anatomical coordinate system](https://www.slicer.org/wiki/Coordinate_systems#Anatomical_coordinate_system) i.e. RAS means is Right-Anterior-Superior ``` ╔════════╦════════════╗ ║ Common ║ Anatomical ║ ╠════════╬════════════╣ ║ Left ║ Left ║ ║ Right ║ Right ║ ║ Up ║ Superior ║ ║ Down ║ Inferior ║ ║ Front ║ Anterior ║ ║ Back ║ Posterior ║ ╚════════╩════════════╝ ``` * `intensity` - `{"min": int, "max": int}` - intensity range. Depends on the device getting the data * `windowWidth` - float - Specify a linear conversion. Window Width contains the width of the window * `windowCenter` - float - Specify a linear conversion. Window Center contains the value that is the center of the window * `channelsCount` - float - channel count of your image data. Default: 1 * `dimensionsIJK` - dict {"x": int, "y": int, "z": int} - dimensions of volume described as vector in [IJK notation](https://en.wikipedia.org/wiki/Unit_vector) * `IJK2WorldMatrix` - matrix to transform coordinates from IJK to world (cartesian). See [here](https://www.slicer.org/wiki/Coordinate_systems#Image_transformation) Grayscale transformations to be applied to Pixel Data are defined by the equivalent of the Modality LUT and Rescale Intercept, Value of Interest Attributes, Photometric Interpretation and the equivalent of the Presentation LUT. `units = m*SV + b` * `rescaleSlope` - float - m in the equation specified by Rescale Intercept * `rescaleIntercept` - float - The value "b" in the relationship between stored values (SV) in Pixel Data and the output units specified in Rescale Type. **`objects` fields description:** * `key` - string - a unique identifier of given object represented as `UUID.hex` value (used in `key_id_map.json` to get the object ID) * `classTitle` - string - the title of a class. It's used to identify the class shape from the `meta.json` file * `tags` - list of strings that will be interpreted as object tags * `labelerLogin` - string - the name of the user that added this figure to the project * `updatedAt` - string - the date and time when the `object` was updated (ISO 8601 format) * `createdAt` - string - the date and time when the `object` was updated (ISO 8601 format) **`planes` fields description:** * `name` - string - the name of the plane, where the figures are placed. Can be [coronal, sagittal or axial](https://www.slicer.org/wiki/Coordinate_systems#Anatomical_coordinate_system) ![Anatomical space](https://docs.supervisely.com/~gitbook/image?url=https:%2F%2F1080806899-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-M4BHwRbuyIoH-xoF3Gv%252Fuploads%252Fgit-blob-b7938ea148776bd2e8500fd57112903723536900%252Fbody_planes.png%3Falt=media\&width=768\&dpr=2\&quality=100\&sign=144e6a783876f39cd31b16a4617757731b20c88e63d1c8dcc82e5bb2348b2cef) `normal` - dict with x, y, z as keys and 0/1 as values - normal is direction by axis, chosen according to plane name ``` * sagittal - x * coronal - y * axial - z The value is binary `(int 0 or 1)` and one plane must be selected. ``` * `slices` - list of slices on the plane. Each list contains index and may contain figures. **`slices` fields description:** * `index` - int value of slice index * `figures` - list of figures placed on a slice. It can be [bitmap](https://docs.supervisely.com/customization-and-integration/00_ann_format_navi/04_supervisely_format_objects#bitmap) or [rectangle](https://docs.supervisely.com/customization-and-integration/00_ann_format_navi/04_supervisely_format_objects#rectangle). **`spatialFigures` fields description** This list contains 3D objects of type [Mask3D](https://docs.supervisely.com/customization-and-integration/00_ann_format_navi/04_supervisely_format_objects#mask3d-3d-annotation) * `key` - string - unique key for a given figure (used in `key_id_map.json`) * `objectKey` - string - unique key to link figure to object (used in `key_id_map.json`) * `geometryType` - `mask_3d` or other 3D geometry-class shape * `geometry` - geometry of the object **`customData` fields description** This field is used to store additional metadata for the figure, such as scores and comments. It is a dictionary with keys representing the plane and slice index, and values containing the metadata. `CustomData` is a optional dictionary with the following structure: ``` { "0-0-1": { // axial plane "51": { // slice index "score": "0.68", // score value "comment": "some comment" // comment text } }, } ``` * `0-0-1` - plane identifier (`0-0-1` for axial, `0-1-0` for coronal, `1-0-0` for sagittal) * `51` - slice index * `score` - score value for the figure (optional) * `comment` - comment text for the figure (optional) For `Mask3D` objects, the `customData` field can contain metadata for multiple planes and slices. The keys are structured as follows: ``` { "0-0-1": { // axial plane "51": { // slice index "score": "0.11", "comment": "some comment" }, "68": { "score": "0.51", "comment": "some comment" } }, "0-1-0": { // coronal plane "51": { "score": "0.97", "comment": "123456" } }, "1-0-0": { // sagittal plane "51": { "comment": "qwerty" } } } ``` To view the scores and comments in the Labeling Toolbox, you need to enable the "Show Figure Score and Comment" option in the toolbox settings. ![settings](https://github.com/supervisely-ecosystem/import-wizard-docs/releases/download/v0.0.3/settings.jpg) After enabling this option and uploading the volumes and annotations with scores, you will see the scores and comments in the Labeling Toolbox. ![toolbox](https://github.com/supervisely-ecosystem/import-wizard-docs/releases/download/v0.0.3/toolbox.jpg) {% hint style="info" %} **Impotant**: You can import and export scores, but you cannot edit them in the Labeling Toolbox. Comments can be edited in the Labeling Toolbox. {% endhint %} ### NRRD files in `mask` folder These files contain geometry for 3D annotation objects, every file name must be the same as figure key to which it belongs. Example: `/project_name/dataset_name/mask/CTChest.nrrd/daff638a423a4bcfa34eb12e42243a87.nrrd` connected with spatial figure `"key": "daff638a423a4bcfa34eb12e42243a87"` Definitions for its fields can be found [here](https://teem.sourceforge.net/nrrd/format.html) ### Key id map file `/project_name/key_id_map.json` file is optional. It is created when annotating the volume inside Supervisely interface and sets the correspondence between the unique identifiers of the object and the volume on which the figure is located. If you annotate manually, you do not need to create this file. This will not affect the work being done. JSON file format of `key_id_map.json`: ```json { "tags": {}, "objects": { "198f727d40c749eebcacc4aed299b39a": 20520 }, "figures": { "65f21690780e43b49863c3cbd07eab3a": 503130811 }, "videos": { "e9f0a3ae21be41d08eec166d454562be": 42656 } } ``` * `objects` - dictionary, where the key is a unique string, generated inside Supervisely environment to set mapping of current object in annotation, and value is unique integer ID related to the current object * `figures` - dictionary, where the key is a unique string, generated inside Supervisely environment to set mapping of object on volume in annotation, and value is unique integer ID related to the current volume * `videos` - dictionary, where the key is unique string, generated inside Supervisely environment to set mapping of volumes in annotation, and value is a unique integer ID related to the current volume * `tags` - dictionary, where the keys are unique strings, generated inside Supervisely environment to set mapping of tag on current volume in annotation, and value is a unique integer ID related to the current tag * **Key** - generated by [python3 function `uuid.uuid4().hex`](https://docs.python.org/3/library/uuid.html#uuid.uuid4). The unique string. All key and ID values should be unique inside single project and can not be shared between entities. * **Value** - returned by server integer identifier while uploading object / figure / volume / tag. ## Useful links * [Supervisely Annotation Format](https://docs.supervisely.com/customization-and-integration/00_ann_format_navi) * [Supervisely Volume Annotation](https://docs.supervisely.com/customization-and-integration/00_ann_format_navi/08_supervisely_format_volume) * [\[SDK CLI\] Upload projects in Supervisely format](https://developer.supervisely.com/getting-started/command-line-interface/sdk-cli#upload-a-project) * [\[CLI Tool Beta\] Upload projects in Supervisely format](https://developer.supervisely.com/getting-started/command-line-interface/cli-tool/workflow-automation#upload-projects-in-supervisely-format) * [\[Supervisely Ecosystem\] Import Volumes in Supervisely format](https://ecosystem.supervisely.com/apps/import-volumes-with-anns) # .NRRD, .DCM volumes ## Overview This option allows you to upload volumes to the platform without any annotations. All volumes from the input directory and its subdirectories will be uploaded to a single dataset. If you need to preserve the directory structure, you can use the [Import DICOM Volumes](https://ecosystem.supervisely.com/apps/import-dicom-volumes) application from the Supervisely Ecosystem. ## Format description **Supported image formats:** `.nrrd`, `.dcm`.\ **With annotations:** No\ **Supported annotation format:** Not applicable.\ **Grouped by:** Any structure (will be uploaded to a single dataset).
## Input files structure {% hint style="success" %} Example data: [download ⬇️](https://github.com/supervisely-ecosystem/import-wizard-docs/files/15025188/sample_volumes.zip)
{% endhint %} Recommended directory structure: ``` 📦folder ┣ 🩻item_01.nrrd ┣ 🩻item_02.nrrd ┣ 🩻item_03.nrrd ┣ 🩻item_04.nrrd ┗ 🩻item_05.nrrd ``` ## Useful links * [\[Supervisely Ecosystem\] Import DICOM Volumes](https://ecosystem.supervisely.com/apps/import-dicom-volumes) # NIfTI ## Overview This converter allows you to import **NIfTI** files into a Supervisely project. It also supports annotations in the NIfTI format (`.nii` and `.nii.gz`). The converter supports both **semantic** and **instance segmentation** annotations, as well as import of volumes with no annotations. We will provide an examples of the input structure below. {% hint style="success" %} The converter is backwards compatible with the [Export volume project to cloud storage ](https://ecosystem.supervisely.com/apps/export-volume-project-to-cloud-storage)application. {% endhint %} All volumes from the input directory and its subdirectories will be uploaded to a single dataset. #### Prepare data for annotations {% hint style="warning" %} **Important: Spatial Alignment for NIfTI Masks** When uploading 3D masks from NIfTI files, it's crucial to ensure proper spatial alignment with the volume. Without this alignment, annotations may not be correctly positioned relative to the volume data, which can cause issues when using the masks outside of Supervisely's labeling tools. {% endhint %} Why this matters: * Supervisely automatically converts volumes to the RAS coordinate system during upload * If masks are uploaded without spatial alignment information, they won't be transformed accordingly * While Supervisely's labeling tools may display them correctly, the underlying data won't match the volume's coordinate space * This misalignment can cause problems when exporting or using annotations in external tools ### Best practices for NIfTI mask upload **Recommended approaches** (choose based on your situation): **Option 1: Attach volume header when you already have mask data** Use this when you already have the mask as a NumPy array (e.g., converted from NIfTI or other format) and need to align it with your volume: ```python import nrrd import supervisely as sly # Read the header from your reference volume header = nrrd.read_header("path/to/your/volume.nrrd") # Create Mask3D geometry with the volume header mask_np = ... # your mask data as NumPy array geometry = sly.Mask3D(mask_np, volume_header=header) ``` **Option 2: Convert NIfTI mask to RAS coordinates and use mask's own header** Use this to convert a NIfTI mask to RAS coordinate system. The function returns the mask data and its header in RAS coordinates. This works when mask dimensions match your volume dimensions: ```python import supervisely as sly # Convert NIfTI mask to RAS coordinate system (NRRD format) # Returns both the converted mask data and header with RAS transformation information mask_np, header = sly.volume.volume.convert_3d_nifti_to_nrrd("path/to/mask.nii") # Create Mask3D using the converted mask data and its own header geometry = sly.Mask3D(mask_np, volume_header=header) ``` Following these practices ensures your annotations maintain correct spatial alignment across all tools and workflows. ## Format description **Supported image formats:** `.nii`, `.nii.gz`.\ **With annotations:** Yes (semantic and instance segmentation).\ **Supported annotation format:** `.nii`, `.nii.gz`.\ **Data structure:** Information is provided below.
## Input files structure #### **Example 1: grouped by volume name** The NIfTI file should be structured as follows: ``` 📂 dataset_name # ⬅︎ may be archive, root files or nested directory instead ├── 📂 CTChest # ⬅︎ the same name as the volume name │ │ # ⬇︎ this directory contains annotations for the CTChest volume │ ├── 🩻 lung.nii.gz │ └── 🩻 tumor.nii.gz ├── 🩻 CTChest.nii.gz └── 🩻 Spine.nii.gz # ⬅︎ this volume has no annotations ``` If the volume has annotations, they should be in the corresponding directory with the same name as the volume (e.g. `CTChest`, without extension). Annotation files should be named according to the following pattern: * Name of the class (e.g. `lung`, `tumor`) + `.nii` or `.nii.gz`.
* The class name should be unique for the current volume (e.g. `tumor.nii.gz`, `lung.nii.gz`). * Annotation files can contain multiple objects of the same class (each object should be represented by a different value in the mask).
#### **Example 2: grouped by plane** The NIfTI file should be structured as follows: **For semantic segmentation:** * The filename must contain one of the required plane identifiers: `axl`, `cor`, or `sag`, anywhere in the name. * The file representing the anatomical volume should have `anatomical` string in it's filename representing the volume type. * The annotation file for all classes should also include the prefix, volume type label (`inference`, `mask`, `ann` etc.), ending with `.nii` or `.nii.gz`. **For instance segmentation:** * The anatomical volume file must include the plane identifier (`axl`, `cor`, or `sag`), `anatomic` type label and end with `.nii` or `.nii.gz`. * Each annotation file must also include the plane identifier and type label (`inference`, `mask`, `ann` etc.), ending with `.nii` or `.nii.gz`. * Multiple annotation files per plane are supported, each representing a separate class (and may contain multiple objects). **Note:** Filenames can include other descriptive parts such as patient or case UIDs, body parts, arbituary strings or other identifiers, as long as the required plane and type identifiers are present and the file extension is `.nii` or `.nii.gz`. The plane identifier must be one of: `cor`, `sag`, or `axl`. The converter uses these prefixes to group volumes and their annotation files, requiring exactly three volumes - one for each prefix per folder. Structure example for semantic segmentation: ``` 📂 dataset_name # ⬅︎ may be archive, root files or nested directory instead ├──📄 cls_color_map.txt # ⬅︎ optional file ├──🩻 axl_anatomic.nii ├──🩻 axl_inference.nii ├──🩻 cor_anatomic.nii ├──🩻 cor_inference.nii ├──🩻 sag_anatomic.nii └──🩻 sag_inference.nii ``` Structure example for instance segmentation: ``` 📂 dataset_name # ⬅︎ may be archive, root files or nested directory instead ├──📄 cls_color_map.txt # ⬅︎ optional file ├──🩻 axl_anatomic.nii ├──🩻 axl_inference_1.nii ├──🩻 axl_inference_2.nii ├──🩻 cor_anatomic.nii ├──🩻 cor_inference_1.nii ├──🩻 cor_inference_2.nii ├──🩻 cor_inference_3.nii ├──🩻 sag_anatomic.nii └──🩻 sag_inference_1.nii ``` #### **Example 3: grouped by plane w/ multiple items** If you need to import multiple items at once, place each item in a separate folder. The converter supports any folder structure. Folders may be at different levels, and files will be matched by directory (annotation files must be in the same folder as their corresponding volume). All files will be imported into the same dataset. Structure example for multiple items directory: ``` 📂 dataset_name # ⬅︎ may be archive, root files or nested directory instead ├──📄 cls_color_map.txt # ⬅︎ optional file ├──📂 item_1 │ ├──🩻 axl_anatomic.nii │ ├──🩻 axl_inference_1.nii │ ├──🩻 axl_inference_2.nii │ ├──🩻 cor_anatomic.nii │ ├──🩻 cor_inference_1.nii │ ├──🩻 cor_inference_3.nii │ └──🩻 sag_anatomic.nii ├──📂 item_2 │ ├──🩻 axl_anatomic.nii │ ├──🩻 axl_inference_1.nii │ ├──🩻 axl_inference_2.nii │ ├──🩻 cor_anatomic.nii │ ├──🩻 cor_inference_1.nii │ ├──🩻 cor_inference_3.nii │ └──🩻 sag_anatomic.nii ├──📂 item_2 │ ├──🩻 axl_anatomic.nii │ ├──🩻 axl_inference_1.nii │ ├──🩻 axl_inference_2.nii │ ├──🩻 cor_anatomic.nii │ ├──🩻 cor_inference_1.nii │ ├──🩻 cor_inference_3.nii │ ├──🩻 sag_anatomic.nii └──└──🩻 sag_inference_1.nii ``` #### Example 4: Upload with scores and comments metadata Starting from SDK version v6.73.394 and instance version v6.13.8, the converter supports uploading NIfTI files with additional metadata – scores. To upload this metadata, you need to create corresponding `CSV` files for each volume-annotation pair. Make sure that the CSV file name contains `score` (instead of `anatomic` or `inference`) and has the same prefix as the NIfTI file. Structure example for uploading with scores and comments: ``` 📂 dataset_name # ⬅︎ may be archive, root files or nested directory instead ├──🩻 axl_anatomic.nii ├──🩻 axl_inference.nii ├──🩻 cor_anatomic.nii ├──🩻 cor_inference.nii ├──🩻 sag_anatomic.nii ├──📄 axl_score.csv ├──📄 cor_score.csv └──📄 sag_score.csv ``` Where the `CSV` files should be structured as follows: ```csv Layer, Label-2, Label-4, ... 3, 0.8, 0.9, ... 7, 0.7, 0.6, ... 8, 0.4, 0.3, ... ``` ![csv\_example](https://github.com/supervisely-ecosystem/import-wizard-docs/releases/download/v0.0.3/csv_example.jpg) where: * **Layer**: The frame number in the NIfTI file (starting from 1). * **Label-2, Label-4, ...**: Corresponding labels for the NIfTI file, which contains the `Label-` prefix with the corresponding pixel value in the NIfTI file. To view the scores and comments in the Labeling Toolbox, you need to enable the "Show Figure Score and Comment" option in the toolbox settings. ![settings](https://github.com/supervisely-ecosystem/import-wizard-docs/releases/download/v0.0.3/settings.jpg) After enabling this option and uploading the NIfTI files with scores, you will see the scores and comments in the Labeling Toolbox. ![toolbox](https://github.com/supervisely-ecosystem/import-wizard-docs/releases/download/v0.0.3/toolbox.jpg) {% hint style="info" %} **Impotant**: You can import and export scores, but you cannot edit them in the Labeling Toolbox. Comments can be edited in the Labeling Toolbox, but they will not be saved back to the CSV files. {% endhint %} #### Class color map file (optional) The converter will look for an optional `TXT` file in the input directory. If present, it will be used to create the classes with names and colors corresponding to the pixel values in the NIfTI files. The TXT file should be structured as follows: ```txt 1 Femur 255 0 0 2 Femoral cartilage 0 255 0 3 Tibia 0 0 255 4 Tibia cartilage 255 255 0 5 Patella 0 255 255 6 Patellar cartilage 255 0 255 7 Miniscus 175 175 175 ``` where: * 1, 2, ... are the pixel values in the NIfTI files * Femur, Femoral cartilage, ... are the names of the classes * 255, 0, 0, ... are the RGB colors of the classes ## Upload annotations separately Plane-structured converter supports uploading annotations separately (uploading annotations to existing volumes). This functionality supports both dataset-scope and project-wide annotation imports. By default, annotations are matched with their corresponding volumes based on filenames. However, a custom mapping can be provided via a `.json` file to explicitly define the mapping. Input structure example for dataset scope: ``` 🩻 axl_inference_1.nii 🩻 axl_inference_2.nii 🩻 cor_inference_1.nii 🩻 cor_inference_3.nii 📄 color_map.txt # ⬅︎ optional file 📄 mapping.json # ⬅︎ optional file ``` Input structure example for project-wide import: ``` 📄 mapping.json # ⬅︎ optional file 📄 cls_color_map.txt # ⬅︎ optional file 📂 dataset_name_1 ├──🩻 axl_inference_1.nii ├──🩻 axl_inference_2.nii └──🩻 cor_inference_3.nii 📂 dataset_name_2 ├──🩻 axl_inference_1.nii ├──🩻 axl_inference_2.nii └──🩻 cor_inference_3.nii 📂 dataset_name_3 ├──🩻 axl_inference_1.nii ├──🩻 axl_inference_2.nii └──🩻 cor_inference_3.nii ``` #### JSON mapping Mapping structure should be as follows: ``` { "cor_inference_1.nii": 123, "sag_mask_2.nii": 456 } ``` Where key should be annotation filename, and volume ID as value If you want to import annotations for the entire project via a JSON mapping: 1. Pack annotations inside folders with corresponding dataset name as in an example above 2. Specify the dataset name in a `.json` file in a path-like manner (`dataset_name/annotation_filename`) Example JSON structure with dataset specification: ``` { "dataset1/cor_inference_1.nii": 123, "dataset2/sag_mask_2.nii": 456 } ``` ## Useful links * [\[Supervisely Ecosystem\] Export volume project to cloud storage](https://ecosystem.supervisely.com/apps/export-volume-project-to-cloud-storage) # Meshes # Supervisely ## Overview {% hint style="success" %} Easily import your meshes with annotations in the Supervisely format. Labels are stored in a per-mesh `annotation.json` file, and label geometries (vertex index sets) are stored as binary `.bin` files for efficiency. {% endhint %} {% hint style="info" %} All information about the Supervisely Meshes annotation format can be found [here](/customization-and-integration/00_ann_format_navi/09_supervisely_format_mesh) {% endhint %} ## Format description **Supported mesh formats:** `.ply`, `.stl`, `.obj`\ **With annotations:** yes\ **Supported annotation format:** `.json` + `.bin` geometry files.\ **Data structure:** Information is provided below. ## Input files structure Both directory and archive are supported. Each dataset is a directory with `meshes` and `annotations` subdirectories. Nested datasets are stored in a `datasets` subdirectory of the parent dataset. **Recommended directory structure:** ``` 📦 project_name ├── 📂 dataset_name │ ├── 📂 meshes │ │ ├── 📄 mesh_01.ply │ │ └── 📄 mesh_02.ply │ ├── 📂 annotations │ │ ├── 📂 mesh_01.ply │ │ │ ├── 📄 annotation.json │ │ │ └── 📂 geometries │ │ │ ├── 📄 {label_key}.indices.bin │ │ │ └── 📄 {label_key}.indices.bin │ │ └── 📂 mesh_02.ply │ │ ├── 📄 annotation.json │ │ └── 📂 geometries │ │ └── 📄 {label_key}.indices.bin │ └── 📂 datasets │ └── 📂 nested_dataset_name │ ├── 📂 meshes │ └── 📂 annotations └── 📄 meta.json ``` Project meta file `meta.json` contains classes and tags definitions. Learn more about the `meta.json` file [here](/customization-and-integration/00_ann_format_navi/02_project_classes_and_tags). ## annotation.json Each mesh has a corresponding `annotation.json` describing its labels. ```json { "key": "b4a3dc33f8d842a79b24942f85f3f2ee", "meshId": 6228355, "tags": [ { "name": "confirmed", "tagId": 43398, "value": 1, "id": 1678691 } ], "labels": [ { "key": "6e474a08a13f46bcb4c8ca538c760edb", "id": 25782697, "classTitle": "scratch", "tags": [], "geometryType": "mesh", "geometry": { "indices": null, "indicesPath": "geometries/6e474a08a13f46bcb4c8ca538c760edb.indices.bin" }, "priority": 1, "customData": {} } ] } ``` **Fields definitions:** * `key` — unique annotation key * `tags` — entity-level tags of the mesh * `labels` — list of labeled objects; each label has a `classTitle` and a unique `key`, and stores vertex indices via `indicesPath` * `indicesPath` — path to the `.bin` file containing the vertex index set for this label (little-endian uint32), relative to the annotation directory of the item * `meshId`, `id`, `tagId`, `priority`, `customData` and other server-side metadata fields are written on export and are optional on import — they are not required to create the annotations # Per-Vertex Annotation ## Overview {% hint style="success" %} Import `.ply` mesh files with per-vertex annotations embedded directly in the file. Each annotated vertex is painted with the color of its class (vertex colors are matched against class colors from `meta.json`), and an `object_id` groups vertices into object instances. {% endhint %} This is useful when annotations are produced by external pipelines (e.g. 3D segmentation models) that write labels directly into PLY vertex attributes. ## Format description **Supported mesh formats:** `.ply` (ASCII only)\ **With annotations:** yes\ **Supported annotation format:** Per-vertex PLY properties + `meta.json`.\ **Data structure:** Information is provided below. ## Input files structure Both directory and archive are supported. Datasets may be nested; the directory hierarchy is preserved as a nested dataset hierarchy. Mesh files placed directly next to `meta.json` are imported into a default dataset. **Recommended directory structure:** ``` 📦 project name ├── 📂 dataset_name │ ├── 📄 mesh_01.ply │ ├── 📄 mesh_02.ply │ └── 📂 nested_dataset_name │ └── 📄 mesh_03.ply └── 📄 meta.json ``` ## PLY File Requirements The `.ply` file must be in **ASCII** format (`format ascii 1.0`; binary PLY is not supported) and must contain per-vertex color properties (`red`, `green`, `blue` or `diffuse_red`, `diffuse_green`, `diffuse_blue`) **and** the `class_id`/`object_id` properties in addition to the standard geometry properties. Files without `class_id` and `object_id` are not recognized as this format. ``` property uchar red property uchar green property uchar blue property int class_id property int object_id ``` * **Vertex colors** define the class: a vertex whose color exactly matches the color of a class from `meta.json` is annotated with that class. * **`class_id`** is the annotation marker: a vertex with `class_id = -1` is never imported as annotated, even if its color matches a class. This allows background vertices to coexist with a class of the same color (e.g. white). * **`object_id`** carries instance segmentation: vertices sharing the same `object_id` are grouped into a single object instance. Vertices with `object_id = -1` are imported as semantic (non-instance) annotation of their class. **Example PLY header:** ``` ply format ascii 1.0 element vertex 166428 property float x property float y property float z property uchar red property uchar green property uchar blue property uchar alpha property int class_id property int object_id element face 327618 property list uchar int vertex_indices end_header ``` **Vertex value conventions:** | Value | Meaning | | ---------------------------------------- | --------------------------------------------- | | color matches a class + `class_id != -1` | Vertex is annotated with that class | | `class_id = -1` | Vertex is not annotated, regardless of color | | color does not match any class | Vertex is not annotated (background) | | `object_id = -1` | Vertex does not belong to any object instance | | `object_id >= 0` | Unique object (instance) ID within the mesh | {% hint style="info" %} White (`255 255 255`) is the neutral color written by the export for unannotated vertices of colorless meshes. Thanks to the `class_id`/`object_id` markers it can also be used as a class color. {% endhint %} ## meta.json The `meta.json` file defines the classes. Vertex colors in the `.ply` files are matched against the `color` field of each class, so **class colors must be unique**. Classes must have shape `mesh` or `any`, and `projectType` must be `meshes`. The file follows the standard Supervisely project meta format. **Example `meta.json`:** ```json { "classes": [ { "title": "dot", "description": "", "shape": "mesh", "color": "#FF0000", "geometry_config": {}, "id": 197748, "hotkey": "" }, { "title": "scratch", "description": "", "shape": "any", "color": "#6200FF", "geometry_config": {}, "id": 197761, "hotkey": "" } ], "tags": [ { "name": "significant", "value_type": "none", "color": "#FFC705", "id": 36942, "hotkey": "", "applicable_type": "all", "classes": [], "target_type": "all" } ], "projectType": "meshes" } ``` At least one mesh in the project must contain annotated vertices (colors matching a class), otherwise the format will not be detected. ## Semantic vs Instance Segmentation The `object_id` value controls how annotated vertices are grouped into objects. Vertices are grouped by the `(class, object_id)` pair: **Instance segmentation** — give each object its own `object_id` (`>= 0`). Vertices sharing the same `object_id` (and the same class color) are imported as one object instance. This also allows multiple disconnected regions of the mesh to be grouped into a single object: if three separate mesh patches all have the color of class `scratch` and `object_id = 42`, they become a single object of class `scratch`. ``` # two separate scratch instances x y z 42 x y z 42 x y z 43 x y z 43 ``` **Semantic segmentation** — set `object_id = -1` for annotated vertices. All vertices of the same class are then merged into a single object per class, even if they form disconnected regions: ``` # one merged "scratch" object, no instances x y z -1 x y z -1 x y z -1 ``` Both modes can be mixed in one file: vertices of a class with `object_id = -1` form one semantic object, while vertices with explicit IDs form separate instances of that class. {% hint style="warning" %} An `object_id` is expected to belong to a single class — an object instance cannot span two classes. If the same `object_id` does appear with two different class colors, the import will not fail: the vertices are split into separate objects, one per class. {% endhint %} ## Mesh Cleanup on Import The colors and `class_id`/`object_id` values baked into the `.ply` files are only a transport for the annotations. During import they are extracted and converted into regular, editable annotation objects — the same objects you get when labeling in the Mesh Labeling Toolbox. The mesh file itself is stored in a cleaned-up form: * the `class_id` and `object_id` properties are removed; * previously annotated vertices are repainted with neutral white — the original color underneath is unknown, because the export overwrote it with the class color; * vertices that were not annotated keep their original colors; * if every vertex carried label paint, the color properties are removed from the file entirely. This keeps the mesh looking clean in the labeling tool: annotations are displayed as an editable overlay on top of the mesh instead of colors permanently painted into the file, and exporting and re-importing the project gives consistent results on every round trip. # Import sample dataset Save valuable time by starting with already prepared datasets. We provide access to a variety of ready-made data to speed up your start. Just go to the [Ecosystem](https://ecosystem.supervisely.com/), find the **Import** section, select the modality (images, videos, etc) and then click **Demo projects**. Click the dataset you like and finally click `Get project`.
## [**Dataset Ninja**](https://datasetninja.com/) Or you can take advantage of [DatasetNinja](https://datasetninja.com/) new initiative, an easy-to-use service for searching and exploring Computer Vision datasets. And it is available to the entire machine learning community completely free of charge.
Find any dataset you like and click “Train in Supervisely” button to add it to your list of projects.
# Import into an existing dataset It is possible to add more assets such as images to the existing project or dataset. Just go to the **Projects** or **Datasets** page and click on interactive tile.
You can also import data into an existing project or dataset as follows: {% hint style="info" %} Depending on the [Supervisely App](https://app.supervisely.com/ecosystem/all-apps), some of them (not every application!) support additional context, such as Input Project or Input Dataset. You can look for it in the Run Application modal window, but there is a more simple way. {% endhint %} ## Example: add more images to a dataset ### Method 1: Loading via the dataset context menu and application Go to the **Projects** or **Data** page, select any dataset and use the context menu by **three dots icon (⋮)** and select Add more items → Import. You find the list of [Supervisely Apps](https://app.supervisely.com/ecosystem/all-apps) that support project as an input and allow adding more content to it.
### Method 2: Loading via the dataset page * Enter the required dataset and click "import more data" ①
* A page will open showing an indicator that you are adding data to the selected dataset ②, and its name cannot be changed (in your screenshot, you are adding data as a new dataset). * On the upload page, you can review the required format by selecting it ③.
* On the page with the format description, you can find example data ④, download it and compare the structure. You can read more about the structure on the page or via the provided link. * Additionally, this page features a simplified example of the Supervisely format structure ⑤. Please double-check the data you are uploading.
* The screenshots (⑥ and ⑦ points) show that the images from the example data have been uploaded.
# Import using Team Files In fact, when you drag and drop your files into the upload window, we upload them to an automatically created folder in Team Files, launch the application of choice, passing in the path to that folder, and then automatically delete it. Sometimes, you have too many files to upload via drag-and-drop. Or, you may want to upload files via the API first, and then run a Supervisely App on those files. Or, maybe, you launched a Supervisely App that generated a new project, saved it to your [team files](/data-organization/team-files), but those files are not a project yet. In that case, you can just select the appropriate Supervisely App from the context menu of a folder in your [Team Files](/data-organization/team-files) - and enjoy. ![](/files/AwkKbc5xKStrOatuLrZM) # Import from Cloud It's also worth mentioning that we have applications to import data not from your computer, but from cloud services. No matter which method you choose, you have the option to import files by links, meaning they won't be stored directly on the Supervisely disk. These files will only be accessible through the provided links. This method offers flexibility in data management, and you can still use both this approach and the traditional file import for the convenience of your workflows. ## Connect Cloud Storage for a Team {% hint style="success" %} Available by default on **Enterprise Edition**. On **Community Edition**, it requires a **Pro** subscription and is enabled upon request — contact Supervisely support to turn it on for your team. {% endhint %} Any team can connect its own cloud storage account (AWS S3, Google Cloud Storage, Azure Blob Storage, or any S3-compatible storage) directly to Supervisely. Once connected, team members can browse the storage and import images, videos, or entire annotated projects straight from the bucket — without downloading the data to their computer first and, optionally, without duplicating it inside Supervisely at all. This is the recommended way to import from cloud storage: you set up the connection once, then reuse it for any future import. The apps described further down on this page are an older alternative — you provide credentials each time you run them, without saving a reusable connection. {% hint style="info" %} This is a **Team**-level connection: it is visible and usable only within the team that created it. It's a separate feature from the [instance-wide Remote Storage](/enterprise-edition/advanced-tuning/s3) that an Enterprise admin can configure as the platform's primary data backend. {% endhint %} ### Open Remote Storages 1. In the left sidebar, click on your current team name (bottom-left corner). 2. In the menu that opens, under **Current team**, click **Remote Storages**. This opens the **Remote Storages** page, listing every cloud storage connected to the instance. Entries you add here are marked with the **Team** scope; any entries configured by an instance administrator are marked **Global**. {% hint style="warning" %} Only a team **Admin** can open the Remote Storages page and add, edit, test, or remove connections. See [team roles](/collaboration/members) for the full list of permissions per role. {% endhint %}
In the **Buckets** column, each bucket name is followed by a unique identifier in parentheses, e.g. `test-bucket (test-bucket-2ee7eps5)`. Supervisely generates this identifier automatically so that buckets sharing the same name — across different connections or providers — can still be told apart. ### Add a new connection 1. On the **Remote Storages** page, click **+ ADD** in the top-right corner. 2. In the **Add new cloud provider** dialog, select a **Cloud Provider**: **AWS S3**, **Google Cloud Storage**, or **Azure Storage**. 3. Fill in the fields for the selected provider (see below). 4. Optionally restrict the connection to specific **Buckets** and/or specific **Users**. 5. Click **ADD**. The new connection appears in the table with scope **Team**. #### AWS S3 | Field | Description | | ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Endpoint | **Auto** uses `s3.amazonaws.com`. Switch to **Manual** to point at an S3-compatible endpoint, e.g. a MinIO server or another on-prem S3 (`http://:`). | | Region | Optional, e.g. `eu-central-1`. Leave empty for S3-compatible storages that don't use regions. | | Access key / Secret key | Standard AWS credentials (**Keys** tab). Both are required. | | IAM Anywhere | Alternative to static keys (**IAM Anywhere** tab): `Role Arn`, `Profile Arn`, `Trust Anchor Arn`, a base64-encoded PEM `Signing certificate`, and a base64-encoded PEM `Sign private key`. Use this to authenticate without long-lived access keys. See [Keys from IAM Role](/enterprise-edition/advanced-tuning/s3#keys-from-iam-role) for the full setup (generating certificates, creating a trust anchor, role, and profile in AWS). | #### Google Cloud Storage | Field | Description | | ---------------- | -------------------------------------------------------------------------------------- | | Endpoint | **Auto** uses `storage.googleapis.com`, or switch to **Manual** for a custom endpoint. | | Credentials file | Drag and drop (or select) the service account JSON key file generated in Google Cloud. | #### Azure Storage | Field | Description | | ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | | Storage account name | Your Azure storage account name. | | Authentication | Choose **Access key or SAS token** for shared-key authentication, or **Entra ID** for a Microsoft Entra service principal. | | Secret key or SAS token | With **Access key or SAS token**, enter either the account access key or a SAS token. | | Tenant ID | With **Entra ID**, enter the Microsoft Entra tenant (directory) ID. | | Client ID | With **Entra ID**, enter the service principal's application (client) ID. | | Client secret | With **Entra ID**, enter the secret value created for the service principal. Do not enter the secret ID. | | Endpoint | **Auto** derives the endpoint from the account name, or switch to **Manual** to set a custom one (e.g. Azurite or another Azure-compatible endpoint). | **Authenticate with Microsoft Entra ID** Entra ID authentication lets an Azure administrator disable storage account key access without interrupting Supervisely. Supervisely authenticates as a service principal, so the credentials remain separate from an individual user's Azure account. 1. Sign in to Azure CLI and select the subscription that contains the storage account: ```bash az login az account set --subscription ``` 2. Create a service principal and grant it the **Storage Blob Data Contributor** role at the storage account scope: ```bash az ad sp create-for-rbac \ --name supervisely-remote-storage \ --role "Storage Blob Data Contributor" \ --scopes /subscriptions//resourceGroups//providers/Microsoft.Storage/storageAccounts/ ``` This role lets Supervisely list containers and read, create, update, and delete blobs. Azure permissions can take several minutes to propagate after the role assignment. 3. Store the command output in your secret manager. Map `tenant` to **Tenant ID**, `appId` to **Client ID**, and `password` to **Client secret**. Azure displays the client secret value only when it is created. 4. In Supervisely, set **Storage account name**, select **Entra ID** under **Authentication**, fill in all three Entra fields, and click **ADD**. Use **Test** from the connection's **⋮** menu to verify access. {% hint style="warning" %} Treat the client secret like a password. Do not paste it into tickets, logs, or source control; rotate it in Microsoft Entra ID before it expires. {% endhint %} #### Restricting buckets and users * **Buckets** — if the provided credentials only grant access to specific buckets/containers, list them here, one per line. You can optionally scope a bucket to a prefix by adding it after a slash, and separate multiple prefixes with a colon: ``` bucket1 bucket2:prefix1 bucket3:prefix1/abc:prefix2:prefix3/123 ``` Leave empty to let Supervisely discover all buckets the credentials have access to. * **Users** — by default, **All users** in the team can use this connection to import data. Pick specific team members instead to limit who can use it for imports. This only restricts the *import* feature — it does not affect who can view or edit a project once the data is imported; that's still governed by each member's [team role](/collaboration/members). ### Verify, edit, or remove a connection Click the **⋮** menu at the end of a connection's row: * **Test** — pick one of the available buckets and check that Supervisely can connect to it with the saved credentials. * **Edit** — update endpoint, credentials, buckets, or user restrictions. * **Remove** — delete the connection. This does not delete any data already imported into Supervisely, only the stored connection/credentials. ### Import data from the connected storage Once a cloud storage is connected, it becomes available as a source in the import wizard: 1. Open a project (or start creating a new one) and go to the **Data** tab. 2. Click **Add** → **Import data**. 3. Choose the **Cloud Storage** import method. 4. Browse **Cloud Storages** → the provider you connected → the bucket → the folder/files you need, and check the ones to import. Buckets are listed here by their unique identifier (the same one shown in parentheses on the Remote Storages page) rather than the plain bucket name, so buckets with identical names don't get mixed up.
5. Optionally check **Import as Links** to keep the files in the cloud and reference them by link, instead of copying them into Supervisely storage — useful for very large datasets. 6. Click **Run**. Supervisely detects the data and annotation format automatically (images, videos, point clouds, COCO, Pascal VOC, Supervisely format, and more) and imports it into the selected dataset. ## Import images without labels from S3, Google Cloud, Azure and others [This apps](https://ecosystem.supervisely.com/apps/import-images-from-cloud-storage) allows to import images from most popular cloud storage providers to Supervisely Private instance. **List of providers:** * Amazon S3 * Google Cloud Storage (GCS) * Microsoft Azure * and others with S3 compatible interfaces **App supports two types of import:** * copy images from cloud to Supervisely Storage * add images by link {% embed url="" %} ## Import images **with** labels from S3, Google Cloud, Azure and others [This app](https://ecosystem.supervisely.com/apps/import-images-in-sly-format-from-cloud-storage) enables the straightforward import of images with associated annotations from various cloud storage services like S3, Google Cloud, Azure, and others. It provides a convenient way to handle annotated images, facilitating effective data management for various purposes. You can learn about Supervisely format [here](/customization-and-integration/00_ann_format_navi/01_project_structure_new). ## Import videos, point clouds and other from S3, Google Cloud, Azure [This apps](https://ecosystem.supervisely.com/apps/import-videos-from-cloud-storage) allows to import videos from most popular cloud storage providers to Supervisely Private instance. List of providers: * Amazon S3 * Google Cloud Storage (GCS) * Microsoft Azure * and others with S3 compatible interfaces **App supports two types of import:** 1. Copy videos from cloud to Supervisely Storage (pros: fast video streaming, cons: data is duplicated) 2. Add videos by link (pros: data will not be duplicated, cons: video streaming lags are possible - it depends on cloud configuration) ## Import images from disk without copying You also have the opportunity to use applications [Import images from cloud storage](https://ecosystem.supervisely.com/apps/import-images-from-cloud-storage) and [Import image projects in Supervisely format from cloud storage](https://ecosystem.supervisely.com/apps/import-images-in-sly-format-from-cloud-storage) with the FileSystem serving as the data provider. This provides the capability to import images directly from the disk without copying, offering convenience and efficiency when working with local data. ## Apps for importing public data * [Pexels downloader](https://ecosystem.supervisely.com/apps/pexels-downloader) * [Flickr downloader](https://ecosystem.supervisely.com/apps/flickr-downloader) * [Import Cityscapes](https://ecosystem.supervisely.com/apps/import-cityscapes) ## **Import in Supervisely Format from a Web-server** [Remote import](https://ecosystem.supervisely.com/apps/remote-import) allows you connect your remote data storage to Supervisely Platform without data duplication. Most frequent use case is when Enterprise Customer would like to connect huge existing data storage (tens of terabytes) and avoid data duplication. In other cases we recommend to use general import procedure to store data in [Supervisely Data Storage](https://github.com/supervisely/docs/blob/master/data-organization/import/import/storage/README.md) ![Remote Import APP](/files/j3rXb9R490Wm0UkRVpZB) # Import using API & SDK Our software provides you with a Software Development Kit (SDK), which equips you with tools and resources to optimize data import processes. With the SDK, you can create custom solutions that best suit your needs. ## **How to Use the Software Development Kit (SDK):** 1. Access the SDK for our system and its [documentation](https://supervisely.readthedocs.io/en/latest/sdk_packages.html). 2. Utilize the SDK to create custom solutions that align with your specific needs. 3. Integrate customized applications and solutions into your system to optimize data import. By using the Software Development Kit (SDK), you gain a powerful tool for optimizing data import and creating custom solutions that align with your unique needs and goals. This makes the data management process more efficient and effective. ## **How to Import Through APIs:** Our system offers the flexibility to integrate data from various sources using Application Programming Interfaces (APIs). This powerful feature enables seamless data transfer and management, connecting our platform with external systems or services. 1. Access the [API documentation](https://api.docs.supervisely.com/) for our platform to understand the available endpoints and data import methods. 2. Configure your data source to connect with our platform's API. 3. Use API requests to initiate data transfer, specifying the data you want to import and its destination within our platform. Importing data through APIs allows for seamless and automated data transfer, making it an ideal choice for integrating external data sources with our system. It empowers you to have more control over your data management and ensures data consistency and efficiency. # Import using agent We have several ways to import data from the agent. Find out which of these methods is best for you. ## **Agent host directory** A temporary folder in which internal files necessary for the operation of the agent will be stored. Basically these are some kind of caches and other temporary application files. Through this folder you can run [import applications](https://dev.supervise.ly/ecosystem/import). You can safely clear this folder at any time, unless you have no applications running or, for example, one of your training jobs failed and you want to restore missing checkpoints.
## **Folder to mount** Is a permanent folder you can use to hold files and folders you wish to give applications access to. In some cases, you can provide a mounted folder with your images and use the context menu in the [team files](/data-organization/team-files) to initiate the import process. ## **Another way** But, it turned out we have an even better way to do this without the use of agents or the team files. You can actually add any folder on your server as a remote storage at the Instance Settings page using the “filesystem” provider, then use any [Supervisely App](https://ecosystem.supervisely.com/import), such as [Import images from cloud storage](https://ecosystem.supervisely.com/apps/import-images-from-cloud-storage) or [Import image projects in Supervisely format from cloud storage](https://ecosystem.supervisely.com/apps/import-images-in-sly-format-from-cloud-storage) and select your filesystem remote - this gives you are the flexibilities, plus, you don't have to actually copy image files to the Supervisely storage - files will be added to your [datasets](broken://pages/-M54fC5lCGh1KuU-VTCW) “by links”. # Migrations Ready to move to Supervisely? Explore our migrations tools for a smooth transition. Supervisely provides user-friendly tools to easily move your projects and data. Whether it's annotations, datasets, or entire projects, we've made the process straightforward. ## **Features:** **Easy Import**: Our platform supports importing data from various sources. Whether you're coming from another annotation tool or computer vision platform, Supervisely makes it simple. **Data Alignment**: Intuitive tools help match your existing data structure with Supervisely's setup, ensuring an organized project transition. ## **Supported Platforms for Migration to Supervisely** * [Labelbox](/import-and-export/migrations/migration-labelbox) * [Roboflow](/import-and-export/migrations/migration-roboflow) * [CVAT](/import-and-export/migrations/migration-cvat) * [V7](/import-and-export/migrations/migration-v7)
Roboflow to SuperviselyThis application allows you to copy multiple projects from Roboflow instance to Supervisely instance./pages/wrUn4p6ABLTTjg0F77oQ
Labelbox to SuperviselyThis application allows you to copy multiple projects from Labelbox instance to Supervisely instance./pages/qnkVB567EnGeY4WZEbwu
V7 to SuperviselyThis application allows you to copy multiple datasets from V7 instance to Supervisely instance./pages/wrUn4p6ABLTTjg0F77oQ
CVAT to SuperviselyThis application allows you to copy multiple projects from CVAT instance to Supervisely instance./pages/pEoQNmLnVXQfpYnqPGQb
# Roboflow to Supervisely Convert and copy multiple Roboflow projects into Supervisely at once. [This application](https://ecosystem.supervisely.com/apps/roboflow-to-sly) allows you to copy multiple projects from Roboflow instance to Supervisely instance, you can select which projects should be copied, labels and tags will be converted automatically. You can preview the results in the table, which will show URLs to corresponding projects in Roboflow and Supervisely. {% hint style="info" %} [Complete migration guide](https://ecosystem.supervisely.com/apps/roboflow-to-sly) from Roboflow to Supervisely. {% endhint %} ## Preparation ℹ️ NOTE: Every project in Roboflow MUST contain at least one version, otherwise it's impossible to export data from Roboflow. Learn more about Versions in [Roboflow documentation.](https://docs.roboflow.com/datasets/create-a-dataset-version) In order to run the app, you need to obtain `Private API key` to work with Roboflow API. You can refer to [this documentation](https://docs.roboflow.com/api-reference/authentication) to do it. Now you have two options to use your API key: you can use team files to store an .env file with or you can enter the API key directly in the app GUI. Using team files is recommended as it is more convenient and faster, but you can choose the option that is more suitable for you. ### Using team files You can download an example of the .env file [here](https://github.com/supervisely-ecosystem/roboflow-to-sly/files/13214150/roboflow.env.zip) and edit it without any additional software in any text editor. NOTE: you need to unzip the file before using it. 1. Create a .env file with the following content: `ROBOFLOW_API_KEY="qASymt32UTnQV1qABszF"` 2. Upload the .env file to the team files. 3. Right-click on the .env file, select Run app and choose the Roboflow to Supervisely Migration Tool app. The app will be launched with the API key from the .env file and you won't need to enter it manually. If everything was done correctly, you will see the following message in the app UI: * ℹ️ Connection settings was loaded from .env file. * ✅ Successfully connected to . ### Entering credentials manually 1. Launch the app from the Ecosystem. 2. Enter the API key. 3. Press the Connect to Roboflow button. ![](/files/25npsKqT6jx435sfMmEK) If everything was done correctly, you will see the following message in the app UI: * ✅ Successfully connected to . NOTE: The app will not save your API key, you will need to enter it every time you launch the app. To save your time you can use the team files to store your credentials. # Labelbox to Supervisely [This application](https://ecosystem.supervisely.com/apps/labelbox-to-sly) allows you to copy multiple projects from Labelbox instance to Supervisely instance, you can select which projects should be copied. You can preview the results in the table, which will show URLs to corresponding projects in Labelbox and Supervisely. {% hint style="info" %} [Complete migration guide](https://ecosystem.supervisely.com/apps/labelbox-to-sly) from Labelbox to Supervisely. {% endhint %} ## Preparation {% hint style="info" %} **ℹ️ NOTE:** There are some limitations on the Labelbox side depending on your subscription plan. You can find more information about it [here](https://docs.labelbox.com/docs/limits). {% endhint %} {% hint style="info" %} **ℹ️ NOTE:** It is allowed to export only Images or Videos projects from Labelbox, so you need to make sure that your projects meet this requirement, otherwise it's impossible to export data from Labelbox. {% endhint %} In order to run the app, you need to obtain Private API key to work with Labelbox API. You can refer to [this documentation](https://docs.labelbox.com/reference/create-api-key) to do it. Now you have two options to use your API key: you can use team files to store an .env file with API key or you can enter the API key directly in the app GUI. Using team files is recommended as it is more convenient and faster, but you can choose the option that is more suitable for you. ### Using team files You can download an example of the .env file [here](https://github.com/supervisely-ecosystem/labelbox-to-sly/files/13227776/labelbox.env.zip) and edit it without any additional software in any text editor. ℹ️ NOTE: you need to unzip the file before using it. 1. Create a .env file with the following content: `LB_API_KEY=` 2. Upload the .env file to the team files. 3. Right-click on the .env file, select Run app and choose the **Labelbox to Supervisely Migration Tool app**. The app will be launched with the API key from the .env file and you won't need to enter it manually. If everything was done correctly, you will see the following message in the app UI: * ℹ️ Connection settings was loaded from .env file. * ✅ Successfully connected to . ## Entering credentials manually 1. Launch the app from the Ecosystem. 2. Enter the API key. 3. Press the Connect to Labelbox button. ![](/files/96LuPdSJiAeVOFkpdkwL) ![](/files/30vQntLGnKvuwQfCDjAv) If everything was done correctly, you will see the following message in the app UI: * ✅ Successfully connected to . **ℹ️ NOTE:** The app will not save your API key, you will need to enter it every time you launch the app. To save your time you can use the team files to store your credentials. You can see all the functionality of our migration application [here](https://ecosystem.supervisely.com/apps/labelbox-to-sly) # V7 to Supervisely Convert and copy multiple V7 datasets into Supervisely at once. [This application](https://ecosystem.supervisely.com/apps/v7-to-supervisely/migration_tool) allows you to copy multiple datasets from V7 instance to Supervisely instance, you can select which projects should be copied, labels and tags will be converted automatically. You can preview the results in the table, which will show URLs to corresdponding projects in V7 and Supervisely. Every V7 dataset will be converted in a separate Supervisely project. {% hint style="info" %} If you want to upload data, which was already exported from V7 instance, you can use this [Import V7](https://ecosystem.supervisely.com/apps/v7-to-supervisely/import_v7) app from Supervisely Ecosystem. {% endhint %} {% hint style="info" %} [Complete migration guide](https://ecosystem.supervisely.com/apps/v7-to-supervisely/migration_tool) from V7 to Supervisely. {% endhint %} ## Preparation In order to run the app, you need to obtain API key to work with V7 API. You can refer to [this documentation](https://docs.v7labs.com/reference/introduction#generating-an-api-key) to do it. Now you have two options to use your API key: you can use team files to store an .env file with or you can enter the API key directly in the app GUI. Using team files is recommended as it is more convenient and faster, but you can choose the option that is more suitable for you. ### Using team files You can download an example of the .env file [here](https://github.com/supervisely-ecosystem/v7-to-supervisely/files/13297606/v7.env.zip) and edit it without any additional software in any text editor. NOTE: you need to unzip the file before using it. 1. Create a .env file with the following content: V7\_API\_KEY="JUFxKUH.wfjargM-xZ3-K2wR2kkZaxFM-AqTiZWs" 2. Upload the .env file to the team files. 3. Right-click on the .env file, select Run app and choose the V7 to Supervisely Migration Tool app. The app will be launched with the API key from the .env file and you won't need to enter it manually. If everything was done correctly, you will see the following message in the app UI: * ℹ️ Connection settings was loaded from .env file. * ✅ Successfully connected to V7. ### Entering credentials manually 1. Launch the app from the Ecosystem. 2. Enter the V7 api key. 3. Press the Connect to V7 button. ![](/files/YYxqaM9TgQyUW1cp86V4) If everything was done correctly, you will see the following message in the app UI: * ✅ Successfully connected to V7. **NOTE:** The app will not save your API key, you will need to enter it every time you launch the app. To save your time you can use the team files to store your credentials. # CVAT to Supervisely Convert and copy multiple CVAT projects into Supervisely at once. [This application](https://ecosystem.supervisely.com/apps/cvat-to-sly/migration_tool) allows you to copy multiple projects from CVAT instance to Supervisely instance, you can select which projects should be copied, labels and tags will be converted automatically. You can preview the results in the table, which will show URLs to corresdponding projects in CVAT and Supervisely. {% hint style="info" %} If you want to upload data, which was already exported from CVAT instance, you can use this [Import CVAT](https://ecosystem.supervisely.com/apps/cvat-to-sly/import_cvat) app from Supervisely Ecosystem. {% endhint %} {% hint style="info" %} [Complete migration guide](https://ecosystem.supervisely.com/apps/cvat-to-sly/migration_tool) from CVAT to Supervisely. {% endhint %} ## Preparation In order to run the app, you need to obtain credentials to work with CVAT API. You will need the following information: * CVAT server URL (e.g. `http://192.168.1.100:8080`) * CVAT username (e.g. `admin`) * CVAT password (e.g. `qwerty123`) You can use the address from the browser and your credentials to login to CVAT (you don't need any API specific credentials). Now you have two options to use your credentials: you can use team files to store an .env file with or you can enter the credentials directly in the app GUI. Using team files is recommended as it is more convenient and faster, but you can choose the option that is more suitable for you. ### Using team files You can download an example of the .env file [here](https://github.com/supervisely-ecosystem/cvat-to-sly/files/12748716/cvat.env.zip) and edit it without any additional software in any text editor. NOTE: you need to unzip the file before using it. 1. Create a .env file with the following content: `CVAT_SERVER_ADDRESS="http://192.168.1.100:8080" CVAT_USERNAME="admin" CVAT_PASSWORD="qwerty123"`. 2. Upload the .env file to the team files. 3. Right-click on the .env file, select `Run app` and choose the `CVAT to Supervisely Migration Tool` app. The app will be launched with the credentials from the .env file and you won't need to enter it manually. If everything was done correctly, you will see the following message in the app UI: * ℹ️ Connection settings was loaded from .env file. * ✅ Successfully connected to `http:/192.168.1.100:8080` as `admin`. ![](/files/HGCgYmgKbCBDAcZe7lkR) If everything was done correctly, you will see the following message in the app UI: * ✅ Successfully connected to `http://192.168.1.100:8080` as `admin`. NOTE: The app will not save your credentials, you will need to enter them every time you launch the app. To save your time you can use the team files to store your credentials. # Export While your data and annotations are stored securely in Supervisely, you can export and save your valuable assets and labels in various formats at any time. Whether you need to exchange data, create backups, or interact with external applications, there are multiple ways to achieve your goal. ## Export using Supervisely Apps You can export data in different ways: from the context menu of a project or a dataset or you can launch the export application directly from the [Ecosystem](https://ecosystem.supervisely.com/export). ### Export to Supervisely Format The best way to export and download your dataset from Supervisely is by saving it to the [Supervisely Format](https://github.com/supervisely/docs/blob/master/data-organization/supervisely-format.md). Since it has the full support of every capability available on the platform, no matter how complex your annotations and data are (for example, 3D labels of cloud point episodes with photo context), you can be 100% sure that all the information is saved in the convenient .json-based format. To download a project or dataset in `Supervisely format`, select Download in the context menu by **three dots icon (⋮)** and choose **Export in Supervisely Format**.
The appropriate Supervisely App will generate an archive, save it to your Files and provide you a download link at the **Tasks** page.
### **Export to other Formats** Of course, there are countless other Supervisely Apps that will help download your dataset in any desirable format. Just like with the Supervisely Format, select Download via App in the project or dataset context menu and choose any format of your likening. Here are some examples: * [Export to COCO](https://ecosystem.supervisely.com/apps/export-to-coco) * [Export to Pascal VOC](https://ecosystem.supervisely.com/apps/export-to-pascal-voc) * [Export activity as csv](https://ecosystem.supervisely.com/apps/export-activity-as-csv) * [Export to YOLOv8 format](https://ecosystem.supervisely.com/apps/export-to-yolov8) * [Export to DOTA](https://ecosystem.supervisely.com/apps/export-to-dota) * [Export to Cityscapes](https://ecosystem.supervisely.com/apps/export-to-cityscapes) * [Export to SemanticKITTI](https://ecosystem.supervisely.com/apps/export-to-semantic-kitti) - for point cloud episodes Explore more [export applications](https://ecosystem.supervisely.com/export) in the Ecosystem. **For Python developers** –you can use our Python SDK to download and convert your data to COCO, YOLO, Pascal VOC formats: ```python sly.Project.download(api, project_id, "./sly_project", log_progress=True) # Convert project to COCO format sly.convert.project_to_coco("./sly_project", "./result_coco") # Convert project to YOLO format (you can specify task_type) sly.convert.project_to_yolo("./sly_project", "./result_yolo") # Convert project to Pascal VOC format sly.convert.project_to_pascal_voc("./sly_project", "./result_pascal") ``` Check out the [documentation](https://github.com/supervisely/docs/blob/master/data-organization/Operations-with-Data/Converting-Splitdata.md#convert-data-using-supervisely-python-sdk) for more details on how to convert data using the Supervisely Python SDK and the [Developer Portal](https://developer.supervisely.com/getting-started/basics-of-authentication) to get started with the API. ### **Export only certain items** Some other Supervisely Apps will help you export only subsamples of your data. For example, select [Export only labeled items](https://ecosystem.supervisely.com/apps/export-only-labeled-items) application to skip export of any unlabeled items. ### **Export to cloud storage** Some Supervisely Apps can do more than just convert your project to some format, but actually provide an interactive web interface to configure more complex exports. For example, [Export to cloud storage](https://ecosystem.supervisely.com/apps/export-project-to-cloud-storage) allows exporting a project with annotations in Supervisely format directly from Supervisely platform to the most popular cloud storage providers, without any need to download archives. The application supports: * Amazon S3 * Google Cloud Storage (GCS) * Microsoft Azure * Any S3 compatible storage {% hint style="info" %} For developers: you can use the sources of this app as a starting point for your custom export to cloud. {% endhint %}
## **Export via SDK** Utilize our [Software Development Kit (SDK)](https://supervisely.readthedocs.io/en/latest/sdk_packages.html) to create custom solutions for data export, tailored to your unique requirements and processes. ## **Export via API** [Our API](https://api.docs.supervisely.com/) provides the ability to automate the data export process by integrating our system with external applications and services. # Core concepts To organize data for labeling and neural net training is not an easy task. If there is no structure, things get messy pretty quick. Learn those essentials to organize your efficiently. ☝️ ## 🗂️ Projects Project is a combination of datasets and related meta information (like classes and tags) and it's a major building block of data organization in Supervisely. {% content-ref url="/pages/-M54fC5kfcVDMQPT05GQ" %} [Broken mention](broken://pages/-M54fC5kfcVDMQPT05GQ) {% endcontent-ref %} ## 📂 Datasets This is where your labeled and unlabeled images, videos and point clouds live. A dataset is some sort of data folder with stuff to annotate. {% hint style="success" %} You can upload images multiple times, but only one copy will be stored inside Supervisely. This approach allows us to effectively work with cloud storage. Also this has a good effect on system performance and backup speed. {% endhint %} {% content-ref url="/pages/-M54fC5lCGh1KuU-VTCW" %} [Broken mention](broken://pages/-M54fC5lCGh1KuU-VTCW) {% endcontent-ref %} ## 📖 Definitions Definitions tab is for managing the classes and tags used in the project. This tab allows you to create, edit and delete object classes and tags. ### 🎨 Classes Classes are pre-defined types of your annotations, for example `Person` or `Background`. Thus, every label you create has defined class. ### 🏷️ Tags To associate some extra information with annotations (or images, or videos, ...) you can define a Tag, for example `Needs review`. {% content-ref url="/pages/bjYPjkKuRAhywYT2m24B" %} [Broken mention](broken://pages/bjYPjkKuRAhywYT2m24B) {% endcontent-ref %} ## Example For instance, in [Team](/collaboration/teams) "Driving Division" you can have a **Workspace** "Pre-Labeling". In this Workspace you can have a Project "Cityscapes" with two Datasets: "Zurich" and "Stuttgart". There are several classes defined in this Project (and, thus, in every Dataset): a building, a traffic light, a vehicle and so on. All classes are set to the "bitmap" shape, so that there is no way someone will accidentally create some "Cars" with polygon tool (a set of points), and some "Cars" with bitmap tool (a set of pixels). Also, there are two tags defined: a "vehicle\_type" of several pre-defined options ("Bus", "Bicycle", "Train") and a "color" that accepts any string value. Datasets are used to split data into a "subfolders" to make data management easier. For example, you can then define a [Labeling Job](/labeling/jobs) to label all Vehicles in Zurich. {% hint style="info" %} This is just an example, experiment! Maybe you would like to have projects like "Data from Data from 14 august" or "From Mike" or even "To train v2 (final) (2)" 😃 {% endhint %} # Project and Dataset # Create This article provides a step-by-step guide to creating classes and tags: how to define them in the project before annotation and how to add them directly in the labeling tools. In Supervisely, a Project is a core element of the system. It is where your data and annotations are stored, and it serves as the foundation for training and evaluating models. ### Creating a Project 1. To create your first project in the current Team and workspace, click the “Import Data” button. 2. If you don't have any data yet, you can also upload ready-to-use sample projects to explore and learn what Supervisely has to offer.
3. You can also use the **`New`** button and select the **New Project Wizard** option.
4. To create another project to the existing ones, simply click on the section with “+” and the text “Create project and import data” next to the folder of the created project.
So, we've covered all the main ways to create a project, and now there's just one final general step left. 5. Supervisely supports different project types depending on the kind of data you're working with: * Images * Videos * DICOM volumes * Point clouds Each project type is tailored to its specific data format, making it easier to manage annotations and prepare datasets for your Computer Vision tasks. After entering a **Name** and **Description** for your project, make sure to select the correct **Project type** based on your data format, as each project can contain only one type of data. Next, select a **Labeling interface** - you can read a description of each one by hovering over the question mark icon next to it. Finally, click the **`Create`** button.
🎉 Great! The project has been successfully created - you are now inside it. {% hint style="info" %} **Note:** Once the project is created, its type cannot be changed. {% endhint %} ### Creating a Dataset Now that your project is ready, it's time to import your data and start working. 1. So click the **`Import Data`** button. A dataset will be created automatically, and your data will be placed inside it.
2. Notice that next to the Project Name, a field with the **Dataset Name** has appeared.\ By default, the dataset is named after the date it was created, but you can rename it now or later. 3. Supervisely will now offer you the option to use the **Quick Auto Import** feature.\ In this tab, you can review the supported data formats, annotation formats, and file size limits based on your current subscription Plan.\ Once you're ready, go ahead and import your data.
Below the **Quick Auto Import** tab, you'll also find other import options, each offering different ways to bring your data into Supervisely.
Right after uploading your data, you'll be taken to the **Tasks** page, where you can monitor the upload process.
Once the upload is complete, the task will be marked as finished.\ You can then go to your **Dataset** by clicking the link (1) in the task card or by navigating to it through the **Main Menu** (2).
And here it is - your **Dataset** with the imported data! Now let's take a quick look at how to rename your project.\ Just follow the simple steps shown in the illustrations below - you'll see how easy it is to change the **Project Name**, as well as explore other features and options available for managing your projects in Supervisely.
So now you've learned how to **сreate a Project** and import data into a **Dataset**, which is located inside the Project.
To learn more about the structure and organization of data in Supervisely, check out the article [Data Structure](/data-organization/project-dataset/data-structure). # Data Structure This article explains the Data structure in Supervisely, including how Projects, Datasets, and Files are organized in Team and Workspace. Learn how to navigate, manage, and structure your data. ### Teams and Workspaces #### 1. Automatic Creation at Registration When a user registers on the Supervisely platform, one **Team** and one **Workspace** are automatically created in their account. By default, this workspace is named ***First Workspace***. This is how it looks on the Supervisely platform:
And this is how it looks schematically:
#### 2. Team & Workspace Structure Rules Every Member with [Full-scope Permission](/collaboration/admin-panel/users-management) is always a part of at least one **Team**.\ A member may leave or delete a personal **Team**, provided that they remain associated with at least one **Team** that includes at least one **Workspace** at all times. Each **Team** must always have at least one **Workspace**, although it doesn't have to be the original one created at registration. The Supervisely system strictly enforces these rules and will not allow any actions that would violate them. For example, you won't be able to delete your last remaining Team or its only Workspace. #### 3. Creating & Managing Teams A Member with the Admin role can invite other Members to their **Team**. The invited member will gain access to all projects within this **Team**.
In addition to their default **Team**, a Member can also create new **Teams** to collaborate on separate projects or with different groups. #### 4. Switching Between Teams To manage or switch between **Teams**, click the arrow next to the name of your current Team.\ A menu will appear with a list of all Teams you are a member of (not necessarily the ones you created) along with other settings.
When you [invite other Members](/data-organization/project-dataset/data-structure) to your **Team**, make sure you have the correct **Team** selected as active.\ The invitations will be sent specifically to the currently active **Team** that you created. ### Projects and Datasets Inside a **Workspace**, a Member can create an unlimited number of Projects.\ Each **Project** can contain multiple **Datasets**, which store the actual data and annotations. This flexible structure allows Members to organize data in a way that fits their workflow. Furthermore, a Member can create additional **Workspaces** inside any **Team** where they have the Admin role.\ Inside a **Dataset**, you can create **Sub-Datasets**, enabling flexible and deeply nested data structures - just like folders and subfolders on your computer. There are no limitations on nesting depth, so you can organize your data in whatever hierarchy makes sense for your workflow.
Let's repeat an important rule: at the **Project** level, you cannot store files directly - only **Datasets** can exist there. Files and **Sub-Datasets** can only be added inside a **Dataset**. You can think of **Datasets** as folders and **Sub-Datasets** as **subfolders**. This allows you to recreate complex directory structures exactly the way you organize data on your local machine or in your company's cloud storage. It's especially useful if you're working with a shared storage system that already follows a specific hierarchy - you can mirror that same structure inside Supervisely without restrictions. To create a sub-dataset inside an existing dataset: 1. Click the `Add` button 2. Select **Create New Dataset**\ Then, you can navigate into the newly created sub-dataset and upload your files there.
Great! Your sub-dataset with files is now ready.
### Project versions [**Project Versions**](/data-organization/project-dataset/project-versions) let you manage and track project states over time, allowing you to save, instantly preview, restore, and compare different iterations of your data. Key features of project versioning include: * **Version Control**: Track and manage changes made to the project over time, ensuring every specific state is recorded. * **History Tracking & Metadata**: Maintain a comprehensive history of all modifications, making it easy to understand the project's evolution. You can edit version titles and descriptions at any time to keep context clear. * **Instant Previews**: Create read-only project snapshots (for Images and Videos) to instantly inspect annotations in the labeling toolbox, use AI Search, and filter data without requiring a full project restore. * **Reverting Changes**: Restore any previous version of the project, allowing you to get a new project with a specific state of data. * [**MLOps Workflow**](/data-organization/mlops-workflow): Execute machine learning tasks on specific versions of the project data, ensuring that the exact state of the data used is known. Previews are also fully integrated into the workflow graph. This guarantees consistency and reproducibility of results.
Project versions overview
{% hint style="info" %} **Note**: **Project versions** are not available on the Free plan. {% endhint %} # Define Classes & Tags This article explains how to create and manage classes and tags for data annotation in the Labeling Tool to ensure structured and consistent labeling within a project. Definitions are a centralized place where all classes and tags used for data annotation are managed.\ A class defines an object category and specifies the type of geometry that will represent this class in annotations, such as bounding boxes, masks, polygons, etc.\ Tags describe not only the annotations themselves but can also apply to frames, images, and more.\ Properly defined classes and tags ensure consistent and structured annotations throughout the entire project The **Definitions** tab, available both in the Project and in the Labeling Tool, is divided into two main categories: 1. **Classes** - for managing object classes. 2. **Tags** - for managing tags used in data annotation. ## 1. In Panel ### Definitions tab functions * **Adding a new classes and tags**: you can create a new class or tag by clicking the `+ NEW CLASS` or `+ NEW TAG` button and filling in the required fields. * **Editing**: each class and tag can be edited by changing its parameters such as name, color, scope and value type. * **Archiving**: if a class or tag is not needed temporarily, it can be archived so that it is not displayed in the list of active elements but remains in the project. Archived classes and tags can be restored at any time. * **Sorting**: Sort classes and tags to quickly find the elements you need. * **Newest (default)** - displays the most recently created items first. * **Oldest** - displays the oldest items first. * **Name (A-Z)** and **Name (Z-A)** - sorts items alphabetically in ascending or descending order. * **Shape (A-Z)** and **Shape (Z-A)** - **(for classes only)** sorts items by shape type in ascending or descending order.
## Classes & Tags: distinctions Although both tags and classes are used to identify objects, they serve distinct purposes: * **Classes:** Represent clear categories that an object belongs to, such as "car", "truck", or "bus" for vehicles. * **Tags:** Provide specific information about objects or images, such as context or properties. Tags are more flexible and can include details not tied to formal classifications. For example, the tag "Traffic Density" can have values "High" or "Low" indicating the level of traffic density, and the tag "Action" can signify the actions of an object (e.g. "Stopped" or "Moves"). An image or an object can have multiple tags assigned, while each object usually belongs to a single class, i.e. classes are used to explicitly categorize objects. Tags can be more personalized and focused on specific characteristics and attributes. Tags typically add context and descriptive attributes not necessarily related to the formal classification of an object. {% hint style="info" %} Tags provide a more flexible and free way to describe, while classes provide a formalized structure for training models. {% endhint %} ## Classes **Classes** are object categories used to annotate images and video by creating shapes on objects. Each class represents an object type, and each annotated object in an image or video frame has exactly one class associated with it. * **TITLE** - the name of the class. The name should be unique and clearly describe the object, for example, "Person," "Car," or "Tree." * **SHAPE** - the annotation shape for the class. Available options include [Bounding Box](/labeling/labeling-tools/bounding-box-rectangle-tool), [Mask](/labeling/labeling-tools/mask-pen-tool), [Polygon](/labeling/labeling-tools/polygon-tool), [Keypoints](/labeling/labeling-tools/graph-keypoints-tool), [Points](/labeling/labeling-tools/point-tool), [Line](/labeling/labeling-tools/polyline-tool), Cuboid 2D, Alpha Mask, and Any Shape. * **COLOR** - the color assigned to the class, displayed on the screen to visually differentiate the annotation. * **HOTKEY** - a shortcut key assigned to the class for quick annotation during labeling.
### How to create a new class 1. Click on the `+ NEW CLASS` **button** to start creating a new class. 2. Set a **unique title** for the class to clearly identify it. 3. Add a **description (optional)** and assign a [**hotkey**](#user-content-fn-1)[^1] for quick selection during labeling. 4. Choose the **Shape** for the class (e.g., Bounding Box to label an object with rectangular frame). {% hint style="info" %} **Note**: Some shapes are only applicable to specific labeling tools (e.g., Cuboid 2D is relevant for Point Cloud and Point Cloud Episodes). {% endhint %} 5. Generate or choose a **color** for the class to visually differentiate it from other classes.
## Tags **Tags** serve as data annotation and classification tool. These attributes, assigned to images/videos or labeled objects, make it simple to sort things and give important information about what's in an image. * **TITLE** - the name of the tag. Like classes, tags should have unique names, for example, "Highway", "Traffic Density", "Action". * **APPLICABLE TO** - the scope of the tag's application (e.g., video only, objects only, videos and objects). * **Image Tags:** Apply to images and provide information like category, properties (resolution), geographic details, and content. **Object Tags:** Apply to objects within images, detailing characteristics (e.g., "broken" equipment), state (e.g., "ripe" fruit), and localization (e.g., "anterior" placenta). * Some tags can be applied to **both images and objects**. Such Tags may describe both image characteristics and individual objects at the same time, providing comprehensive labeling. * **SCOPE** (**for videos project)** - the tag's range of application: * **Global** - the tag applies to the entire video or object. * **Frame-based** - the tag applies only to specific frames in the video. * **Global and Frame-based** - allows you to use a tag for the entire video or object as well as for individual frames at the same time. This means that you can set a global tag for the entire video or object, denoting a permanent property, and apply it to individual frames to label temporary changes or events. * **TAG VALUE TYPE** - the type of value associated with the tag: * **None (Tag without Value):** Used to flag specific properties. For example, a tag "train" might mark data for neural network training. * **Text Tag:** Contains textual descriptions or comments about the object or image. * **Number Tag:** Represents numeric properties, useful for regression tasks (e.g., size, weight). * **One of:** Indicates that the value must be one of a predefined set, such as colors (Red, Blue, Green). * **Date Tag:** Stores a datetime value in ISO 8601 format. Useful for recording timestamps such as review date, creation date, or any event time. * **COLOR** - the color displayed on the screen to make the tag easily identifiable.
### How to create a new tag 1. Click on the `+ NEW TAG` **button** to start creating a new tag. 2. Set a **unique title** for the tag to clearly identify it. 3. Assign a **hotkey (optional)** for quick selection during labeling. 4. Choose or generate a **color** for the tag to visually distinguish it from other tags. 5. Select **"Applicable to"** to define whether the tag will be used for: * Images and objects * Images only * Objects only 6. Define the **tag's value type**: * **None**: a tag without a value. * **Text**: allows adding a text description or comments. * **Number**: represents numerical properties. * **Single choice (One of)**: select one option from a predefined set. * **Date**: stores a datetime value in ISO 8601 format (e.g. `2026-04-23T15:15:48`). 7. Additional **Scope** option for video projects: * **Global and Frame-based**: tag can be applied globally or to specific frames. * **Frame-based**: tag is applied only to specific frames. * **Global**: tag applies to the entire video or object as a whole. 8. Click **Save** to complete the creation of the new tag.
### Multiple tags mode To apply the same tag multiple times: 1. **From the Project:** Go to Settings > Tags and find Multiple Tags Mode. 2. **From the Image Labeling Toolbox:** Click on the Tag icon, hover over 🔁, and click the blue link Setting to enable Multiple Tags Mode. To learn more about the practical uses of tags and explore advanced tools, check out our in-depth blog post: [Mastering Image Tagging](https://supervisely.com/blog/mastering-image-tagging/). This guide provides valuable insights and real-world examples to help you maximize the potential of tagging in your projects. {% embed url="" %} ## 2. Defining Classes and Tags Directly in Labeling Toolbox **Classes** and **Tags** created manually by an annotator inside the Labeling Tool (during the annotation process) are automatically added to the project's **Definitions** on the Projects page. These classes and tags immediately become available to all team members - they appear in the Labeling Tool for other annotators as well. **This behavior enables the following:** * Dynamic expansion of the project structure, even if the Definitions were not configured in advance. * Consistency across the team, by maintaining a single, shared list of classes and tags. * Avoiding duplication, since new items are added directly to the central schema. ### Where to Define Classes and Tags in the Labeling Tool The locations where classes and tags can be defined directly in the Labeling Tool vary depending on the type of data being annotated. This is expected, since the interface of the Labeling Tool also slightly differs based on the data type in use. Let's go through each case one by one. #### 1. When annotating an Images: When working with images, the **Definitions panel** is located on the left side of the Labeling Tool. To add a new class or tag: 1. click the plus **`+`** on the panel, then choose whether you want to add a class or a tag. 2. In the modal window that appears, configure the class or tag parameters, then click **`Create`** to save it.
The newly created class or tag will immediately appear in the **Definitions panel** for all annotators and become available for annotation. To create a new **Class** and simultaneously reassign it to an existing object in the project: 1. In the Objects panel (located on the right side of the Labeling Tool), select an object and click **`⌄`** next to its current class name. And in the dropdown menu, select ***Create class***.
2. In the modal window that appears, configure the new class parameters, then click **`Create`** to save it.
The new class will immediately appear in the Definitions panel on the left and will be automatically assigned to the selected object in the **Objects panel**.
#### 2. When annotating a Video: When working with videos, the **Definitions panel** is located on the right side of the Labeling Tool. To add a new class or tag: 1. click the plus **`+`** on the panel, then choose whether you want to add a class or a tag. 2. In the modal window that appears, configure the class or tag parameters, then click **`Create`** to save it.
The newly created class or tag will immediately appear in the **Definitions panel** for all annotators and become available for annotation. #### 3. When annotating a Dicom Volume or a Point Cloud: The locations where classes and tags can be defined in the Labeling Tool are the same for **Dicom Volume** and **Point Cloud**. Let's walk through the process using **Dicom Volume** as an example. So you can create a new class directly from the Objects panel and simultaneously reassign it to an existing object in the project. To do this: 1. In the Objects panel (located on the right side of the Labeling Tool), select an object and click **`⌄`** next to its current class name. And in the dropdown menu, select ***Create class***.
2. In the modal window that appears, configure the new class parameters, then click **`Create`** to save it.
**Creating a New Tag from the Objects Panel** You can also create a new **Tag** directly from the **Objects panel** while annotating an object: In the **Objects panel**, select an object. In the **Tags Available** section, click the ***Add project tags definitions*** icon. In the modal window that appears, configure the new tag parameters, then click **`Create`** to save it.
The newly created tag will immediately appear in the **Objects panel** for all annotators and become available for annotation. [^1]: You can only set a single latin character (because other combinations may be unavailable). # Gallery & Table views This article is about how gallery and table views let you customize data display: gallery provide a quick visual overview, while tables offer detailed, sortable comparisons. Supervisely provides multiple data display modes to help you **explore and analyze your data more effectively**. Depending on the context, you can choose between visual gallery-based views and structured table-based views. These display modes are available both on the **project list page** and **inside individual projects and datasets**. Gallery views are best suited for getting a quick **visual overview** of your data. Table views offer a **structured and sortable layout**, which helps when comparing metadata or filtering based on specific attributes. These features are designed to help teams **understand the composition of their datasets**, **inspect content before labeling**, and **navigate large projects more efficiently**. The data display modes on the project list page and within a project differ slightly: ## On the Project List Page On the main project list page (inside a workspace), you can toggle between two display modes. Use the **view switcher icon** in the upper-right corner of the page to switch between **gallery view** and **table view**. ### Gallery View Projects are displayed as visual cards with thumbnails (default), making it easier to recognize them at a glance.
### Table View Projects are listed in a tabular format, which allows for sorting by various parameters such as **project name**, **creator**, and **project size** (based on the number of datasets and files inside each project).
{% hint style="info" %} **Note**: To select all projects, switch to the **Table View** mode. {% endhint %} ## Inside a Project Within a specific project - whether on the datasets overview page or inside an individual dataset - you have additional display modes to choose from. These let you control how files and annotations are visualized. Use the display settings icon located in the upper-right corner of the screen (next to the **Add** button) to switch between modes: ### Gallery with Hierarchy Datasets are displayed as visual cards with thumbnails (default), making it easier to recognize them at a glance.
### Gallery Expanded Flattens all datasets and sub-datasets into a single, continuous gallery. This is helpful for **visually scanning large volumes of data** across the entire project.
### Table with Hierarchy Displays a **nested table** that shows the list of datasets and their files at the **top level** of the project. Useful for **reviewing dataset structure** and inspecting associated metadata in a structured format.
### Table Expanded Displays all data from all datasets - including nested sub-datasets - in a **single-level table**. Ideal for **sorting and comparing** files across the entire project, regardless of their position in the dataset hierarchy.
{% hint style="info" %} **Note**: To select all datasets within a project, switch to one of the **Table View** modes. {% endhint %} ## Grid Size (Gallery Views Only) When using any gallery mode, you can adjust the **grid size** to control how many items are shown per row: * Smaller grid size: More thumbnails per row, helpful for scanning large datasets. * Larger grid size: Bigger previews, better suited for reviewing image or point cloud content in detail. This allows users to **customize the visual density** of the gallery depending on their review or labeling context. #### Summary Different display modes are tailored for different stages of dataset management - from **visual exploration** to **structured inspection and filtering**. By choosing the right view, users can better navigate complex projects, understand dataset composition, and prepare data for annotation workflows. # Collections Collections are custom selections of data within a project. They enable flexible filtering and control over annotation workflows. Collections are custom groups of images, videos, or other items within a single Supervisely project. They serve as flexible selections of data, independent of dataset structure, and are often used for filtering, reviewing, or as a source for annotation queues. ## Key concepts * A collection is always linked to a specific **project**. * Each collection contains only **one data type** - images, videos, or point clouds - consistent with the project type. * Collections are **independent of datasets**. A collection can include items from different datasets within the same project. * You can **add or remove** items from a collection at any time. * Internally, a collection is just a **list of item IDs**, along with a name and optional description. ## Step 1. Creating collection For creating collection open any dataset and switch to a flat list view like ***Gallery Expanded*** or ***Table Expanded*** at first.
\
1. If you want to create collection from all dataset items do not select any items, just click the ***Arrow*** to the right of the ***Annotate*** button and select option ***Add to collection***.
\ \ If you want to create collection from several items, select the desired items **using filters** (1.1) or/and manual (1.2) selection and click the ***With (number) selected*** button and select option ***Add to collection*** (1.3).
\
2. A modal window will appear. Give a name for new collection and press ***Add*** button.
You can also add descriptions when creating a collection for better organization. 3. From this moment, your newly created collection will be available in the list of collections.
{% hint style="info" %} **Note:** Creating or editing collections is available only in flat views. Dataset (hierarchical) view does not support this feature. {% endhint %} ## 2. Adding items to collection You can add items to an existing collection from any dataset inside one project. For example, let's go to a different dataset and: 1. Select the desired items **using filters** (1.1) or/and manual (1.2) selection. Click the ***With (number) selected*** button and select option ***Add to collection*** (1.3).
2. In the pop-up window, select an existing collection from the list and press ***Add*** button.
3. Now, while in any dataset, you can select the desired collection from the filter bar, and you'll see that you are taken directly to the selected collection.
## 3. Deleting items from collection For deleting items from collection just select the desired items using **filters** (1) or manual selection (2). Click the ***With (number) selected*** button and select option ***Remove from collection*** (3).
{% hint style="info" %} **Note:** The **Remove selected items** option located above means deleting the items from the dataset where they were originally located. {% endhint %} ## Annotating collections Collections provide a powerful way to organize and manage subsets of data across different datasets. Once you've grouped items into a collection, you can send them for annotation with full flexibility and control. There are two main ways to annotate collections: #### 1. Direct annotation inside the collection: Open a collection, browse through items, and annotate them just like in any regular dataset. All changes will be saved directly to the original dataset the item belongs to.
#### 2. Labeling Queue source Create a **Labeling Queue** based on a collection:\ Use a collection as the data source when setting up a **Labeling Queue**. This allows you to assign specific items from the collection to annotation teams or individuals. Collections maintain a dynamic link to their items. Any annotations made within a collection are instantly reflected in the dataset of origin. For example, to create a Labeling Queue based on an existing Collection: 1. Go to **Labeling Job** from the main menu, 2. Select the **Queue** tab, 3. In the **Data to Annotate section**, choose Collection as the source and then select the specific collection you want to use.
You can create a queue with **Collection** as the source, and later add the desired items to it. **Example workflow:** 1. Create an empty collection inside a new or existing project. 2. Create a Labeling Queue that pulls data from this collection. 3. As you import new data or identify specific items for annotation, add them to the collection. 4. The queue will automatically reflect the collection's contents. {% hint style="success" %} This setup allows dynamic control over labeling pipelines, especially useful when data arrives in batches or needs manual pre-selection. {% endhint %} {% hint style="info" %} Note: Collections do not duplicate data - they act as smart references to existing items across multiple datasets. {% endhint %} ## Data filtering Collections are ideal for creating reusable subsets of data. For example, after training a model, you might group all false positives into a collection for further analysis or re-labeling. **Benefits:** * Simplifies repeated access to specific groups of items * No need to store or pass item ID lists in code * Easily maintain and update the selection over time ## Automating Collection Management with Python SDK Learn how to programmatically create, retrieve, and manage collections using the Supervisely Python SDK. The following examples provide step-by-step guidance for efficient collection handling.\ Check out the [SDK Reference](https://supervisely.readthedocs.io/en/stable/sdk/supervisely.api.entity_collections.EntitiesCollectionApi.html) for more details. {% tabs %} {% tab title="Create" %} ```python import supervisely as sly api = sly.Api() new = api.entity_collections.create( project_id=project_id, name="my collection", description="my collection description", ) print(new.id) # Output: 123 ``` {% endtab %} {% tab title="Get info" %} ```python import supervisely as sly api = sly.Api() collection_id = 2 info = api.entities_collection.get_info_by_id(collection_id) print(info.name) # Output: my collection ``` {% endtab %} {% tab title="List Collections" %} ```python import supervisely as sly api = sly.Api() project_id = 1 collections = api.entities_collection.get_list(project_id) for collection in collections: print(collection.name) # Output: ["my collection", "another collection"] ``` {% endtab %} {% tab title="Add items" %} ```python import supervisely as sly api = sly.Api() collection_id = 2 item_ids = [525, 526] res = api.entities_collection.add_items(collection_id, item_ids) print(res) # Output: [ # {"id": 1, "entityId": 525, 'createdAt': '2025-04-10T08:49:41.852Z'}, # {"id": 2, "entityId": 526, 'createdAt': '2025-04-10T08:49:41.852Z'} # ] ``` {% endtab %} {% tab title="Get items" %} ```python import supervisely as sly api = sly.Api() collection_id = 123 project_id = 111 res = api.entities_collection.get_items(collection_id, project_id) print(res) # Output: [ # ImageInfo(id=525, name='image1.jpg', ...), # ImageInfo(id=526, name='image2.jpg', ...) # ] ``` {% endtab %} {% tab title="Remove items" %} ```python import supervisely as sly api = sly.Api() collection_id = 2 item_ids = [525, 526, 527] res = api.entities_collection.remove_items(collection_id, item_ids) # print(res) # Output: [{"id": 1, "entityId": 525}, {"id": 2, "entityId": 526}] ``` {% endtab %} {% tab title="Remove collection" %} ```python import supervisely as sly api = sly.Api() collection_id = 2 api.entities_collection.remove(collection_id) ``` {% endtab %} {% endtabs %} You can also use Collections to create a **Labeling Queue** programmatically. This allows you to automate the process of labeling data based on specific collections. Adding new items to a collection will automatically update the Labeling Queue, allowing for dynamic management of your annotation tasks. Here's a simple example of how to integrate collections into your labeling workflow: Script 1 creates a collection and adds items to it. The second script checks the queue stats, retrieves items that finished labeling and removes them from the collection (for example, if you want to move them to another project or collection).
Script 1 (create Collection and Labeling Queue) ```python import random import supervisely as sly api = sly.Api() project_id = 1 all_images = api.image.get_list(project_id=project_id) # let's choose 10 random images random_images = random.sample(all_images, 10) # create a collection collection = api.entity_collections.create( project_id=project_id, name="Collection 1 for labeling", ) print(f"Collection created: {collection.id}") # get my info me = api.users.get_my_info() # create a labeling queue queue = api.labeling_queue.create( collection_id=collection.id, name="Labeling Queue 1", user_ids=[me.id], reviewer_ids=[me.id], dynamic_classes=True, dynamic_tags=True, allow_review_own_annotations=True, skip_complete_job_on_empty=True, ) print(f"Labeling queue created: {queue.id}") # collection is empty, let's add items item_ids = [image.id for image in random_images] api.entities_collection.add_items( collection_id=collection.id, item_ids=item_ids, ) # after ``` ```bash python script1.py ```
Script 2 (get labeled items and remove them from collection) ```python import supervisely as sly import time api = sly.Api() project_id = 1 collection_id = 2 queue_id = 3 res = api.labeling_queue.get_entities_all_pages( queue_id, collection_id, status="accepted", ) images = res["images"] img_ids = [i["id"] for i in images] # create a new collection new_collection = api.entity_collections.create( project_id=project_id, name="Collection 2 for training", ) print(f"Collection created: {new_collection.id}") # add items to the new collection api.entities_collection.add_items( collection_id=new_collection.id, item_ids=img_ids, ) # remove items from the old collection api.entities_collection.remove_items( collection_id=collection_id, item_ids=img_ids, ) ``` ```bash python script2.py ```
## API support Collections are fully accessible through the Supervisely API. With these [Entity Collections](https://api.docs.supervisely.com/#tag/Entities-Collections), you can: * [Add Items to Entities Collection](https://api.docs.supervisely.com/#tag/Entities-Collections/paths/~1entities-collections.items.bulk.add/post) * [Create Entities Collection](https://api.docs.supervisely.com/#tag/Entities-Collections/paths/~1entities-collections.add/post) * [Get Entities Collection info by ID](https://api.docs.supervisely.com/#tag/Entities-Collections/paths/~1entities-collections.info/get) * [List Entities Collections](https://api.docs.supervisely.com/#tag/Entities-Collections/paths/~1entities-collections.list/get) * [Remove Entities Collection](https://api.docs.supervisely.com/#tag/Entities-Collections/paths/~1entities-collections.remove/delete) * [Remove Items From Entities Collection](https://api.docs.supervisely.com/#tag/Entities-Collections/paths/~1entities-collections.items.bulk.remove/delete) * [Update Entities Collection](https://api.docs.supervisely.com/#tag/Entities-Collections/paths/~1entities-collections.editInfo/put) ## Limitations * Collections are limited to a **single project**. * Only one **data type per collection** is supported. * There's currently no dedicated collections page - access and management is done through flat views or Labeling Queue creation. # Project Versions Learn how to use Project Versions in Supervisely. Save, restore, and track project states, instantly preview data, and visualize data evolution with MLOps Workflow. Project Versions let you capture, restore, and compare specific states of your project data over time. Combined with the visual MLOps Workflow, this ensures reproducibility, traceability, and efficient collaboration across labeling, augmentation, training, and evaluation stages. Project Versions currently support the following project types: * Images * Videos * Volumes
Project Versions Overview
## Why tracking and reproducibility matter * Data evolves over time. Track when and how changes occur to maintain model accuracy. * Avoid confusion after multiple iterations. Keep a clear history of which datasets and models were used. * Ensure reproducibility and collaboration. Make it easy to reproduce results and share accurate insights across teams. ## Key capabilities * **Version control:** Save and manage different states of a project. * **Centralized history:** View and describe all changes in one place. * **Instant previews:** Create read-only snapshots to quickly inspect data without restoring the full project (Images and Videos only). * **One-click restore:** Create a separate project that represents the current project's state at the selected version. * **Flexible metadata:** Edit version titles and descriptions at any time to keep context clear. * **Efficient storage:** Versions are stored in a secure binary format. Previews only store unique annotations, while media files are linked, not duplicated. {% hint style="info" %} **Note:** Project Versions are available on Pro and Enterprise plans. {% endhint %} ## Versions tab Use the **Versions** tab on a project to: * Create versions at any stage (data upload, annotation, augmentation, or transformation). * Edit the **Title** and **Description** of existing versions for future reference. * Access an instant Preview of the data by clicking on the version's Title link. * Restore a prior state by creating a new project from a selected version. {% hint style="warning" %} You can create a version only if the project has changed since the last version. If there are no changes, creating a new version is not available. {% endhint %}
Versions Tab Interface
## Version Previews For Image and Video projects, you can instantly inspect a version's data using **Project Previews**. A preview acts like a standard project but operates in a [read-only state](/data-organization/project-dataset/project-settings#read-only-mode), allowing you to verify data without waiting for a full project restoration from a binary version.
Version Preview Page
### Creating and managing Previews * **During creation:** Check the **Enable Preview** flag when creating a new version to automatically generate a snapshot accessible in the well-known UI. * **For existing versions:** Retroactively enable previews for older versions. This is especially useful for checking versions automatically generated by apps during training pipelines.
Enabling Preview
* **Restoring state:** If a preview snapshot is accidentally altered, you can easily restore it to its original version state.
Restore Preview
### Using Previews Once enabled, access the preview by clicking the version's **Title** in the Versions tab. Inside the preview snapshot, you can: * Open images or videos in the labeling toolbox to visualize annotations. * Use **AI Search** and data filtering to find specific assets. * Clone the snapshot or download it using ecosystem apps. * Perform any standard project action that does not modify the data state.
Labeling Tool
💡 **Tip:** Previews are highly storage-efficient. They only consume local storage for annotations. Media data (images or videos) is safely linked without duplication. ## Typical workflow 1. Import data and annotate with labeling tools. 2. Create a version (with **Enable Preview** checked) to capture the baseline state before transformations. 3. Apply augmentations, filtering, or other data operations. 4. Train a model — training apps automatically create versions and produce checkpoints and reports. 5. Quickly inspect auto-generated versions using Previews. 6. Compare versions and, if needed, restore to a previous state to branch experiments. ## Visualizing data evolution with MLOps Workflow The MLOps Workflow provides a visual map of how data flows through apps and operations in your project. Use it to inspect application sessions, navigate to projects and files, and understand how each transformation impacts results. * **Preview Integration:** Previews are fully integrated into the workflow. You can see preview availability and access them directly from the MLOps Workflow cards. Creating a preview from an existing version is also documented on the workflow graph.
MLOps Integration
* **Data operations include:** * Augmentation (cropping, resizing, rotation, blurring, adding noise, etc.) * Annotation transformations (convert shapes, merge, rasterize) * Dataset split/merge/subsample/filter * Training data generation and format conversions See the dedicated guide for details and best practices: [MLOps Workflow](/data-organization/mlops-workflow). ### Minor versions on the Workflow graph Minor versions indicate that the project changed after the last major version was cut and before the next major version is created. On the screenshot below, minor versions are underlined (e.g., **1.1** and **2.1**).
Minor Versions in MLOps Workflow
* Shown as 1.1, 2.1, ... where 2 is the last restorable (major) version. * Each minor version reflects the current state of the project at that point on the graph. * You cannot restore a project from a minor version; only major versions are restorable. * Minor versions appear automatically on the graph when the project changes (manual edits or app sessions) without cutting a new major version. * If nothing changed after a major version, minor versions will not appear. **Example** (as shown on the screenshot): You cut version **2** (major). After that, you changed the project (e.g., edited annotations manually or ran an app without cutting a new major version) — the graph displays **2.1** to indicate the project state has changed since version 2. # AI Search This article is about AI Search, which quickly finds images using semantic similarity powered by CLIP. It supports prompt-based and diverse search modes with automatic embedding updates. **AI Search** allows users to intelligently search for images within a project or dataset using semantic similarity. It leverages **CLIP** under the hood to generate vector embeddings of images, which are stored in a dedicated embedding database (Qdrant). These embeddings are used both to process search queries and to keep the data up to date through automated updates. {% hint style="info" %} **AI Search** is only available to: * Users with a **PRO subscription** * Clients using **Supervisely Enterprise Instances** If you do not have access, a prompt will appear explaining the feature requirements. {% endhint %} {% hint style="success" %} **Data Privacy & Security** All data processed for AI Search remains completely private and secure. No data or embeddings are transmitted to external services or shared outside environment, ensuring full data privacy. {% endhint %} ## Enabling AI Search When a user opens a project in Supervisely, the **AI Search** button appears at the top of the interface.
### First-time Activation 1. Click the **AI Search** button. 2. If AI Search is not yet enabled for this project, a modal dialog will appear asking if you want to enable it.
3. Upon confirmation: * The project is connected to the AI Search service. * Image embeddings for all images in the project begin generating automatically. * While embeddings are being prepared, search is temporarily unavailable. {% hint style="info" %} While embeddings are being created, the **AI Search** button shows animated **sparkling stars**.\ Once complete, the star icon in the button turns **solid blue**, indicating that the project is ready for searching. {% endhint %}
Embeddings are updated automatically on a regular schedule. Manual updates can also be triggered via API. ## Performing a Search After embeddings are ready, clicking the **AI Search** button opens a modal window with two available search modes: ### 1. Prompt-based Search Search for images using a natural language prompt. * **Description**: The system embeds your text prompt and compares it with image embeddings to find the most semantically similar images. * **Score Chart**: After search, a distribution chart shows similarity scores between the prompt and images. * **Filtering**: You can filter results by adjusting score thresholds directly on the chart. * **Results Limit**: You can set how many top images to return. If the number of relevant images is lower than the limit, all available matches are shown.
**Example:** > *"A person riding a bicycle"* returns images that visually and semantically match this description - regardless of labeling. #### Filter by Score Range After performing a prompt-based search, a score distribution chart is shown: * **X-axis** – similarity score * **Y-axis** – number of images Use the slider below the chart to filter results by similarity: * **Above threshold**: shows images most similar to the prompt * **Below threshold**: shows edge cases with low similarity * **Full range (default)**: all results Images are always sorted by score, with the most relevant first.
### 2. Diverse Search Explore a representative variety of images based on the semantic structure of your dataset. #### Available Methods: * **Random**: A simpler approach that selects points evenly across the cluster, but may miss edge cases or unique examples. * **Centroids**: Returns a more diverse sample by selecting representative points closer to cluster centers. Useful for getting typical examples from each semantic group in your data. No text input is required in this mode.
Diverse Search is particularly valuable for complex machine learning scenarios such as iterative active learning workflows, additional sampling for underrepresented classes, and refining labeling tasks based on insights from previous annotation iterations. ## Search Results and Collections Each search creates a temporary **collection** that acts as a dynamic filter within your project: * This collection contains only the images that matched the query. * If needed, it can be saved as a separate collection for future use as a filter, since each new search overwrites the previous results and renaming the temporary collection is not possible. * Useful for organizing search results and building datasets based on semantic criteria. ## AI Search in Machine Learning Workflows Beyond basic image search, AI Search serves as a powerful tool for various machine learning scenarios. Here are some practical applications that can significantly improve your ML workflow efficiency: ### Rare Instance Detection in Video Data When monitoring production lines through continuous video streams, defective products or anomalies represent only 0.1-2% of all captured frames, while 98%+ show normal operation. Traditional manual review is almost impossible at this scale. Supervisely AI Search allows you to simply search for relevant instances of interest using an example image query, or trying a text-based query. This could be useful for finding out all relevant instances and using them in training an AI model, which can automatically detect such instances in production. ### Active Learning Approach When developing AI models, some industries collect millions of images daily, but manually annotating even 1% would be very expensive. The Active Learning approach aims to select the most informative samples that will maximally improve your model with minimal annotation effort. The **Diverse Sampling with Centroids** is very helpful in this context. Instead of randomly selecting data to be manually annotated, selecting more diverse data proves to be more effective. For example, in autonomous vehicle development, such sampling would intelligently select images across different conditions: daylight scenes with clear visibility, nighttime scenarios with artificial lighting, rainy weather with reduced visibility, urban environments with heavy traffic, rural roads with minimal infrastructure, and edge cases like unusual vehicle types or unexpected road obstacles. This ensures your model learns from the full spectrum of driving conditions rather than being biased toward the most common scenarios, hence, creating more robust and reliable AI systems with significantly less annotation effort. ### Data Curation & Quality Control AI Search transforms manual data curation into an intelligent, systematic approach. **Data exploration:** Start by understanding what's actually in your dataset using diverse search, getting a representative sample of your entire dataset. **Systematic collection building:** You can create targeted collections for different aspects of your dataset. For example, search "nighttime urban scenes" and save the images to a "Lighting\_Night" collection. Search "blurry images" and save as "Quality\_Blur" collection or remove them. ## Managing AI Search ### Temporary Collections All search results are shown as a temporary collection in the Filters panel. This collection acts as a dynamic filter within your project, containing only the images that matched your search query. #### Using Temporary Collections as Filters The temporary collection functions as a base filter that can be combined with additional filtering options available in the Filters panel. You can: * **Apply additional filters**: While the temporary collection is active, you can further refine your results by adding filters with AND logic: * **AI Search results**: Filter by similarity score threshold (Above/Below) when using AI Search * **Objects Class**: Filter by specific object classes and set the number of objects (between min and max values) * **Images Tag**: Filter images by their tags (is/is not) * **Objects Tag**: Filter by tags assigned to objects (is/is not) * **Labeling Job**: Filter by labeling job and status (Pending, Accepted, Rejected, etc.) * **Objects Author**: Filter by the author who created the objects * **Issues**: Filter images with open issues * **Combine multiple criteria**: All filters work with AND logic, allowing you to create precise queries. For example, you could search for "nighttime scenes" and then additionally filter to show only images containing more than 5 annotated objects with class "car". * **Real-time filtering**: All filter operations work in real-time, providing instant results even on large datasets.
#### Working with Temporary Collections With images in a temporary collection, you can: * Copy them to other datasets * Move or delete images * Create annotation jobs from the filtered results * Use the collection to start annotating only the filtered subset of images {% hint style="info" %} **Important Notes** * Collections behave like any other filter but are **not saved automatically** * Each new search **overwrites** the previous temporary collection * Renaming the temporary collection is **not possible** * To preserve search results for future use, you must save the collection as a separate, permanent collection {% endhint %}
This temporary collection feature is particularly useful for organizing search results and building datasets based on semantic criteria combined with precise filtering conditions. ### AI Search Similar (Search by Image) An additional feature allows searching by image similarity: * Only available if **AI Search** is enabled for the project * Triggered through the **image context menu** * Additionally, developers can access this functionality **programmatically** via API
### Batch AI Search Similar You can also search for similar images using multiple selected images as reference: 1. Select multiple images using checkboxes in the project 2. Click **"With N Selected"** button 3. Choose **"AI Search Similar images"** from the dropdown menu 4. The system will find images most similar to your selected set This is useful when you want to find images that are semantically similar to a group of reference images rather than just one.
### Visual Embedding Explorer {% hint style="warning" %} **Private Beta Feature**\ Visual embedding exploration in 2D and 3D space is currently available in **private beta**. This feature allows you to visualize your dataset's semantic structure and interactively explore image clusters in dimensional space. {% endhint %} The Visual Embedding Explorer provides an interactive way to understand your dataset's semantic organization: * **2D/3D Visualization**: View your images plotted in reduced dimensional space based on their CLIP embeddings * **Interactive Clustering**: Explore natural groupings and identify outliers visually * **Zoom and Navigate**: Pan, zoom, and rotate through the embedding space to discover patterns * **Click-to-Search**: Click on any point to find similar images in that semantic region This feature is particularly useful for dataset analysis, quality control, and discovering unexpected patterns or edge cases in large image collections.
### Disable AI Search Next to the **AI Search** button, there's a dropdown menu: * **Disable AI Search** * Disconnects the project from AI Search. * The project is removed from the auto-update queue. * Note: Image embeddings are not deleted upon disabling. {% hint style="info" %} Embeddings are automatically refreshed on a schedule (e.g., every few days) if AI Search is enabled. {% endhint %}
# Quality Assurance & Statistics Understanding your data's characteristics is a key aspect of data preparation. The Statistics section provides tools for data analysis, calculation of statistical metrics and data visualization. Supervisely is excited to introduce advanced interactive statistics designed to power dataset analysis and ensure efficient quality assurance (QA) for your custom computer vision training datasets. In this detailed guide, you will: * Explore statistical insights and learn how to engage with them to optimize your data validation process. * Detect hidden anomalies, errors, and annotation outliers. * Improve data acquisition methods, draw the right conclusions on annotation distributions. * Enhance neural network performance and much more. To view statistics for an entire project - that is, across all datasets within the project - select the desired project and, just below the project name, click on the **QA & Stats** tab.
**Discover the best interactive dataset statistics such as:** 1. [Overview:](#overview) Explore the balance and the distribution of the objects per classes. 2. [Class balance:](#class-balance) Find out the distribution of classes in your dataset. 3. [Co-occurrence matrix:](#co-occurrence-matrix) Explore the relationships between classes. 4. [Image statistics:](#per-image-statistics) Get detailed information about image characteristics and its objects. 5. [Object distribution: ](#object-distribution-heatmap)Analyze the localization of objects in images with their properties. 6. [Class sizes:](#object-class-sizes-and-overall-properties) Analyze the sizes of objects of all annotation classes in your dataset. 7. [Spatial heatmap:](#spatial-heatmap) Visualize the most frequent spatial location of objects and placement patterns in images. 8. [Objects properties](#objects-properties): Compare the characteristics of individual objects. 9. [Image tags co-occurrence: ](#image-tags-co-occurrence)Analyze the relationships between different tags and attributes associated with images. 10. [Objects tags co-occurrence:](#objects-tags-co-occurrence) Explore the co-occurrence of various tags associated with objects to uncover potential correlations. 11. [Class to tags co-occurrence:](#class-to-tags-co-occurrence) Understand the associations between different classes and the tags assigned to their objects. 12. [Categorical tags distribution:](#categorical-tags-distribution) View the occurrence of each value for categorical (OneOf) key-value tags in the dataset. 13. [Other statistic:](#other-statistic) Explore additional properties and metrics specific to your dataset. 14. [Actions with the filtered data:](#actions-with-the-filtered-data) Perform actions like copy, move, delete, and create labeling jobs based on specific data filters. 15. [Example apps:](#example-apps) Use applications such as Classes Stats for Images, Labeling Jobs Stats, and Object Size Stats to gain deeper insight into your data. ## Overview The charts below present general project statistics. The chart on the left shows the balance between annotated and unlabeled images, while the chart on the right shows the distribution of objects per class. Click on a section to preview images from that partition.
## Class Balance A proper understanding of the class distribution in your dataset is essential for effective model training and evaluation. Class distribution analysis highlights the prevalence and rarity of different annotation classes, enabling you to address class imbalance directly impacts the performance and generalization capacity of neural networks. Class imbalance occurs when some classes are over-represented while others are under-represented in the dataset. This imbalance can lead to biased model training, where the neural network performs better on common classes and poorly on rare ones. By analyzing and adjusting the class distribution, you can ensure balanced learning and robust model performance. Class imbalance table helps you to adjust and modify data sampling algorithms during training to automatically normalize classes distribution. * Click on any row to explore all images containing objects of the selected class. * Use the search function to quickly filter and find specific classes. * Sort by specific columns to identify the rarest or most frequent classes.
**Components**: The class statistics table provides a comprehensive overview of the dataset. Here's a breakdown of each column in the table: **Class**: Lists the annotation classes present in the dataset. Datasets can contain a large number of classes, in some cases 1000+. To view all available classes, just scroll the rows. **Images**: Shows the total number of images containing at least one object of the given class. **Objects**: Presents the total number of objects of the corresponding class in the dataset. **Average Count on Image**: Provides the average number of objects per image for a specific class, excluding images without such objects. **Area on Image**: Indicates the average image area occupied by objects of the given class, excluding images without such objects. ## Co-Occurrence Matrix The Co-Occurrence Matrix is an analytical tool that reveals how frequently pairs of classes appear together in the same images within a dataset. It provides insights into potential interactions and correlations between classes, which can be crucial for understanding the dataset's structure and improving model performance. In image datasets, certain classes may co-occur frequently, indicating a semantic or contextual relationship. For instance, in a dataset of animal images, classes like "cat" and "dog" might appear together often due to their common domestic environment. In case of analyzing custom neural network performance, this matrix is called **confusion matrix** - where the rows are actual classes in the ground truth data and the columns are predicted classes. The Confusion Matrix compares the actual classes (ground truth) with the predicted classes by the model. For example, a cell at the intersection of the actual class "cat" (row) and the predicted class "dog" (column) with a value of 7 indicates that the model misclassified 7 "cat" objects as "dog". * Click on any cell in the matrix to access corresponding annotated images containing objects from both classes. This feature facilitates understanding of class interactions by allowing users to explore visual examples of identified relationships.
**Components**: **Matrix Cells:** Each cell in the matrix represents the co-occurrence of two classes. The number within the cell indicates the number of images containing objects of both classes simultaneously. **Tooltip Explanations:** Hovering over a cell reveals a tooltip explanation, making it easy for new users to interpret the matrix values and draw right conclusions. ## Per Image Statistics Analyzing images in the dataset based on the number of annotations for each class provides valuable insights into the distribution and characteristics of object instances. By examining the annotations per image across various classes, users can identify anomalies, edge cases, and patterns that may influence model performance and dataset quality. * Sort the table by any column to identify anomalies or edge cases, such as images with a high number of annotations of specific class or unusual object distributions. * Quickly locate specific images or classes of interest within the dataset. * Clicking on any row in the table allows users to preview the selected image with labels, allowing them to instantly locate an object or class of interest in the dataset.
**Components**: The `Images` table provides detailed information about each image contained in the dataset: **Image** - Each image is identified by a unique name or identifier. **Dataset** - Each dataset is identified by a unique name. **Dimensions** - Height and width dimensions of each image are provided in pixels, aiding in understanding the scale and resolution of the dataset. **Object Information** - Detailed statistics on objects within each image are listed. This includes for every class in a dataset the number of objects presented on the image and the total area covered by those objects. ## Aggregated Statistics per Dataset The table presents aggregated statistics per dataset, including all nested datasets. It displays average values for class area and object count per image, along with general statistics. Click a row to preview dataset images with annotations, including those from nested datasets. Sort by any column to find outliers, like unannotated datasets or datasets with abnormal average object count per image.
**Components**:\ \ The `Datasets` table provides detailed information about each dataset contained in the dataset: **Dataset / ID** – Each dataset is identified by a unique name and an associated identifier (ID), which together help distinguish and reference it within the project. **Size** - Total number of images in the dataset. **Annotated** – Number of images that contain at least one labeled object. **Tagged** – Number of images that have at least one tag assigned. **Objects** – Total number of labeled object instances across all images in the dataset. **Tagged Objects** – Total number of object instances that have one or more tags assigned to them. **Class (Objects per Image)** – The average number of objects of the selected class per image. Only images that contain at least one object of this class are included in the calculation. **Class (Average Area)** – The average area (in pixels) covered by objects of the selected class per image. Only images containing at least one object of this class are considered in the calculation. ## Object Distribution Heatmap The Object Distribution Heatmap offers an interactive visualization of how objects are frequently presented across images for every class in the dataset. This heatmap chart exposes the images with unusual number of objects, facilitating detailed exploration and analysis of class annotations. For example, let's consider the row for class `train` (Y axis) and the column 3 objects (X axis). The cell on the intersection has the value 2 which means that there are only **2 images with 3 objects of class `train` simultaneously** in our training dataset. Once user hovers the mouse cursor over the cell, the helpful tooltip appears on the screen supporting users in interpretability and understanding of the selected value. Clicking on that cell would display a list of all images in your dataset that contain only 3 objects labeled as `train`. This functionality enables users to quickly identify images with the exact number of object for a particular class. * Clicking on a cell in the heatmap displays a list of images containing the specified number of objects for the selected class. This feature allows users to explore specific object distributions and analyze the corresponding images in detail in a thumbnail preview mode or open them in the image annotation toolbox. * Users have the option to download the heatmap chart in various formats, including `SVG`, `PNG`, and `CSV`, for further analysis or documentation purposes.
**Components**: The heatmap chart presents the following axes: **Vertical Y Axis (Classes):** Lists the classes presented in the dataset. **Horizontal X Axis (Number of Objects):** Represents the number of objects on the image (e.g., 0, 1, 2, 3 objects, etc...). ## Object Class Sizes The Class Sizes table provides detailed size properties of objects for each class in the dataset, offering insights into their dimensions and size variations. Users can interact with the table to examine various size metrics and view images with annotations of selected classes, facilitating comprehensive analysis and fast data understanding. * Clicking on a row allows users to view images with all object annotations of the selected class, simplifying visual examination, analysis or correction. * Sortable columns enable users to identify classes with the smallest or largest objects, as well as understand size differences between classes.
**Components**: The table presents the following properties for each class **Class:** Name or identifier of the class. **Object Count:** Total number of objects of the class in dataset. **Avg Area %:** Average object area as a percentage of total image area. **Max Area %:** Maximum object area as a percentage of total image area. **Min Area %:** Minimum object area as a percentage of total image area. **Height (Min/Max/Avg):** Object height presented as number of pixels and percentage of image height. There are three different columns for minimum, maximum, and average values. **Width (Min/Max/Avg):** Object width presented as number of pixels and percentage of image width. There are three different columns for minimum, maximum, and average values. ### Class Area Sizes in Treemap view The Class Area Sizes Treemap offers an alternative visualization method for understanding the properties of object sizes for all classes in the dataset. This interactive chart presents class area sizes in a 2D layout, allowing users to explore the relative proportions of object areas for each class and helps to perform fast visual analysis and inspection. * Hovering on rectangles show the tooltip for easier understanding. * Users can download the Treemap chart in various formats, including `SVG`, `PNG`, and `CSV`, for further analysis, sharing or documentation purposes.
**Components**: The Treemap chart presents the following components: **2D Layout:** Classes are represented as rectangles, with larger rectangles indicating classes with greater average object area sizes. **Color Coding:** Each class rectangle is color-coded for easy identification. **Tooltip Information:** Hovering over a class rectangle displays additional information, such as the class name and average objects area size. ## Spatial Heatmap The heatmaps below show the average spatial location of all objects for each class. These visualizations help to understand how objects are distributed on the images highlighting most common and rare object locations. It helps to analyze the placement of objects in a dataset. These statistics are mostly relevant for spatial datasets for such Computer Vision tasks as object detection, semantic segmentation, instance segmentation, panoptic segmentation, etc. For example,m as you can see on the image below, objects of class `airplane` are mostly located in the center of the image. Knowing that information will help data scientists to configure custom data augmentations to train the model that will be robust to `airplane` locations, and as a result will be able to predict airplanes on any position, not only in the center of the image. * Hovering on heatmap provides detailed insights into spatial object distribution. * Export options allows downloading of heatmaps for detailed study and presentations.
## Objects Properties The Objects table contains all objects and their geometrical properties. Users can preview images with specific label by clicking on a object's row, utilize search and pagination features for navigation, and identify outliers by sorting by specific column. * Clicking on a row opens the related image, allowing users to visually inspect annotations and object properties. * Use the search function to quickly locate specific objects or classes of interest in a dataset. * Sortable columns allow users to identify outliers or patterns in object characteristics, such as extreme height or width values. For example, user can find and inspect the largest and the smallest objects in the entire dataset.
**Components**: The table presents the following information for each object: **Object ID:** Unique identifier assigned to each object. **Class:** Indicates the class (category) to which the object belongs. **Image Name:** By selecting a row, users have access to the associated image, with a preview of the image annotations. **Image Size:** Displays the dimensions of the image in height and width, and allows to compare the object size with respect to the size of the image. **Height (%/px):** Indicates the object's height relative to the image height, expressed as a percentage or pixels. **Width (%/px):** Specifies the width of the object relative to the width of the image, expressed as a percentage or pixels. **Area:** Shows the area occupied by the object, expressed as a percentage. This metric provides insight into the extent of coverage in the image. ## Image Tags Co-Occurrence The co-occurrence matrix for [image and object tags and attributes](https://supervisely.com/blog/mastering-image-tagging/) provides a rich analytical view of the relationships between different tags and attributes associated with both images and objects in a dataset. It provides a comprehensive overview into how different tags and attributes tend to co-occur, shedding light on potential correlations and trends in the data. Also it help to find the images that have both tab "A" and "B". It help on analyzing datasets for image and object classification Computer Vision task. In you work in the settings of classical classification, you may find the mistakes in your datasets - co-occurence matrix has to be diagonal. For the multi-label classification it uncovers the correlation between all tag pairs. Find the example below for the [MVTec Logical Constraints Anomaly Detection dataset](https://datasetninja.com/mvtec-loco-ad). * Each cell in the matrix represents the co-occurrence of two tags or attributes. The numeric value in the cell indicates the number of images in which the pair of tags (attributes) co-occur. * Hovering over a cell provides a tooltip that shows the degree of co-occurrence between the tags (total number of images). * Clicking on any cell in the matrix allows users to view corresponding images containing the selected pair of attributes.
## Objects Tags Co-Occurrence The objects tags co-occurrence matrix provides a detailed analytical view of the relationships between various tags associated with objects in a dataset. It offers a comprehensive understanding of how different tags tend to co-occur, revealing potential correlations and trends in the data. This tool is particularly useful when analyzing datasets for object classification tasks in computer vision. In classical classification settings, errors in your datasets can be identified if the co-occurrence matrix is diagonal. For multi-label classification tasks, the matrix helps uncover correlations between all pairs of tags. For example, in the MVTec Logical Constraints Anomaly Detection dataset, each cell in the matrix represents the co-occurrence of two tags. The numeric value in the cell indicates the number of images in which the pair of tags co-occur. * Hovering over a cell provides a tooltip that shows the degree of co-occurrence between the tags (total number of images). * Clicking on any cell in the matrix allows users to view the corresponding images containing the selected pair of tags. ## Class To Tags Co-Occurrence The class and tag association statistic provides a comprehensive view of the associations between different classes, their objects and tags assigned to the objects. This matrix shows the big picture on which tags are commonly associated with objects of specific classes, allowing users to understand the semantic relationships between the object classes and the descriptive attributes assigned to them. In Computer Vision datasets for object multi-label classification tasks objects on images may have more than one tag assigned, certain tags may frequently co-occur with particular classes, indicating the descriptive attributes commonly associated with those classes. For example, in a dataset of live-stock images, the class `cow` may be associated with tags such as `age`, `size`, `health condition` and `pose type` which reflect the typical characteristics of the cows for animal health and well-being monitoring systems. Class and tag co-occurrence matrix provides a powerful way to explore relationships between objects and the attributes assigned to them. ## Categorical Tags Distribution In Supervisely, you can create [categorical (OneOf) key-value tags](https://supervisely.com/blog/mastering-image-tagging/#types-of-tag-values) and use them in various Computer Vision tasks, such as image retrieval and classification. These tags have the defined set of possible values. For every tag in the dataset, categorical tag distribution chart shows the number of objects and images, that contain this tag with its specific value. This chart provides a clear view of how frequently different values of the categorical tags occur in the dataset. All cells are clickable and the corresponding images with their annotations will be opened in dialog window in thumbnails preview mode. Analyze occurrences of categorical key-value tags (OneOf) assigned to images or objects. The row for every tag includes it's all possible values and shows the number of images or objects containing corresponding tag with this specific value. Note that you can always change the applicability of tags (images or objects) in the `Tags` tab.
## Other Statistic Additionally, apart from the tables mentioned earlier, data scientists can easily spot some basic stats for training data, covering Datasets, Images, Image Tags, Object Tags, Objects, and Object Area. These basic statistics shows general characteristics of the data and annotations.
## Actions with the filtered data These quality assurance tools can be used as a specific advanced visual filters allowing users to quickly explore and subsample data with very specific properties. With dataset filters, statistics and quality assurance tools users can easy manipulate the data, performing different actions on images and their annotations in Supervisely Computer Vision Platform: **copy, more, delete, create labeling jobs, search similar images or assign tags and attributes**. Supervisely is designed as a full-stack solution, thus the combination of: * [conditional dataset filters](https://supervisely.com/blog/advanced-dataset-filters/) * [labeling toolboxes](https://supervisely.com/blog/releasing-new-image-annotation-tool/) * [QA tools](https://supervisely.com/blog/dataset-quality-assurance-and-interactive-statistics/) * [node-based pipelines](https://ecosystem.supervisely.com/apps/data-nodes), * [SDK & APIs](https://developer.supervisely.com/) streamline data management and labeling workflows, facilitate user collaboration and boost the speed of the data labeling, data review and model training loops. As a result, custom pipelines for Active Learning and Continuous Model Improvement can be organized smoothly. ## Example apps * [Classes stats for images.](https://app.supervisely.com/ecosystem/apps/classes-stats-for-images?id=16) Data Exploration for Segmentation and Detection tasks is underestimated by many researchers. The accuracy of your models highly depends on how good you understand data. This app "Classes Stats for Images" generates report with detailed general and per image statistics for all classes in images project. It allows to see big picture as well as shed light on hidden patterns and edge cases * [Labeling Jobs Stats.](https://app.supervisely.com/ecosystem/apps/labeling-jobs-stats?id=20) Report provides both high-level and detailed statistics for all labeling jobs in a team. * [Object Size Stats.](https://app.supervisely.com/ecosystem/apps/object-size-stats?id=18) Data Exploration Tools provide deep understanding of your data and are crucial for building high quality models (better you understand data, more accurate models are). This app generates report with detailed statistics for objects (Bitmap / Rectangle / Polygon, objects of other shapes are ignored) in images project. It allows to see big picture as well as shed light on hidden patterns and edge cases * [Labeling Events Stats.](https://app.supervisely.com/ecosystem/apps/labeling-events-stats?id=23) Supervisely stores full activity log almost for every action. This app uses activity log to restore all labeeling actions in team (table can be huge) and performs some basic aggregations shown on the screenshot below. All tables can be sorted by any column. * [Project 3D statistics.](https://app.supervisely.com/ecosystem/apps/project-3d-stats?id=202) Application generates report with detailed general and per pointcloud statistics for all classes in pointcloud and episodes project. # Practical applications of statistics Learn how to use best quality assurance and interactive statistical tools to perfect your custom training datasets and improve neural network performance. In this guide, we explore how to leverage best-in-class quality assurance and interactive statistical tools to improve the quality of your custom training datasets. These tools are crucial for identifying and rectifying data issues, such as class imbalances, annotation errors, and outliers, which can significantly impact the performance of neural network models. The guide includes practical applications of statistics, providing insights on data validation, anomaly detection, and optimizing the data acquisition process. ## Use Case 1: Missing or misclassified annotations When analyzing the class `dog`, it is found that the selected class is presented on 12 images. However, when reviewing these images, it is found that there are no actual objects related to this class in one image. This may indicate problems with the clarity of the annotation or insufficient quality of the data labeling - annotators put the bounding box of class "dog" but there are no dogs on the image.
Steps to solve using Class Balance and Image Statistics 👇 #### **Step 1. Class Balance Analysis** 1. Review the distribution of objects across different classes to identify any anomalies. For example, check the frequency of the "dog" class. 2. Compare the total number of objects labeled as "dog" with the expected number based on the visual review. #### **Step 2. Check Images Statistic** 1. Use the per image statistics to find and analyze images containing the "dog" class. Verify the presence and correctness of the annotations. 2. Look for images that either lack the annotated object or where the annotation is incorrect (e.g., wrong bounding box placement). Sorting images by the number of objects or area covered can help identify these inconsistencies. #### **Step 3.** Make corrections and provide feedback 1. Use the information from the tables to document specific cases of misclassification or missing annotations. 2. Provide feedback to the data annotation team for corrections and improve future annotations. #### Step 4. Update the annotations 1. Based on the issues identified, update the annotations to accurately match the objects in the images. ## Use Case 2: Mismatch in the number of objects When sorting the images by the Object Distribution for the class `skis`, it is found that all of images have 4 objects in this class. However, a detailed view of some images reveals that they do not contain such a large number of objects of the class `skis`. This may indicate errors in objects segmentation (one object is partially covered and labeled as two separate masks), objects duplication or a mismatch between the actual content of the images and their annotations.
Steps to solve Using Object Distribution and Co-Occurrence Matrix 👇 #### **Step 1. Object Distribution Heatmap** 1. Check the heatmap to see the distribution of the "skis" class objects across images. Look for inconsistencies, such as images annotated with more objects than actually present. 2. Click on specific cells to view images with annotated object counts and verify these counts visually. #### **Step 2. Co-Occurrence Matrix** 1. Analyze the co-occurrence of the "skis" class with other classes. Unusual or unexpected co-occurrences may indicate annotation errors or misunderstandings during the labeling process. #### **Step 3. Identify and correct mistakes** 1. Record any discrepancies found during the analysis. 2. Provide detailed feedback to the data annotation team about identified issues, such as over-segmentation or duplicate annotations. #### **Step 4. Review and update annotations** 1. Re-annotate the images where discrepancies were found, ensuring that the number of objects is accurate. ## Use Case 3: Errors in model predictions Quality assurance tools can be used to quickly expose and review anomalies in model predictions. When reviewing objects co-occurrence matrix, data scientists can investigate unusual object pairs: for example `horse` and `chair` are presented on the image at the same time. It may help to find errors in training data. Or this may indicate that the model is not working properly or that there is a lack of training data for certain classes. The purpose of analyzing the statistics for model predictions is to debug model mistakes and to determine what data needs to be added to training dataset of how training data augmentations can be improved. For example, having found a lack of images with objects of the class `chair`, a decision can be made to expand the training dataset by acquiring and labeling more images for this class to achieve a better class balance distribution and quality of the model.

Green arrow - model correctly detected the chair on the left image. Yellow arrows - errors in predictions, objects are not chairs on the right image.

Steps to Solve Using Co-Occurrence Matrix and Spatial Heatmap 👇 #### **Step 1. Co-Occurrence Matrix Analysis** 1. Use the co-occurrence matrix to identify unusual pairs of predicted and actual classes, such as a high frequency of "horse" and "chair" co-occurrences which may not make semantic sense. 2. Pinpoint cells in the confusion matrix where the model frequently confuses certain classes, indicating potential issues in the training data or model architecture. #### **Step 2. Spatial Heatmap** 1. Examine the spatial heatmap to understand where objects are typically located in the images. This can reveal biases in the model's learning, such as consistently predicting objects in certain areas. #### **Step 3. Data Analysis and Augmentation** 1. Use the insights from the matrices to review the training data for underrepresented classes or missing annotations. 2. Plan data augmentation techniques to address class imbalance or introduce variability in object locations to improve model generalization. #### **Step 4. Model Retraining and Evaluation** 1. Based on the analysis, augment the dataset or collect additional data to balance class representation. 2. Train the model with the updated dataset and evaluate its performance using standard metrics like precision, recall, and F1-score. #### **Step 5. Continuous Monitoring** 1. Regularly review the co-occurrence matrix and other statistics to identify and rectify new issues, ensuring ongoing model robustness and accuracy. # Project Settings Configure project-level settings in Supervisely, including the labeling interface and Read-only mode to protect your data from accidental changes. Every project in Supervisely has a dedicated **Settings** tab where you can configure the labeling interface and control how the project behaves in the UI. The available settings depend on the project type — Images projects have an extended set of options split across two toolbox tabs, while Videos projects have a single, focused settings page. ## Accessing Settings Open any project and click the **Settings** tab in the top navigation bar. * For **Images** projects, the tab is divided into two sections: **Basic Labeling Toolbox** and **Advanced Labeling Toolbox**. * For **Videos** projects, all settings are available on a single page.
Project Settings tab
*** ## Images Project ### Basic Labeling Toolbox The Basic Labeling Toolbox is organized into four sub-tabs: **Scene**, **Axes**, **Tags**, and **Visuals**. #### Scene * **Show grid** — overlays a grid on the image to help with precise labeling. * **Grid rows count** — number of grid rows (default: 3). * **Grid columns count** — number of grid columns (default: 3). * **Zoom multiplier** — controls how much the mouse wheel affects the zoom level (default: 1.1). * **Default polygon tool option** — default mode when the Polygon tool is selected (e.g., Consecutive). * **Use "Erase underlaying pixels" option** — pre-enables the "Erase underlying pixels" option when the Bitmap tool is selected. * **Keep zoom factor** — preserves the current zoom level when switching between images. * **Show keypoints labels mode** — controls when keypoint labels are visible: Always, On hover, or Never. * **Object dashed borders** — objects carrying specified tags are rendered with a dashed border. * **Hook on image change** — a JavaScript hook that fires whenever the active image changes. * **Open properties when edit** — automatically opens the image/object properties panel when editing begins. * **Transparent parts color** — color used to fill transparent areas of an image (default: #FFFFFF). * **Show point position** — displays the cursor coordinates while dragging a point. * **Use only original images** — disables image conversion and compression. * **Default Smart Model** — the Smart Tool model pre-selected in the annotation toolbox. #### Axes * **Enable axes on rectangle** — shows auxiliary crosshair lines while drawing a rectangle. * **Enable axes on smart object** — shows auxiliary crosshair lines while using the Smart tool. * **Enable axes on graph** — shows auxiliary crosshair lines while creating a graph (skeleton) object. * **Axes opacity** — opacity of the auxiliary crosshair lines. * **Axes border size** — thickness of the auxiliary crosshair lines. * **Axes main color** — color of the primary auxiliary line. * **Axes additional color** — color of the secondary auxiliary line. #### Tags * **Category separator** — a character used to split tag names into hierarchical categories. * **Show tags mode** — when to display tags when no tool is active: Never, On hover, or Always. * **Tags location over objects** — where tags appear relative to the object: Top point or Top left point. * **Display class** — shows the object class as a tag label. * **Display author** — shows the user who created the object as a tag label. * **Display tag author** — shows the user who created a tag. * **Display object size** — shows the size of the object as a label. * **Multiple tags mode** — allows the same tag to be applied to an object more than once. * **Toggle tags** — pressing a tag hotkey again removes the tag instead of adding a duplicate. #### Visuals * **Border size** — border width for all visible figures in pixels (or `auto`). * **Show bitmap contours** — toggles the visibility of bitmap mask contours. * **Polygon edit fill** — fills the polygon area while it is being drawn or edited. * **Points size** — size of annotation points in pixels. * **Point shape radius** — radius of the point shape in pixels. * **Rectangle opacity** — fill opacity for rectangle objects. * **Rectangle border size** — border thickness for rectangle objects. * **Default opacity** — default fill opacity applied to all figures. * **Default brush size** — default radius of the brush tool. * **Project labeling interface** — selects a specialized labeling interface tailored to a specific industry or annotation scenario. * **Group Images mode** — enables grouping of images by a selected tag. * **Group Images by Tag** — the tag used to group images (only active when Group Images mode is enabled). * **Group Images sync mode** — synchronizes pan, zoom, and figure visibility across images in a group. * **Show all objects from the group on sync** — displays all objects from the entire group instead of only those belonging to the current image; requires sync mode to be enabled. * **Show hotkeys hint** — shows a hotkey reference overlay in the labeling toolbox. ### Advanced Labeling Toolbox * **Project labeling interface** — selects a specialized labeling interface tailored to a specific industry or annotation scenario. * **Hide unavailable tools** — hides disabled annotation tools from the toolbar instead of showing them as inactive. * **How to handle mask overlaps in brush tool** — defines behavior when brush strokes overlap existing masks: * **Overlay (default)** — draws on top of existing masks without modifying them. * **Overwrite** — replaces the pixels of existing masks where they overlap. * **Preserve** — draws only on unpainted areas; existing masks are never changed. * **Skip boundary validation checks** — allows drawing outside the image boundaries; also affects API calls. * **Read-only project** — restricts all editing actions in the UI (upload, delete, context menus). Does not affect API methods. *** ## Videos Project Videos projects have a single **Settings** page without the Basic/Advanced split. * **Project labeling interface** — selects a specialized labeling interface tailored to a specific industry or annotation scenario. * **Hide unavailable tools** — hides disabled annotation tools from the toolbar instead of showing them as inactive. * **Show object trajectories** — displays the movement trajectories of annotated objects across frames. Useful when labeling footage from a static CCTV camera. * **How to handle mask overlaps in brush tool** — defines behavior when brush strokes overlap existing masks: * **Overlay (default)** — draws on top of existing masks without modifying them. * **Overwrite** — replaces the pixels of existing masks where they overlap. * **Preserve** — draws only on unpainted areas; existing masks are never changed. * **Skip boundary validation checks** — allows drawing outside the frame boundaries; also affects API calls. * **Read-only project** — restricts all editing actions in the UI (upload, delete, context menus). Does not affect API methods. *** ## Volume Project Volume projects have a single **Settings** page without the Basic/Advanced split. * **Hide unavailable tools** — hides disabled annotation tools from the toolbar instead of showing them as inactive. * **How to handle mask overlaps in brush tool** — defines behavior when brush strokes overlap existing masks: * **Overlay (default)** — draws on top of existing masks without modifying them. * **Overwrite** — replaces the pixels of existing masks where they overlap. * **Preserve** — draws only on unpainted areas; existing masks are never changed. *** ## Point Cloud and Episodes Project Point Cloud and Episodes projects have a single **Settings** page without the Basic/Advanced split. * **Hide unavailable tools** — hides disabled annotation tools from the toolbar instead of showing them as inactive. *** ## Read-only mode {% hint style="info" %} Images and Videos only {% endhint %} The **Read-only project** setting is available for both Images and Videos projects. It puts the project into a protected state that prevents accidental edits through the UI while keeping full access for viewing and exporting data.
Read-only project list
### What you can do in Read-only mode * Browse the project in the **project panel** — gallery, table view, filters, AI Search, QA & Stats — exactly as with a regular project. * Open images or videos in the **labeling toolbox** to visualize annotations. * **Clone** the project to another workspace. * **Download** and export data using ecosystem apps.
Read-only project page
### What is restricted in Read-only mode All UI actions that modify the project or its data are disabled, including: * Uploading or deleting images, videos, or datasets. * Editing, adding, or removing annotations. * Renaming or deleting the project and datasets. * Context menu actions that change data state.
Read-only project labeling tool
{% hint style="info" %} **Note:** Read-only mode only restricts the UI. API access is not affected — programmatic operations via the Supervisely SDK or REST API remain fully available. {% endhint %} ### Enabling and disabling Read-only mode
Read-only project settings
**For Images projects:** 1. Open the project and navigate to **Settings → Advanced Labeling Toolbox**. 2. Check the **Read-only project** checkbox. 3. Click **Save**. **For Videos projects:** 1. Open the project and navigate to **Settings**. 2. Check the **Read-only project** checkbox. 3. Click **Save**. To allow editing again, uncheck the **Read-only project** checkbox and click **Save**. You can also toggle Read-only mode programmatically using the [`set_read_only`](https://supervisely.readthedocs.io/latest/sdk/supervisely.api.project_api.ProjectApi.html?h=set_read#supervisely.api.project_api.ProjectApi.set_read_only) method of the Supervisely Python SDK. {% hint style="success" %} **Tip:** Use Read-only mode to lock down a project after reaching a stable state, so collaborators can safely browse and export data without the risk of accidentally modifying it. {% endhint %} # Advanced # Custom Data Explore how to store and manage technical metadata, configurations, and integration settings using Custom Data in JSON format. ## Overview Custom Data allows you to store additional technical information in JSON format for projects and datasets. This feature enables you to save configuration parameters, metadata, and settings that can be used by Supervisely Apps and custom tooling. ## Accessing Custom Data Navigate to any project or dataset and click the **Info** tab. You'll find two sections: * **README** - Documentation in Markdown format * **Custom Data** - Technical data in JSON format
## When to Use Custom Data Use Custom Data when you need to: * Store app configuration parameters * Save processing results and metadata * Share settings between different applications * Keep references to external resources Use README for project descriptions, annotation guidelines, and human-readable documentation. ## Common Use Cases ### Training Configuration Store model training parameters for training apps: ```json { "training": { "epochs": 100, "batch_size": 16, "learning_rate": 0.001 }, "model": { "architecture": "YOLOv8", "input_size": [640, 640] } } ``` ### Export Settings Define default export parameters: ```json { "export": { "format": "COCO", "include_masks": true, "min_object_area": 100 } } ``` ### External Integration Store references to external systems: ```json { "integration": { "mlflow_experiment_id": "exp_12345", "external_project_id": "proj_abc789" } } ``` ### Processing Results Save processing outcomes and metrics: ```json { "last_validation": { "date": "2024-03-15", "total_images": 1500, "issues_found": 8, "quality_score": 0.94 } } ``` ## Data Structure Keep your JSON structured and include version information: ```json { "version": "1.0", "created_by": "username", "config": { // Your configuration here } } ``` ## Editing and Retrieving Custom Data There are two ways to work with Custom Data: ### 1. Web Interface The web interface is ideal for static settings that don't change frequently during data processing: 1. Navigate to your project or dataset 2. Click on the **Info** tab 3. Find the **Custom Data** section 4. Click **Edit** to modify the JSON 5. Save your changes
### 2. Python SDK For dynamic data that needs to be updated programmatically, use the Python SDK: #### Retrieving Custom Data for a Project ```python import supervisely as sly # Initialize API api = sly.Api() # Get project custom data project_id = 12345 custom_data = api.project.get_custom_data(project_id) print(custom_data) ``` #### Updating Custom Data for a Project ```python import supervisely as sly # Initialize API api = sly.Api() project_id = 12345 # Get current custom data custom_data = api.project.get_custom_data(project_id) # Modify the data custom_data["processing_status"] = "completed" custom_data["processed_items"] = 150 # Update the custom data api.project.update_custom_data(project_id, custom_data) ``` #### Silent Mode for Projects When updating project custom data, you can use silent mode to prevent updating the project's modification timestamp: ```python # Update without changing the project's updated_at timestamp api.project.update_custom_data(project_id, custom_data, silent=True) ``` This is useful when you want to update technical metadata without marking the project as recently modified. #### Working with Dataset Custom Data For datasets, retrieve custom data through the dataset info: ```python # Get dataset info dataset_id = 67890 dataset_info = api.dataset.get_info_by_id(dataset_id) # Access custom data custom_data = dataset_info.custom_data # Modify only the keys you need to update custom_data["validation_score"] = 0.95 custom_data["last_checked"] = "2024-03-20" # Update dataset custom data api.dataset.update_custom_data(dataset_id, new_custom_data) ``` ## Best Practices * Use clear, descriptive key names * Group related settings together * Include version numbers for configurations * Keep file size under 1MB * Store sensitive data externally, use references only ## Limitations * Must be valid JSON format * Visible to all users with project access * No comments supported in JSON * Recommended size limit: 1MB ## How Apps Use Custom Data Supervisely Apps can read custom data to: * Configure their behavior based on stored settings * Resume processing from saved state * Share parameters between different workflow steps * Store results for later reference This enables building automated workflows where configuration and state persist across different operations. # Validation Schemas ## Overview **JSON Schema** Validation allows you to enforce a consistent structure for image metadata when uploading images to projects. By defining a **JSON schema** at the project level, you ensure that all uploaded metadata adheres to the required structure. If the metadata does not match the schema, the upload will be rejected with a clear validation error, preventing inconsistent or incomplete data from entering your project. ## Use Case Example Imagine you're uploading images to a project where each image needs specific metadata: * Camera settings (ISO, aperture, shutter speed) * Location information (GPS coordinates, address) * Quality metrics (brightness, contrast, sharpness) Without validation, different team members might upload images with inconsistent metadata structure. With JSON schema validation, you ensure all metadata follows the same format. ## Setting Up Validation ### Step 1: Define Your Schema Create a JSON schema that defines the required structure for your image metadata. For example: ```json { "type": "object", "required": ["camera", "location", "quality"], "properties": { "camera": { "type": "object", "required": ["iso", "aperture"], "properties": { "iso": {"type": "number"}, "aperture": {"type": "string"}, "shutter_speed": {"type": "string"} } }, "location": { "type": "object", "required": ["lat", "lng"], "properties": { "lat": {"type": "number"}, "lng": {"type": "number"}, "address": {"type": "string"} } }, "quality": { "type": "object", "properties": { "brightness": {"type": "number"}, "contrast": {"type": "number"} } } } } ``` ### Step 2: Set Schema for Project Apply the schema to your project using the API: ```python import supervisely as sly api = sly.Api.from_env() project_id = 12345 # Set validation schema for project api.project.set_validation_schema(project_id, schema) # Get current validation schema current_schema = api.project.get_validation_schema(project_id) ``` ### Step 3: Upload Images with Validation When uploading images, enable validation to ensure metadata compliance: ````python # Upload images with validation enabled image_paths = ["/path/to/image1.jpg", "/path/to/image2.jpg"] names = ["image1.jpg", "image2.jpg"] metas = [ { "camera": {"iso": 800, "aperture": "f/2.8"}, "location": {"lat": 37.7749, "lng": -122.4194} }, { "camera": {"iso": 400, "aperture": "f/1.8"}, "location": {"lat": 40.7128, "lng": -74.0060} } ] api.image.upload_paths( dataset_id=dataset_id, names=names, paths=image_paths, metas=metas, validate_meta=True, # Enable validation use_strict_validation=False # Optional: strict mode ) ``` Valid metadata example: ```json { "camera": { "iso": 800, "aperture": "f/2.8", "shutter_speed": "1/60" }, "location": { "lat": 37.7749, "lng": -122.4194, "address": "San Francisco, CA" }, "quality": { "brightness": 0.7, "contrast": 0.8 } } ```` ## Validation Options ### Standard Validation ```python # Standard validation - allows extra fields api.image.upload_paths( dataset_id=dataset_id, names=names, paths=image_paths, metas=metas, validate_meta=True, use_strict_validation=False # Default Value ) ``` ### Strict Validation ```python # Strict validation - exact schema match required api.image.upload_paths( dataset_id=dataset_id, names=names, paths=image_paths, metas=metas, validate_meta=True, use_strict_validation=True # Set to True for strict validation ) ``` ### Optimized Validation with Caching ```python # Use caching for better performance with multiple uploads api.image.upload_paths( dataset_id=dataset_id, names=names, paths=image_paths, metas=metas, validate_meta=True, use_caching_for_validation=True # Schema cached for 1 hour ) ``` ## Validating Existing Projects For projects with existing images, you can validate all current data: ```python # Validate all existing images in project validation_result = api.project.validate_entities_schema(project_id) # Check validation results if not validation_result: print("All entities are valid according to the schema!") else: print(f"Found {len(validation_result)} entities that don't match the schema:") for entity in validation_result: print(f"\nEntity: {entity['entity_name']} (ID: {entity['entity_id']})") if entity['missing_fields']: print(f" Missing fields: {', '.join(entity['missing_fields'])}") if entity['extra_fields']: print(f" Extra fields: {', '.join(entity['extra_fields'])}") ``` This process helps you: * Identify non-compliant images in existing projects * Get detailed error reports for each failed image * Fix metadata issues before enforcing strict validation ## Benefits * **Consistency**: All images have the same metadata structure * **Quality Control**: Prevent incomplete or incorrect metadata uploads * **Team Coordination**: Everyone follows the same metadata standards * **Data Integrity**: Maintain clean, structured datasets * **Error Prevention**: Catch metadata issues at upload time ## Common Schema Patterns ### Simple Required Fields ```json { "type": "object", "required": ["timestamp", "source"], "properties": { "timestamp": {"type": "string"}, "source": {"type": "string"} } } ``` ### Nested Structures ```json { "type": "object", "required": ["equipment"], "properties": { "equipment": { "type": "object", "required": ["camera_model"], "properties": { "camera_model": {"type": "string"}, "lens": {"type": "string"} } } } } ``` ### Optional Fields with Defaults ```json { "type": "object", "required": ["image_id"], "properties": { "image_id": {"type": "string"}, "quality_checked": {"type": "boolean", "default": false}, "notes": {"type": "string"} } } ``` ## Complete Example Here's a full workflow example: ```python import supervisely as sly # Initialize API api = sly.Api.from_env() project_id = 12345 dataset_id = 67890 # 1. Define schema schema = { "type": "object", "required": ["camera", "location"], "properties": { "camera": { "type": "object", "required": ["iso", "aperture"], "properties": { "iso": {"type": "number"}, "aperture": {"type": "string"} } }, "location": { "type": "object", "required": ["lat", "lng"], "properties": { "lat": {"type": "number"}, "lng": {"type": "number"} } } } } # 2. Set schema for project api.project.set_validation_schema(project_id, schema) # 3. Upload images with validation image_paths = ["/path/to/image1.jpg"] names = ["image1.jpg"] metas = [{ "camera": {"iso": 800, "aperture": "f/2.8"}, "location": {"lat": 37.7749, "lng": -122.4194} }] try: api.image.upload_paths( dataset_id=dataset_id, names=names, paths=image_paths, metas=metas, validate_meta=True ) print("Upload successful - metadata valid!") except Exception as e: print(f"Upload failed: {e}") ``` ## Best Practices * **Start simple**: Begin with basic required fields, add complexity gradually * **Document your schema**: Include field descriptions and examples * **Test thoroughly**: Validate your schema with sample data before deployment * **Version your schemas**: Track changes when updating validation rules * **Communicate changes**: Inform team members about new validation requirements ## Requirements * Supervisely instance version: 6.12.5 or later * Supervisely Python SDK: 6.73.228 or later # MLOps Workflow The main features, capabilities and usage recommendations of MLOps Workflow are described in this documentation. MLOps Workflow streamlines the machine learning lifecycle, focusing on data version control, reproducibility, and collaboration. It provides an easy-to-use visual interface that integrates data management, experiment tracking, and model evaluation. #### Why do you need trial tracking and version control? **Data evolution:** Datasets change over time, and tracking these changes is critical to maintaining model accuracy. **Avoid confusion**: Repeated iterations can lead to errors if teams lose track of which data and model versions were used. **Reproducibility:** Proper tracking ensures that results can be reproduced and shared with team. ## 1. Data Version Control Data version control simplifies managing changes in datasets throughout the project lifecycle. Learn more about [Project Versions](/data-organization/project-dataset/project-versions). **Capabilities:** * Track changes and create backups. * Restore previous states with a single click. * Centralized tracking of data evolution. **How It Works:** * Create versions at any stage - data upload, annotation, or transformation. * Each version is stored in a secure binary format, avoiding file duplication. * Restore data by creating a new project version from a previous state. > **Note:** Available only on Pro and Enterprise plans. ## 2. MLOps Workflow Visualization The visual map shows the entire data lifecycle, including the apps and operations that modify it. **Key Features:** * **Data Operations:** * Augmentation (cropping, rotation, adding noise, etc.). * Annotation transformation. * Training data generation. * Dataset splitting, merging, and filtering. * **Navigation:** Easily access data versions, projects, and application sessions. **Example:** 1. **Data Import**: Upload and annotate images using tools like Smart Tool. 2. **Version Creation**: Capture the project's state after annotation. 3. **Model Training**: Train YOLOv10 while automatically generating checkpoints and reports. 4. **Deployment**: Apply the model to new datasets or export results. ## 3. Model Benchmarking The platform generates automatic reports to evaluate model performance using metrics such as: * mAP, Precision, Recall. * Inference speed. * Classification accuracy and IoU. **Benefits:** * Compare model versions to identify improvements. * Understand how architecture or hyperparameter changes affect results. ## Building a Workflow #### **Best Practices:** 1. **Use Projects Instead of Individual Datasets**: This improves Workflow readability. 2. **Optimize Nodes**: Combine file cards into folders to reduce redundancy. 3. **Add Descriptions**: Provide context for each Workflow element. 4. **Avoid Overwriting Node States**: Maintain a clear history of changes. 5. **Organize Session-Based Applications**: Prevent clutter and simplify Workflow structure. ### **Practical Usage** #### **Workflow Entry Points:** * Project context menu. * Task context menu. * Workspace options. ## **Integrating Workflows into Applications:** Detailed instructions for integrating Workflows into your custom applications are available on the **Supervisely Developer Portal**. # Team Files In our platform, each project has its own data storage area known as **Team Files**. This feature serves as a convenient and secure repository for storing and organizing all the necessary files and information your team requires. Whether it's project-related documents, datasets, data import history, or neural network training history, the Team Files section provides centralized and easy access to resources. {% hint style="success" %} The security of your team's data is paramount; Team Files keeps it secure with robust access controls and permissions, ensuring that sensitive information is protected. {% endhint %} It is a key tool for managing and organizing files and data within the team workspace. It also manages artifacts generated by applications, users, and processes on the platform and collects information from applications, including data import history and neural network training history initiated by team members.

Team Files page

*** ### Data Storage Team Files allows users to store various types of files - images, videos, annotations, neural network models, reports and more. This makes it convenient to work on projects and applications, as all necessary data is stored in one accessible location for the whole team. *** ### File Organization Team Files supports creating folders for structuring data. You can organize files into logical structures (e.g., by projects or data types), which makes managing large amounts of data easier. For example, you can create a folder for a specific project where all images, annotations, and models related to that project will be stored. #### Context menu of files in Team Files You can right-click on a file or folder. From here (we call this the "context menu") you can perform many important actions related to files, for example: clone, save a file path to launch an application, launch an application so that they immediately download the selected file, download locally or delete files.

Folder context menu

*** ### Collaboration Team Files provides team members with access to all files, which is critical for collaboration and makes it easy to work together on tasks and projects in real time. Any team member can: * Upload and download files. * Organize the data structure. * Use the files in their applications. * Easily access and share project-specific resources. *** ### Application Artifacts Many applications in the Supervisely Ecosystem generate artifacts, such as: * Neural network model checkpoints after training. * Reports on data processing results. * Additional files used by applications for various tasks. These artifacts are automatically stored in Team Files, making it easy to access them for analysis, reuse or loading into other projects. *** ### System Directories Team Files contains important system directories that help you manage temporary files and offline sessions: * **/tmp/supervisely/export:** This directory stores temporary files created during the export process. Once you have exported data, these files will remain here so that you can access them later. These files allow for a smooth export process without affecting the main data storage. * **/offline-sessions:** This directory stores the settings (UI) for all sessions of all applications, including NN trainings and ML pipelines. It is particularly relevant for stopped applications, as it allows you to reopen a previously saved session and view their configurations. To ensure continued access to your sessions and their settings, don't delete this directory, or you'll lose progress. *** ### SDK Integration Team Files is integrated with the Supervisely SDK, allowing users to automate file management tasks. You can programmatically: * Create folders. * Upload and download files. * Organize and manage data using Python scripts. *** ### Version Control The platform includes version control features, allowing your team to track changes and revisions to documents, ensuring transparency and accountability. *** ## Limits Supervisely's tiered file limits offer a range of options that can accommodate everything from hobby projects to enterprise-level workflows, ensuring that teams of all sizes have the resources they need to manage and store their data. ### General file types covered by limits The file limits in each plan apply to a wide variety of files you upload or generate, including: * **Images**: Datasets used for image classification, object detection, or segmentation tasks. * **Videos**: Files for video annotation or tracking. * **Annotations**: JSON, XML, or other format annotations generated or imported for labeling tasks. * **Models and Artifacts**: Neural network checkpoints, reports, and other output files generated from training or processing tasks. ### **Community Free plan** The Community Free plan provides a limit of **10,000 files**. This includes images, videos, annotations, and any other files that you upload or generate within the platform. This limit is ideal for small-scale projects, individual developers, or teams just getting started with AI and machine learning experiments. ### **Pro plan** For teams that require more extensive file storage, the Pro plan offers a limit of **50,000 files**. If your team grows or your projects scale beyond that, you have the flexibility to extend this limit to **100,000 files**. This is particularly useful for teams working with larger datasets, such as those involved in computer vision tasks like object detection, image segmentation, or video analysis ### **Enterprise plan** For organizations that handle extensive data, the Enterprise plan offers **unlimited storage**. This plan is designed for large enterprises, research institutions, or AI development teams that manage massive datasets on a regular basis. For detailed information on features and limitations, visit our [Pricing](https://supervisely.com/pricing/) page. {% embed url="" %} # Disk usage & Cleanup One of the important clarifications. [Team Files](/data-organization/team-files) is an exception and it needs to be cleared specifically from Team Files and the deleted files will not go to the trash bin and they cannot be recovered. ### Disk usage To view disk usage in current team you can visit **Disk usage** page.
Here you can see Projects from all Workspaces in current Team.
### Cleanup When you remove Projects they will be moved to Trash Bin.
All removed Projects from current Team are located here.
To delete or restore an item you need to select it by clicking the check mark next to it. Then click `Delete forever` or `Restore`
# Permanent removal How to permanently prune a Team, Workspace, Project or Dataset: archive first, remove permanently as root, and understand how storage is actually reclaimed. Regular removal in Supervisely is reversible — a removed Project or Dataset goes to the [Trash Bin](/collaboration/admin-panel/server-trash-bin) and can be restored. Permanent removal is the second, irreversible step: it drops the entity from the database and releases the storage behind it. This page explains the model, the API methods for every level, and — most importantly — why disk usage does not drop to its final value the moment a removal finishes. {% hint style="danger" %} Permanent removal cannot be undone. There is no trash bin, no restore, and no backup taken on your behalf. Export anything you might still need before you start. {% endhint %} ## The two-step model Removal is always two steps, at every level: 1. **Archive** — a soft, reversible removal. The entity disappears from the UI and moves to the Trash Bin. Any user with sufficient permissions can do this. 2. **Remove permanently** — a hard, irreversible removal. **Only a root (instance administrator) user can do this.** Permanent removal only accepts entities that are already archived. If you call it on a live entity, the request is rejected — archive it first. | Level | Step 1 — archive (soft) | Step 2 — remove permanently (root only) | | --------- | ----------------------- | --------------------------------------- | | Team | `teams.archive` | `teams.remove.permanently` | | Workspace | `workspaces.archive` | `workspaces.remove.permanently` | | Project | `projects.archive` | `projects.remove.permanently` | | Dataset | `datasets.archive` | `datasets.remove.permanently` | {% hint style="info" %} `projects.remove` and `datasets.remove` still work as deprecated aliases of `projects.archive` and `datasets.archive`. Prefer the `*.archive` names in new integrations. {% endhint %} ## API reference All methods live under `/public/api/v3/` on your instance and authenticate with the `x-api-key` header. See the [API reference](https://api.docs.supervisely.com) for full schemas. | Method | Request body | Returns | | ------------------------------- | ----------------------------------------------------------- | ----------------- | | `teams.archive` | `{"id": 42}` | — | | `teams.remove.permanently` | `{"teamsIds": [42, 43]}` — max 50 ids | `{"taskId": 987}` | | `workspaces.archive` | `{"id": 7}` | — | | `workspaces.remove.permanently` | `{"workspacesIds": [7, 8]}` — max 50 ids | `{"taskId": 988}` | | `projects.archive` | `{"id": 111}` | — | | `projects.remove.permanently` | `{"projects": [{"id": 111}], "preserveProjectCard": false}` | — | | `datasets.archive` | `{"id": 222}` | — | | `datasets.remove.permanently` | `{"datasets": [{"id": 222}]}` | — | | `instance.data.cleanup-unused` | — (root only) | `{"taskId": 989}` | ## Team and Workspace removal runs in the background `teams.remove.permanently` and `workspaces.remove.permanently` return a **task id** immediately and then drain in the background. A single team can hold thousands of projects, files and job artifacts, so the work is deliberately asynchronous. What this means in practice: * The Team or Workspace disappears from the UI as soon as the call returns. * The actual database and storage work continues afterwards. Poll `tasks.info` with the returned `taskId` to follow it. * When the task reaches `finished`, that entity's own data is gone. * If a removal fails, the task reaches a terminal `error` status — it does not retry forever. Inspect the task, resolve the cause, and call the method again (see [idempotency](#notes-and-limitations) below). Project and Dataset removal is synchronous — the call returns when the entity is gone. ## How storage is actually reclaimed This is the part worth understanding before you measure your bucket. Image and video data on a Supervisely instance is stored **instance-globally and reference-counted by content hash**. One stored object can be referenced from any number of Projects, Workspaces and Teams — that is what makes cloning a project cheap and what stops duplicate uploads from consuming space twice. Because of that, permanently removing an entity drops its **references** to the data, not the data itself. The underlying objects are reclaimed once nothing references them any more *and* a grace window has passed. This happens in two waves: **Wave 1 — inline, during the removal.** Data whose last request is older than `REMOVE_IMAGE_REQUESTED_THRESHOLD` (**12 hours** by default) is reclaimed as part of the removal itself. Recently-requested data is deliberately left alone, so that an in-flight download or an open labeling session is not pulled out from under it. **Wave 2 — the unused-data garbage collector.** Everything left over is swept by the instance-wide GC, which runs on a **daily schedule** and can also be triggered on demand with `instance.data.cleanup-unused` (root only). The GC applies a **3-day grace window** before reclaiming an unreferenced object, and it also cleans up orphaned figure geometries left behind by removed annotations. {% hint style="warning" %} Bucket usage does not drop to its final value the instant a removal finishes. Expect the remainder to be released over the following days as the two waves complete. If storage does not shrink immediately, that is the design, not a failure. {% endhint %} ## What each level reclaims Every level releases the data belonging to the entities nested inside it, plus its own artifacts: * **Team** — Team Files (`teams_storage`), labeling materials, python notebooks, task files, custom data, export archives, and labeling job debug backups. Plus everything in its Workspaces. * **Workspace** — models and checkpoints (`models/archives`). Plus everything in its Projects. * **Project / Dataset** — images, videos, point clouds and volume slices (`images/original`, `videos`, `point_clouds`), per-video metadata folders (`videos_meta`), mask and mesh geometries (`figures/geometries`), README images (`assets/projects/images`) and labeling job debug backups (`debug_backups/jobs`). ## Example: pruning a Team end to end The recipe below archives a Team, removes it permanently, waits for the background task, and then triggers the garbage collector to release the shared data. ### Step 1. Archive the Team {% tabs %} {% tab title="cURL" %} ```bash export SERVER_ADDRESS="https://app.supervisely.com" export API_TOKEN="" curl -X POST "$SERVER_ADDRESS/public/api/v3/teams.archive" \ -H "x-api-key: $API_TOKEN" \ -H "Content-Type: application/json" \ -d '{"id": 42}' ``` {% endtab %} {% tab title="Python SDK" %} ```python import supervisely as sly api = sly.Api() team_id = 42 api.post("teams.archive", {"id": team_id}) ``` {% endtab %} {% endtabs %} ### Step 2. Remove it permanently This call requires a root user and returns a task id. {% tabs %} {% tab title="cURL" %} ```bash curl -X POST "$SERVER_ADDRESS/public/api/v3/teams.remove.permanently" \ -H "x-api-key: $API_TOKEN" \ -H "Content-Type: application/json" \ -d '{"teamsIds": [42]}' # {"taskId": 987} ``` {% endtab %} {% tab title="Python SDK" %} ```python response = api.post("teams.remove.permanently", {"teamsIds": [team_id]}) task_id = response.json()["taskId"] print(task_id) # Output: 987 ``` {% endtab %} {% endtabs %} {% hint style="info" %} `teamsIds` and `workspacesIds` accept up to **50** ids per call. Split larger cleanups into batches. {% endhint %} ### Step 3. Poll until the task finishes {% tabs %} {% tab title="cURL" %} ```bash curl -X POST "$SERVER_ADDRESS/public/api/v3/tasks.info" \ -H "x-api-key: $API_TOKEN" \ -H "Content-Type: application/json" \ -d '{"id": 987}' ``` Repeat until `status` is `finished`. A `status` of `error` is terminal — the removal stopped and will not resume on its own. {% endtab %} {% tab title="Python SDK" %} ```python import time while True: status = api.task.get_status(task_id) print(status) if status in (api.task.Status.FINISHED, api.task.Status.ERROR): break time.sleep(5) api.task.raise_for_status(status) ``` {% endtab %} {% endtabs %} ### Step 4. Reclaim the shared data Once the removal task is `finished`, the Team's references are gone. Trigger the garbage collector to sweep whatever is now unreferenced — or simply wait for the daily run. {% tabs %} {% tab title="cURL" %} ```bash curl -X POST "$SERVER_ADDRESS/public/api/v3/instance.data.cleanup-unused" \ -H "x-api-key: $API_TOKEN" \ -H "Content-Type: application/json" \ -d '{}' # {"taskId": 989} ``` {% endtab %} {% tab title="Python SDK" %} ```python response = api.post("instance.data.cleanup-unused", {}) gc_task_id = response.json()["taskId"] ``` {% endtab %} {% endtabs %} Remember that the GC honours the grace windows described above: objects requested within the last 12 hours, and objects that became unreferenced less than 3 days ago, are intentionally left for a later run. Running the cleanup twice in a row will not shorten those windows. ## Projects and Datasets Projects and Datasets follow the same archive-then-remove sequence, and the Python SDK exposes dedicated helpers for them. {% tabs %} {% tab title="Project" %} ```python import supervisely as sly api = sly.Api() project_id = 111 # Step 1: archive (this is what `projects.archive` does) api.project.remove(project_id) # Step 2: permanent removal — root only, batches of up to 50 ids api.project.remove_permanently(project_id) ``` {% endtab %} {% tab title="Dataset" %} ```python import supervisely as sly api = sly.Api() dataset_id = 222 # Step 1: archive api.dataset.remove(dataset_id) # Step 2: permanent removal — root only, batches of up to 50 ids api.dataset.remove_permanently(dataset_id) ``` {% endtab %} {% endtabs %} When passing a list of ids to `remove_permanently`, all ids must belong to the same Team — group them before calling. {% hint style="warning" %} Do not confuse `api.project.remove_permanently()` with the SDK's `api.project.archive(id, archive_url)`. The latter is an unrelated legacy method that offloads a project to an external backup archive; it is not the `projects.archive` soft-removal step described on this page. {% endhint %} ## Notes and limitations * **The admin Team (id `1`) cannot be removed.** Attempts to archive or permanently remove it are rejected. * **Permanent removal is idempotent.** Ids that are already removed are silently skipped, so it is safe to retry a batch after a partial failure or a task that ended in `error`. * **Only archived entities can be removed permanently.** The one exception is `projects.remove.permanently` with `preserveProjectCard: true` — see below. * **`preserveProjectCard: true` is a different operation.** Instead of deleting the project, it keeps the project card in place and drops only its data. Use it when you want to retain the project's identity, history and place in the UI while releasing its storage. Because the project itself survives, this variant does **not** require the project to be archived first. ## See also * [Disk usage & Cleanup](/data-organization/storage) — the UI view of storage per Team and Project * [Server trash bin](/collaboration/admin-panel/server-trash-bin) — restore or delete archived entities from the admin panel * [Server cleanup](/collaboration/admin-panel/server-cleanup) — find large and unused entities to prune * [Storage Cleanup](/enterprise-edition/advanced-tuning/cleanup) — instance-level cleanup settings and troubleshooting # Operations with Data The **Operations with Data** section is a dedicated area within our platform where you can perform a variety of data-related tasks and operations. It provides a range of tools and features to streamline your data management processes, ensuring efficiency and organization in handling your valuable data assets. It's also your hub for efficient data handling, from data import to export, annotation, and analysis. It empowers you to manage your data assets effectively and enhance your data-driven workflows. Details about your data and the operations performed on it can be accessed by following our [link](https://app.supervisely.com/ecosystem/data-operations) and utilizing the tools within the **data operations** section. This section provides data management capabilities from import to export, annotation, and analysis. It empowers you to efficiently handle your data and enhance your data-driven workflows.
Data FiltrationUnderstanding your data's characteristics is a key aspect of data analysis and preparation./pages/d0DfhyIvJnT4DMnuILsa
AugmentationsData augmentation is a crucial step in preparing data for machine learning./pages/WFN8j8lTZ8vpRqAieYqO
Converting & Splitting dataConverting data into different formats and splitting it into training and testing sets are essential operations in data preparation./pages/WkKMiiiR2plvij4HVwvc
Pipelines
Easily combine data management, augmentation, filtering, and neural network operations with drag-and-drop nodes system.
# Data Filtration Filter, create records, and process information with confidence. Your data is always under your control. Advanced filters are designed to improve the management, searching, and querying of your custom vision training datasets. This guide will help you understand how to set up and use advanced filters to streamline your data preparation and annotation workflows. #### Key Features 1. **Ease of use:** Suitable for all levels, from beginners to advanced data scientists. 2. **Real-time operation:** Instant results for datasets of any size. 3. **Optimized for large datasets:** Handles datasets with millions of images and annotations. 4. **Quick data preview:** Offers main statistics for quick insights. 5. **Flexible filtering:** Allows filtering based on various criteria like annotation classes, tags, image statuses, and more. 6. **Predefined presets:** Quickstart with predefined filters for common tasks. 7. **Actionable results:** Perform actions on filtered results such as copy, move, delete, create labeling jobs, and annotate.
**Sort by Last Update:** Identify recently modified images. **Sort by Object Number:** Find images with a large number of objects or none at all. **Filter by Dataset:** Preview images from specific datasets. **Search by Name:** Use regular expressions to search for images by name. ## What do filters include?
### Custom Filters **Images Name**: Find images whose names begin with specific characters. **Images Tag:** Search for images that include specific tags. **Images Tags Author:** Find images tagged by a particular author. **Images Without Annotations:** Locate images without annotations or labels. **Objects Class:** Search for objects by their classes. **Objects Tag:** Find objects that include specific tags. **Objects Author:** Locate objects labeled by a specific author. **Issues:** Find images with annotation problems. **Labeling Job:** Search for images involved in labeling jobs with specific statuses. ### Predefined Filters **All Images**: Display all images in the project. **With One Object or More:** Find images with at least one object. **Unlabeled:** Locate images without annotations. **Has Issues:** Find images with annotation problems. **Labeled by Me:** Find images labeled by the current user. *** Also using apps, you can set up filtering quickly and easily, ensuring accuracy when working with a variety of information. ## Example apps: * [Filter images.](https://app.supervisely.com/ecosystem/apps/filter-images?id=187) App filters images from a project and allows you to copy, move, delete images and assign or remove tags. * [Prompt-based Image Filtering with CLIP.](https://app.supervisely.com/ecosystem/apps/prompt-based-image-filtering?id=249) This app allows you to quickly and easily filter and rank images in Supervisely datasets by text prompts. It uses CLIP model to predict the relevance of images to the given text prompt. This app can be useful for filtering or ranking images in a dataset by their content. The relevance (CLIP score) of each image to the given prompt will be shown in a table. The user can choose to filter or sort images by relevance or do both at the same time and then upload images to a new dataset. # How to use advanced filters ### **Open the project and navigate to the Filter tab** Open the images project you are working on and navigate to the Filter tab. Here, you can search among all images, subsample the desired dataset, configure filters, and preview filtering results.
### **Create custom conditional filters** Click the `Filter` button to open a modal window where you can customize your filters based on various criteria. Fine-tune your view by including or excluding specific criteria, such as conditions on the number of objects. For example, you can filter images with the tag validation that contain more than 5 objects of the class *plant*. #### Let's create custom filters step-by-step 1. **Open filter configuration**: Click the `Filter` button to open the filter configuration modal. 2. **Set criteria**: Choose criteria based on datasets, names, image tags, object classes, assignees, issues, or labeling job status. 3. **Fine-tune filters:** Include or exclude specific conditions, such as the number of objects within a specified range. 4. **Apply filters:** Click the `Apply` button to retrieve and preview the data that meets your specified conditions.
*** ### Video tutorial In this 3-minute video tutorial, you will learn how to use advanced conditional filters for Computer Vision datasets, helping you search, filter, and explore annotated images of any size. {% embed url="" %} ## **Tracking image status in labeling jobs** Users can easily view the status of each image within a labeling job, simplifying the monitoring of annotation progress. By clicking on the status of a labeling job, users can access statistics on job activity, labeling time per object, and total labeling time. Detailed statistics are available for individual images, including labeler time in the tool, editing durations, and object counts. This comprehensive information ensures thorough tracking and analysis of the annotation process, enabling managers to accurately assess their annotation workflow.
## **Labeling job management** When selecting multiple images, users can quickly create labeling jobs or delete unnecessary data, allowing for more precise data selection for labeling.
## **Data operations: copy and move** Users can easily copy or move images with annotations from filtering results to other datasets, streamlining training data management. ## Use cases for advanced dataset filters Explore some illustrative examples and common use cases of how filters can be used in real projects. ### 1. Find unique data with custom filters You can easily find unique data by combining different filters. Let's look at an example using the [Pascal VOC dataset](https://datasetninja.com/pascal-voc-2012). Suppose we need to find images from the "train" dataset that contain both *buses* and *cars*. The number of *buses* in the image should not exceed 5, and the number of *cars* should be 1 or more. Here are the steps you can follow: 1. Apply a simple filter to subsample images from the "train" dataset. 2. Add a filter to find images with *buses* and set the maximum number of *buses* in the image to 5. 3. Add another filter to find images with *cars* and set the minimum number of *cars* in the image to 1. 4. Press the `Apply` button to retrieve the data that meets all of the specified conditions and to view the filter results.

Combining filters to search for object classes "bus" and "car" in specific quantities

### 2. Manage huge datasets at any scale with ease Conditional dataset filters make managing large datasets simple. These filters help you to explore and identify images that can be merged into a new labeling job, moved or submitted to additional review and verification. Instead of using API and writing custom Python scripts, data annotation managers can quickly configure custom conditions and integrate them into their labeling pipelines in a few clicks. Creating a new labeling job from unannotated images ### 3. Use filters to explore model predictions Data scientists can use custom filters to evaluate predictions from custom Neural Networks. For example, Supervisely users can easily apply custom object detection model, save model predictions and further use them for analysis or as an initial data prelabeling. In that case, every object (bounding box) will have the tag `Confidence` with some value from 0 to 1. Thus you can create custom filter to find all images with the most or less confident predictions (e.g. "Confidence < 0.5"). Analyzing bounding boxes with low confidence levels can help to gain insights and better understand model performance and uncover the ways to improve it. Or developers can leverage the Supervisely platform for exploratory data analysis, finding data outliers or possible errors in training data. It can be used to improve adaptive learning strategies by dynamically filtering images based on model feedback, performance metrics, or user-defined criteria to iteratively sample, annotate, and improve model performance. Also filters can be utilized to make smaller training datasets from the large ones for model evaluation also known K-fold cross-validation training technique. Searching for images in the training dataset with a low threshold and removing them ### 4. Labeling job tracking for team collaboration Efficient [team collaboration](https://supervisely.com/blog/labeling-queues/) requires streamlined processes for tracking and managing annotation tasks. Supervisely's advanced dataset filtering capabilities optimize the job tracking process. By filtering datasets based on labeling jobs and their status and combining with other conditional filters, you can quickly find relevant images, explore unusual patters in your annotations, find hidden mistakes and perform simple yet effective quality assurance operations. This also improves visibility of job progress and ensures quick access to job activity and statistics. [Labeling job](https://supervisely.com/blog/labeling-jobs/) statuses indicates the progress of the annotation or the quality of the data annotation: for example, images with `rejected` status may require review or correction. Sorting by these statuses helps to quickly identify and respond to potential data quality issues. Optimized job tracking through dataset filtering enables your team to be more productive, resulting in faster annotation project completion. # Pipelines Easily combine data management, augmentation, filtering, and neural network operations with drag-and-drop pipelines, which include a set of over 150 nodes for different modalities. Supervisely introduces a robust Computer Vision Pipelines system designed to simplify MLOps (Machine Learning Operations) and DataOps (Data Operations) with a node-based architecture. This innovative system allows users to manage data, perform augmentations, apply filters and run neural network operations seamlessly using an intuitive drag-and-drop interface. The system includes over 150 nodes to cater to various data processing needs.
## What are Pipelines? Pipelines in Supervisely are a modular approach to data processing and workflow management. They enable users to create complex workflows by connecting different nodes, each representing a specific operation, such as data transformation, neural network application, collaboration or data enhancement. This node-based system, inspired by similar approaches in video editing, 3D graphics, and game development, streamlines the process of building and managing sophisticated data workflows.
* **Transform Data:** Apply a wide variety of data transformation operations to images within a project. These transformations include rotation, cropping, blurring, resizing, and many more. * **Use Neural Networks:** Apply deployed models on your data to perform object detection, instance segmentation, and other tasks. You can use any of the neural network models available in the Supervisely Ecosystem, or train your custom models. * **Enhance Data:** Improve the quality and usability of your image data by adjusting contrast, brightness, and noise levels. * **Object-Level Manipulation:** Perform operations on individual objects or instances within images, such as cropping, duplicating, or changing their color classes. * **Customize Workflows:** Create complex data transformation workflows by combining multiple transformation nodes to meet your specific requirements. * **Node Documentation:** Detailed documentation is available for each transformation node, explaining how to use it effectively. These guides provide step-by-step instructions and examples for each node, making it easy for users to understand and leverage the full power of the application. * **Save & Load Presets:** Save your customized transformation workflows as presets for future use. This feature allows you to store and reuse your preferred configurations quickly. * **Output Flexibility:** Choose from multiple export options to save your transformed data in a format that best suits your needs. * **MLOps:** Manage ML workflows from data annotation to model deployment, incorporating CI/CD (Continuous Integration/Continuous Deployment) and continuous training principles. * **DataOps:** Efficiently process and manage data throughout its lifecycle with an emphasis on collaboration, quality assurance, and automation. ## How to use Pipelines? Using Computer Vision Pipelines in Supervisely is easy and powerful. With an intuitive, easy-to-understand interface, users can create advanced workflows without extensive programming knowledge. Drag-and-drop functionality and a wide range of customization options allow pipelines to be tailored to specific needs. From managing datasets - copying, moving, filtering, merging and splitting - to performing complex transformations and augmentations, Supervisely makes it straightforward. Users can easily crop and resize images, convert shapes (such as polygons to bounding boxes for object detection tasks), and apply a variety of operations to enrich and modify data. ### Step 1. Launch Pipelines You have several options to run a pipeline, offering flexibility based on your workflow and preferences. #### **Running Pipelines from the Project Interface** You can run the `Pipelines` from the project's interface, allowing for project-specific workflows, or start the pipeline from dataset, enabling dataset-specific processing and transformations.
#### **Other Shortcuts** There are several other convenient ways to start a pipeline. You can launch the desired application directly from the Supervisely Ecosystem, access and run the app from the project's context menu, or run the app directly from the dataset's context menu to streamline the process.
In addition, you can apply filters to your data before running pipelines to ensure precise and targeted transformations. ### Step 2. Drag-and-Drop and Connect Nodes Creating a pipeline in Supervisely involves adding and connecting nodes to define your data processing workflow: * **Add Nodes:** Select the necessary nodes from the library or use the context menu to quickly add nodes. Nodes represent different operations such as data transformation, Neural Network application, and data enhancement.
* **Configure Nodes:** For each node, set up its parameters according to your requirements. This could include specifying transformation types, Neural Network models, or enhancement settings. * **Connect Nodes:** Use the **drag-and-drop** interface to connect nodes in the desired sequence. The connections represent the data flow from one operation to the next, creating a streamlined and logical processing path. * **Customize Workflow:** Adjust and rearrange nodes as needed to tailor the workflow to your specific needs. You can combine multiple nodes to handle complex data processing tasks. ### Step 3. Run Pipelines Once your pipeline is set up, you can easily manage its execution and reuse: * **Run Pipeline:** Start the pipeline by clicking the run button. Supervisely will process the data according to the defined workflow, applying each node's operations in sequence. * **Monitor Progress:** As the pipeline runs, monitor its progress through the interface. You can view real-time updates and ensure each step is completed successfully. * **Modify and Re-run:** If needed, modify the loaded pipeline by adding, removing, or reconfiguring nodes. Once adjusted, re-run the pipeline to apply the updated workflow to your data. ### Step 4. Save Pipelines * **Custom Templates:** Create custom templates for your pipelines and save your configured pipelines as a preset for future use. Custom templates allow you to standardize and share specific workflows across your team or organization. By using templates, you can ensure that everyone member follows best practices for data processing tasks and quickly replicates the same workflow without having to set it up again. * **Load Pipelines:** Load a previously saved pipeline preset to reuse your customized workflows. This feature enhances efficiency by allowing you to apply consistent processing steps across different projects and datasets.
### Dataset Management with Pipelines Even basic dataset operations such as copying, moving, filtering, merging, and splitting datasets and projects are now made incredibly easy thanks to pipelines. Imagine you have a large dataset of images that needs to be prepared for a machine learning project. Here's how you can leverage Supervisely Pipelines to streamline this process: **Copying and Moving Data** **Node 1. Copy Dataset:** Use a node to duplicate your original dataset. This ensures that your original data remains intact while you work on the copy. **Node 2. Move Data:** Add a node to move specific subsets of the copied data to different directories based on your organizational needs.
#### **Filtering and Splitting Data** **Node 3. Filter Data:** Apply a filtering node to select images based on specific criteria, such as resolution or file type. This helps in focusing on the most relevant data. **Node 4. Split Dataset:** Use a split node to divide the filtered dataset into training, validation, and test sets, ensuring a balanced distribution for your machine learning tasks. #### **Merging Datasets** **Node 5. Merge Datasets:** If you have multiple datasets that need to be combined, add a merge node. This node will unify different datasets into a single, cohesive set, simplifying further processing and analysis.
#### **Performing Complex Transformations** **Node 6. Data Augmentation:** Add nodes for various data augmentation techniques such as rotation, flipping, and color adjustments to enrich the dataset. **Node 7. Convert Shapes:** Use a node to convert shapes, such as changing polygons to bounding boxes, which is crucial for object detection tasks. By connecting these nodes in a pipeline, you create an automated, repeatable workflow that handles every step of dataset management. ### Transformations & Augmentations The system supports a wide range of data transformations and augmentations. You can easily convert videos to images or resize images. Additionally, users can transform polygons into bounding boxes for object detection tasks. Furthermore, features like rotation, cropping, flipping, and adjusting brightness and contrast are available, along with many other capabilities. Data augmentation allows for the creation of numerous variations from a single image, helping models train under diverse conditions and enhancing their generalization ability.
### Neural Networks Inference & Model Ensembles Our Nodes facilitate the creation and deployment of Neural Networks and their ensembles for inference tasks. You can create a Node to deploy a model and then apply this model to your data using another Node. Moreover, you are not limited to a single model; you can use multiple models in conjunction. For example, you can first apply a model for object detection and then use another model to segment each detected object. This allows you to build model ensembles, significantly improving performance and providing more comprehensive results. ### Labeling Tasks In Supervisely, you can create labeling tasks based on your input project. For instance, you can perform advanced filtering to isolate images with specific tags or those without annotations, and then create a labeling task based on the results of this filtering.
In another scenario, if you have unlabeled images, you can first apply a Neural Network to these images for automatic annotation, and then create a labeling task based on the generated annotations.
### Creating Complex Pipelines With Supervisely Pipelines, you can create complex combinations of your nodes and large and complex pipelines, that include both data processing and Neural Network operations. **For example:** 1. You can take a large image and split it into smaller blocks using a `sliding window` split. 2. Then apply a neural network operation to this image to perform detection and segmentation. 3. Apply a filter to remove all small rectangles with a low-confidence level less than 0.5. 4. Finally, you can create a labeling task based on filtered results. Supervisely Pipelines support multiple modalities, including images, video, and much more. Our comprehensive set of operations, which already includes over 150 nodes, is constantly being expanded. If you need specific functionality or have any questions, please don't hesitate to contact our support team. We're always here to help! # Augmentations Data augmentation is a crucial step in preparing data for machine learning. In this section, you'll find tools for augmenting your data, allowing you to increase its diversity and enhance machine learning model performance. A specialized data augmentation application streamlines this process and broadens your capabilities. ## Example apps: * [ImgAug Studio.](https://app.supervisely.com/ecosystem/apps/imgaug-studio?id=70) ImgAug Studio is a wrapper around great ImgAug Library. Interactive UI helps to understand how image transformations work and illustrates how to use this library with Supervisely Format. Once augmentations are combined into pipeline, they can be exported to both python file (for developers) and safer serialization format json. This json config can be used for real-time augmentations during training for some Neural Networks from Supervisely Ecosystem. Only labels of types Polygon, Rectangle and Bitmap in supervisely format can be converted automatically to imgaug format (and vice versa). * [Create Trainset for SmartTool.](https://app.supervisely.com/ecosystem/apps/create-trainset-for-smarttool?id=19) This app created training dataset for SmartTool from labeled project. All classes in the input project have to be Bitmaps. Please, use app Rasterize objects on images to raster all objects and prepare correct object masks. It is crucial for this app. All classes will be converted to a single class, then instances crop will be performed and then positive/negative points will be randomly generated. # Splitting data Splitting data into training, validation, and testing sets is a common practice in machine learning projects. It helps to evaluate the performance of the model on unseen data and prevent overfitting. In this guide, we'll explore different methods to split data using the Supervisely Ecosystem Apps and the Supervisely Python SDK. ## Splitting Data Using Supervisely Ecosystem Apps Splitting data into training and testing sets is a crucial step in machine learning projects. Here are some apps from the Supervisely Ecosystem that can help you with this task: * [Assign train/val tags to images](https://ecosystem.supervisely.com/apps/tag-train-val-test). This app allows you to assign tags to images in a dataset to split them into training, validation, and testing sets. You can specify the percentage of images for each set and assign tags accordingly. The resulting project can be used in training apps to create sets using tags. * [Split datasets](https://ecosystem.supervisely.com/apps/split-dataset). This app allows you to split selected datasets into parts according to the specified percentage/number of images/number of parts. You can choose to split the dataset randomly or by the order of images. The resulting datasets can be created in the same project or in a new one. ## Splitting Data Using Supervisely Python SDK Here is an example of how you can split a project into training and testing sets using the Supervisely Python SDK: ```python import supervisely as sly # Read the project project_fs = sly.Project("./sly_project", sly.OpenMode.READ) ``` * Splitting by percentage: ```python train_n = int(project_fs.total_items * 0.8) val_n = project_fs.total_items - train_n train_set, val_set = project_fs.get_train_val_splits_by_count("./sly_project", train_n, val_n) ``` * Splitting by dataset names: ```python train_set, val_set = project_fs.get_train_val_splits_by_dataset("./sly_project", ["ds1", "ds2"], ["ds3"]) ``` * Splitting by tags: ```python train_set, val_set = project_fs.get_train_val_splits_by_tag("./sly_project", ["tag1", "tag2"], ["tag3"]) ``` All the above methods will return two lists of `ItemInfo` objects that represent the training and validation sets items. ```python class ItemInfo(NamedTuple): dataset_name: str # Item's dataset name name: str # Item's name img_path: str # Full image file path of item ann_path: str # Full annotation file path of item ``` You can use these items to get the corresponding image name and path, annotation path, and dataset name. ```python for item in train_set: print(f"{item.name=}, {item.img_path=}, {item.ann_path=}") ``` # Converting data Converting data into different formats is essential operation in data preparation. We have specific applications designed for data conversion, simplifying these processes. Additionally, you can use the Supervisely Python SDK to convert data to other formats. Discover the tools for efficiently converting data to different formats in the following sections:
Convert to COCOConvert data to COCO detection, keypoints, and captions formats./pages/pdMaTyZRIYTMzjvPAXBU
Convert to YOLOConverting data to YOLO format for detection, segmentation, and pose estimation tasks./pages/gAAoVBzfqds5WyHKllSc
Convert to Pascal VOCConvert data to Pascal VOC format./pages/PN9NDoGeDuKw3Yfy8iA7
*** # Convert to COCO The [COCO](https://cocodataset.org/#home) format is widely used in the computer vision community and is supported by many popular frameworks and libraries. COCO format is a complex format that can contain multiple types of annotations. Supervisely supports conversions for instances, keypoints, and captions. For more information on how to import COCO format data into Supervisely, see the [Import from COCO](/import-and-export/import/supported-annotation-formats/images/coco) guide. ### Converting data using Supervisely Ecosystem Apps * [Export to COCO](https://ecosystem.supervisely.com/apps/export-to-coco). Convert project to COCO format. It is a simple and efficient way to export your project to COCO format. * [Export COCO Keypoints](https://ecosystem.supervisely.com/apps/export-coco-keypoints). App converts Supervisely format project to COCO Keypoints format as a downloadable `.tar` archive. ### Converting data using Supervisely Python SDK Our Python SDK provides a simple way to convert your data to COCO format, allowing you to convert a Project or Dataset to COCO format. Easily convert your data in one line of code using the Supervisely Python SDK. {% hint style="success" %} `sly.convert.to_coco()` function automatically detects the input data type and converts it to Pascal VOC format. For example, you can pass a path to a project, sly.Project object or sly.Dataset object. To convert a Dataset, you need to provide the project meta information as shown in the example below. ```python # Project path sly.convert.to_coco("./sly_project", dest_dir="./result_coco") # Project object sly.convert.to_coco(project, dest_dir="./result_coco") # or Dataset object sly.convert.to_coco(dataset, dest_dir="./result_coco", meta=project.meta) ``` {% endhint %} Each dataset in the project will be converted to a separate COCO dataset. {% hint style="info" %} It supports the following geometry types: `sly.Rectangle`, `sly.Bitmap`, `sly.Polygon`, `sly.GraphNodes`. Enabling the `with_captions` parameter will include captions in the COCO annotations (if present in the project). {% endhint %} * Convert a project to COCO format: ```python # One line of code sly.convert.to_coco("./sly_project", dest_dir="./result_coco") # Or using the sly.Project object project_fs = sly.Project("./sly_project", sly.OpenMode.READ) project_fs.to_coco("./result_coco") ``` * Convert a specific dataset to COCO format: ```python ds = project_fs.datasets.get("dataset_name") sly.convert.to_coco(ds, dest_dir="./result_coco", meta=project_fs.meta) # Or using the sly.Dataset object ds.to_coco(project_fs.meta, dest_dir="./result_coco") ``` # Convert to YOLO YOLO format is a popular, text-based format for different computer vision tasks, such as object detection, segmentation, and pose estimation. For more information on how to import YOLO format data into Supervisely, see the [Import from YOLO](/import-and-export/import/supported-annotation-formats/images/yolo) guide. ### Converting data using Supervisely Ecosystem Apps * [Convert Supervisely to YOLO v5 format](https://ecosystem.supervisely.com/apps/convert-supervisely-to-yolov5-format). Transform images project in Supervisely (link to format) to YOLO v5 format and prepares downloadable `.tar` archive. * [Export to YOLOv8 format](https://ecosystem.supervisely.com/apps/export-to-yolov8). Transform datasets from the Supervisely format to the YOLOv8 segmentation format or pose estimation format. ### Converting data using Supervisely Python SDK Easily convert your data in one line of code using the Supervisely Python SDK. {% hint style="success" %} `sly.convert.to_yolo()` function automatically detects the input data type and converts it to Pascal VOC format. For example, you can pass a path to a project, sly.Project object or sly.Dataset object. To convert a Dataset, you need to provide the project meta information as shown in the example below. ```python # Project path sly.convert.to_yolo("./sly_project", dest_dir="./result_yolo") # Project object sly.convert.to_yolo(project, dest_dir="./result_yolo") # or Dataset object sly.convert.to_yolo(dataset, dest_dir="./result_yolo", meta=project.meta) ``` {% endhint %} This converter allows you to convert a project or dataset to YOLO format for **detection**, **segmentation**, and **pose estimation** tasks. Project and dataset conversion works similarly and will convert all data in the same structure to YOLO format. {% hint style="info" %} It supports the following geometry types: * **detection**: `sly.Rectangle`, `sly.Bitmap`, `sly.Polygon`, `sly.GraphNodes`, `sly.Polyline`, `sly.AlphaMask` * **segmentation**: `sly.Polygon`, `sly.Bitmap`, `sly.AlphaMask` * **pose estimation**: `sly.GraphNodes` {% endhint %} * Convert a project to YOLO format: ```python # One line of code sly.convert.to_yolo("./sly_project", dest_dir="./result_yolo") # Or using the sly.Project object project_fs = sly.Project("./sly_project", sly.OpenMode.READ) project_fs.to_yolo("./result_yolo", task_type="segmentation") ``` * Convert a specific dataset to YOLO format: ```python ds = project_fs.datasets.get("dataset_name") sly.convert.to_yolo(ds, dest_dir="./result_yolo", meta=project_fs.meta) # Or using the sly.Dataset object ds.to_yolo(project_fs.meta, dest_dir="./result_yolo") ``` # Convert to Pascal VOC The Pascal VOC (Visual Object Classes) format stands as one of the benchmarks established relatively early for object classification, segmentation and detection. It furnishes a standardized dataset for identifying object classes, utilizing an XML-based export format that enjoys widespread adoption in computer vision tasks. For more information on how to import Pascal VOC format data into Supervisely, see the [Import from PascalVOC](/import-and-export/import/supported-annotation-formats/images/pascal) guide. ### Converting data using Supervisely Ecosystem Apps * [Export to Pascal VOC](https://ecosystem.supervisely.com/apps/export-to-pascal-voc). Converts Supervisely format project to Pascal VOC and prepares downloadable `.tar` archive. ### Converting data using Supervisely Python SDK Easily convert your data in one line of code using the Supervisely Python SDK. {% hint style="info" %} `sly.convert.to_pascal_voc()` function automatically detects the input data type and converts it to Pascal VOC format. For example, you can pass a path to a project, sly.Project object or sly.Dataset object. To convert a Dataset, you need to provide the project meta information as shown in the example below. ```python # Project path sly.convert.to_pascal_voc("./sly_project", dest_dir="./pascal_voc") # Project object sly.convert.to_pascal_voc(project, dest_dir="./pascal_voc") # or Dataset object sly.convert.to_pascal_voc(dataset, dest_dir="./pascal_voc", meta=project.meta) ``` {% endhint %} This converter allows you to convert a Project or Dataset. Each dataset in the project will be converted to a separate Pascal VOC dataset. * Convert a project to Pascal VOC format: ```python # One line of code sly.convert.to_pascal_voc("./sly_project", dest_dir="./result_pascal") # Or using the sly.Project object project_fs = sly.Project("./sly_project", sly.OpenMode.READ) project_fs.to_pascal_voc("./result_pascal") ``` * Convert a specific dataset to Pascal VOC format: ```python ds = project_fs.datasets.get("dataset_name") sly.convert.to_pascal_voc(ds, dest_dir="./result_pascal", meta=project_fs.meta) # Or using the sly.Dataset object ds.to_pascal_voc(project_fs.meta, dest_dir="./result_pascal") ``` # Data Commander Data Commander is a tool that combines the classic approach of file management with the modern needs of handling data for training Computer Vision models. The primary goal of this tool is to simplify and speed up data manipulation processes by providing an interface similar to traditional file managers like Double Commander or Total Commander, but tailored to the specific requirements of working with annotations and other training data components. Data Commander is a dual pane data manager (just like Double Commander), but not files - it's for training data. ![](/files/-M54fCcmwL6tvKhsPyEn) ## What you can do with Data Commander: * Explore your teams, workspaces, projects and datasets * Get insights and statistics about your data With over 30 columns of metadata (such as the number of labels of different shapes, the source of the image, etc.), Data Commander helps users better understand the structure and quality of their data, which is crucial for preparing the most accurate and reliable neural network models. * Move, copy, rename and remove your stuff * Create new items, like teams * Do batch operations, like tagging Easily move and copy data between projects and workspaces, supporting standard functions such as renaming, deleting, and creating new items. It's particularly useful for handling large amounts of data, as batch operations can be performed using keyboard shortcuts. * Quickly preview images Data Commander allows you to quickly preview images without having to open the full annotation interface. This helps to quickly analyze data, identify problems, and make decisions about further processing. Wanna learn more? Check out our [blog post](https://medium.com/deep-systems/dual-pane-file-manager-for-training-data-old-school-meets-ai-in-supervisely-d0aa64def296)!
Clone Project MetaIf you need to copy project meta (classes & tags) to other project this section will be useful for you./pages/nfzuXMKbD3q6TVtGfXKd
# Clone Project Meta If you need to copy project meta (classes & tags) to other project this section will be useful for you. ## Open projects panels Go to Data Commander and open projects in both panels ![](/files/-M4ZM-fPDOqolGAWlW0q) ## Select projects Select one or more destination projects ![](/files/-M4ZM-fRiCmRYphxC5wQ) Then switch to second panel and focus source project ![](/files/-M4ZM-fTm1ORkHb1YuNs) ## Copy meta Click "Copy meta" button or press F9 ![](/files/-M4ZM-fVY1SHKtQZXvLY) If destination project already have classes or tags it will be skipped ![](/files/-M4ZM-fXfj6JWDDg3hVb) # Labeling Toolboxes With more than 5 years of constant improvement, proved by hundreds of businesses, Supervisely provides a complete set of labeling toolboxes for various modalities and tasks, starting from images, videos, and including even solutions for 3D point clouds and volumetric data. ## Pre-Requirements Follow those recommendations for the best results: {% hint style="info" %} Though we support all common web browsers, we strongly recommend using **Google Chrome** or **Mozilla Firefox**, because we use latest technologies to render annotations. We also advise you to use the latest version of web browser. {% endhint %} {% hint style="success" %} To work with large images and lots of annotations we recommend to use computer with hardware acceleration available. Check if your browser uses hardware acceleration [here](chrome://gpu). {% endhint %} ## Getting Started First, [import](/import-and-export/import) the dataset you would like to annotate. You can upload images, videos, and many other types of data from your computer or import one of our [sample projects](https://ecosystem.supervisely.com/import+images+project) from the Ecosystem. To open the labeling toolbox, go to the [Projects](broken://pages/-M54fC5kfcVDMQPT05GQ) page, select one of the projects and click on a dataset. Depending on the type of your project, you will see a popup where you can select the right toolbox or, if there is one, the labeling toolbox will open automatically. ![](/files/ngmCsbnaSLtRCjsr7HQP) {% hint style="info" %} When opening a labeling toolbox, you can only annotate a single dataset at a time. {% endhint %} {% hint style="success" %} You can also open the labeling toolbox from a [Labeling Job](/labeling/jobs) or the [Ecosystem](https://ecosystem.supervisely.com/annotation_tools/image-labeling-tool-v2) page. {% endhint %}
Images labeling toolboxThe image labeling toolbox allows you to annotate one image at a time, such as .jpg, .png, .tiff, and many more formats you can import to Supervisely./pages/Q4pJ0vCu09FFWuFL7CNE
Multiview imagesCreate image groups inside your dataset by assigning a grouping tag./pages/PLBjdtjVA4EPcMU1F9bm
OverlayInspect a base image together with linked overlay layers and adjust opacity for direct visual comparison./pages/jHUMAvHf4uaRmHwndVsL
Videos labeling toolboxLabel hours-long videos without cutting them into images. In your browser, with multi-track timeline, built-in object tracking and segments tagging tools./pages/aEsmAIHBkgLxZPsLUMk3
Video trackingThe most simple and straightforward method of importing is uploading your data using one of our Supervisely Apps./pages/zG0ZqIUYYLYyQDW29E2j
3D Point CloudsLabel comprehensive 3D scenes from LiDAR or RADAR sensors with additional photo and video context, AI object tracking and point cloud segmentation./pages/iSy3IlvEWGHtUm6jb5Eg
3D Point Clouds EpisodesOur toolbox for 3D Point Cloud labeling is a great solution for annotation a single point cloud at a time./pages/5Bh49HZksdQXKSYjJjCC
Sensor-fusionAdditionally to a single point cloud and episodes point clouds toolboxes, Supervisely allows you to provide additional photo and video context for accurate labeling./pages/iSP4Qw1tKB1zQwdtwVru
DICOMThe most simple and straightforward method of importing is uploading your data using one of our Supervisely Apps./pages/28mXNC4Xnd8tM0Nbfxkp
# Images The image labeling toolbox allows you to annotate one image at a time, such as .jpg, .png, .tiff, and many more formats you can import to Supervisely. The Supervisely Image Annotation Tool is a web-based image annotation toolbox specifically developed to address the needs of computer vision projects. It is fully web-based, allowing users to access and use the tool from any browser without the need for local installations. The platform provides powerful tools for annotating different types of images, supports different formats, and integrates neural networks to increase the speed and accuracy of annotation tasks. 1. Web-based interfaces - you just need a browser. 2. Fully customizable interface with easy-to-tune visualization settings: light and dark theme, flexible layout, multiple image view modes, multi-spectral view and grid. 3. Supports complex image formats: high-resolution images, high-color depth images with 16-bit per pixel or more, customizable image visualization settings, filter images with conditions, additional image metadata, restore mode and undo/redo functionality. 4. Advanced labeling capabilities: multiple annotation tools -[ Bounding Box](https://supervisely.com/blog/bounding-box-annotation-for-object-detection/),[ Polygon Tool](https://supervisely.com/blog/how-to-use-polygon-anotation-tool-for-image-segmentation/),[ Brush and Eraser Tool](https://supervisely.com/blog/brush/),[ Mask Pen Tool](https://supervisely.com/blog/mask-pen-tool/),[ Smart Tool](https://supervisely.com/blog/smarttool-annotation/),[ Graph (Keypoins) Tool](https://supervisely.com/blog/animal-pose-estimation/), effectively supports 1000+ objects per image, image and object tags and attributes, customizable hotkeys. 5. Collaboration and workflow management features for large annotation teams. 6. Integration with various Neural Networks and AI-assisted annotation tools. 7. Effortless data import and export for seamless sharing across platforms. 8. Compatible with medical, NRRD, NiFTI data. ## Overview
1. **Home button** - returns user to the main menu (`Projects` page) 2. [**Basic interface elements**](#explore-basic-interface-elements) - basic settings, such as history of operations, theme, a hotkeys map and more useful features. 3. [**Main scene & labeling scene settings**](#main-scene-and-labeling-scene-settings)- annotation area for current image and its labels. 4. [**Definitions panel**](#definitions-panel) - make it easy to create and manage classes and tags. 5. [**Instruments panel**](#instruments-panel) - annotation tools used to create annotations. 6. [**Objects panel**](#objects-panel) - list of figures on the current image with additional information like classes and tags. 7. [**Images/Apps/Settings panel**](#images-panel) - list of images in your dataset, list of additional apps you can embed into the labeling toolbox, visualization and other settings. *** ## Basic interface elements The top toolbar contains options for personalizing the interface and managing data and its annotations.
**Image navigation arrows (next, previous):** Allow users to move between images in the dataset. **Undo and redo buttons:** Undo or redo the most recent annotation action. **Select theme (dark or light):** Ability to switch between light and dark interface themes, especially useful for those who prefer to work at night. **Hotkeys:** A list of hotkeys for quick access to tools. **More options:** * **Enter fullscreen** - this option allows the user to switch the interface to fullscreen mode, maximizing the workspace area. It hides browser toolbars and other elements * **Screenshot** - the screenshot function enables users to take a snapshot of the current workspace, including the image and any annotations displayed. This can be useful for documentation, sharing progress, or reviewing annotations with team members. * **Enter restore mode** - enter restore mode provides tools to recover lost or corrupted annotations. When enabled, it offers options to revert changes to a previous state or repair specific parts of the annotation dataset. * **Restore default layout** - this function resets the interface layout to its default configuration. It is useful when the layout has been modified (e.g., panels moved or resized) and the user wants to return to the original setup. * **Multiple image views mode** - multiple image views mode allows users to view and annotate multiple images simultaneously. This feature is particularly useful for comparing images side-by-side, annotating similar objects across different images, or analyzing changes in a sequence of images. Users can customize the arrangement and number of views according to their needs. *** ## **Main scene & display toolbar** settings This is the central area. It displays the image to be annotated, with various display controls that the user can hide or show in the panel as needed. It contains: **Current image view:** shows the image currently being worked on. Users can interact directly with this area using the annotation tools from the sidebar. **Mini-map (top-right corner):** Displays a smaller version of the entire image to help navigate quickly, especially when zoomed in on specific areas.
### Annotation display settings **Opacity:** To modify the transparency of objects, hold and drag the cursor left or right. This allows for a more nuanced view of the objects' layers. Additionally, you can hold the `SHIFT` key and scroll the mouse wheel to adjust the opacity conveniently from anywhere on the screen. **Border:** Enhance the visibility of object boundaries by holding and dragging the cursor left or right. This action changes the width of the objects' borders, allowing for clearer demarcation. Useful when working with huge resolutions or with large number of small objects **Point:** Adjust the radius of object points by holding and dragging the cursor left or right. **Default color:** Paint objects with their original colors as defined in class settings. This is the default setting and helps maintain consistency and recognition. So the objects of different classes are visually distinguishable. **Randomize color:** Randomize object colors to distinguish between objects of the same class. A simple click, followed by `SHIFT+H`, randomizes the object's colors. Can be used in Instance Segmentation Computer Vision task to highlight visual distinction of all objects of the same class on an image.
### Attribute display settings for clearer context **ID:** Toggle the visibility of object IDs near the objects on the scene. This is essential for identifying and referring to specific objects. **Bindings:** Show or hide bindings near objects to understand how various elements are connected to each other. [Objects can be combined into groups](https://developer.supervisely.com/advanced-user-guide/objects-binding). **Tags:** Display tags near objects to provide additional context or categorization. **Classes:** Enable visibility of the classes assigned to each object, helping in the classification and organization of scene elements. **Author:** Display the creator's name near the objects to acknowledge object authorship. **Change Visibility Mode:** This option allows users to switch between different visibility modes, optimizing the scene display as per the user's preference. You can choose how to show the attributes: * **Always** | Users can select full tag display, which means that the information will be visible directly on Objects or Images in the project. * **Show on hover** | Tags are only displayed when the cursor is hovered over the annotated object. * **Show when selected** | The ability to hide Tags until the Object is selected provides a cleaner look and feel to the interface and prevents information overload when working with a project.
### Advanced interaction with scene objects **Auto-select:** Automatically select objects of the current shape when hovering the cursor over them.
### Customizing image display settings **Scene display settings:** Adjust scene display settings like brightness or contrast to suit different viewing conditions or preferences. For example, you can use them while annotation dark or low-contrast images. **Grid:** A grid helps organize the navigation on the images with high resolutions and large number of small objects.
### Visibility and image sizing The ability to hide annotation settings declutters the workspace, focusing attention on the task at hand. Real-time image resizing adapts to various project needs, ensuring optimal viewing and editing conditions. Just zoom-in or out on the images to see object details and perform precise labeling of object boundaries.
*** ## Definitions panel The Definitions panel provides a simple interface for creating and managing classes and tags in a project. It helps users organize and control annotations. Instead of choosing a tool first, you can now click on the desired class from the Definitions Panel. The associated tool will be automatically selected, allowing you to start labeling immediately. To start a new label, simply click on any class (including the currently selected one) in the Definitions Panel. If you want to change the class of the selected object, you can click the small icon on the right, which appears only if the new class shape matches the currently selected object. Tags are also present on the same panel. If no object is selected, image tags are shown. You can check the desired tag or hover the cursor and start typing a tag value or select it from a dropdown, which will automatically assign it. **Improved Search:** To find classes or tags more easily in a long list, click the magnifying glass icon in the top right corner of the panel. Type your query and select the desired class or tag to continue your workflow
### Classes Class may be assigned only to an object and represent a clear category to which the object in the image belongs. For example, a classification of vehicles in an image might include the classes "car", "truck", and "bus". Definitions panel displays the list of annotation classes (e.g., "person", "road sign", "vehicle"). Each class has a unique name, color and shape to identify between different types of objects. * Add new class definitions using the **add new class definition** option. * **Organize classes** in the definitions list to improve navigation. ### Tags Tags are used to add additional information to images, objects, or other data. Tags can describe context, object properties, or other parameters that can be useful for data analysis and model training.
Definitions panel shows tags associated with the current image. Tags are metadata that help to categorize or add additional information to images (e.g., weather conditions, time of day). * Use the **add project tags definitions** feature to create and manage tags at the project level. * **Tag removal:** You can configure the system to ask for confirmation when tags are removed. * **Attaching a single tag multiple times:** You can enable or disable the ability to attach a single tag multiple times to an object. This setting can be adjusted by editing `Project → Settings → Tags → Multiple Tags Mode` in the Dashboard. * **Removing tags with hotkeys:** You can enable or disable the feature that allows a tag to be removed when pressing the corresponding hotkey again. For the classic version of the toolbox (number `1` on the screenshot), these settings are configured through the `Settings` panel - `Tags` tab at the bottom `Toggle tags` section. For the advanced version of the toolbox (number `2` on the screenshot), you need to open the context menu (the three dots ...) in `Definitions` and activate the Toggle Tag on Hotkey option.
Toggle tags
Learn more about the definitions panel from our [blog post](https://supervisely.com/blog/definitions-panel/): {% embed url="" %} *** ## **Instruments panel** **Pan & Move Scene Tool:** Quickly navigate around the image without modifying annotations. **Select Figure**: Select and modify existing annotations; essential for refining objects. **Drag Figure:** Reposition annotations without altering their size or shape. **Issues:** Manage and report issues related to annotations; improves collaboration. **Point Tool:** Label specific points or small objects precisely. [**Bounding Box:**](/labeling/labeling-tools/bounding-box-rectangle-tool) Best for object detection tasks. **Polyline Tool:** Annotate linear objects or edges with multiple connected line segments. [**Polygon Tool:**](/labeling/labeling-tools/polygon-tool) Ideal for irregular and complex shapes. [**Brush and Eraser Tool:**](/labeling/labeling-tools/brush-tool) Flexible for both polygonal and free-form masks. [**Mask Pen Tool:**](/labeling/labeling-tools/mask-pen-tool) Great for segmenting diverse objects with varying shapes. [**Smart Tool:**](/labeling/labeling-tools/smart-tool) Efficient for quick, AI-assisted segmentation. [**Graph (Keypoins) Tool:**](/labeling/labeling-tools/graph-keypoints-tool) For pose-estimation tasks.
*** ## Objects panel The Objects Panel is a dynamic space dedicated to showcasing and managing objects tags, attributes and metadata. Here are some of the features it offers: * **Clone Objects** - Easily replicate selected objects to the next image with a simple right arrow key press or bring objects from the previous image. * **Filter and Manage** - Quickly filter objects, remove all from the image, or toggle their visibility according to your needs. For example you can hide all objects except of the specific class. * **Advanced Interactions** - Select, delete, hide, merge objects, or adjust their layering. Additionally, you can modify metadata and assign or manage tags right from this window, enhancing the object's data with minimal effort.
*** ## Images panel The Images Panel provides a comprehensive view of all the images within a selected dataset. Key functionalities include: * **Tag Management** - Clone tags from the previous image, assign new ones, or modify existing tags to maintain consistency and organization. * **Filter and Manage** - View and edit metadata details or filter through images to find exactly what you need. * **Image Operations** - Delete images, download them individually, or download annotations for external use. *** ## Apps panel: expand your capabilities In an ever-evolving ML landscape, the Apps Panel serves as a portal to a wide range of applications from the [Computer Vision Ecosystem](https://ecosystem.supervisely.com/), enhancing the functionality of your workspace. This window allows you to run and open the public or private apps and extend the Labeling Toolbox with custom UI and functionality. *** ## Settings panel The Settings Panel is the control center for personalizing the interface. It houses various options allowing users to tweak the interface to match their workflow, preferences, and project requirements.\\
# Videos 2.0 Label hours-long videos without cutting them into images. In your browser, with multi-track timeline, built-in object tracking and segments tagging tools. **Advanced Object Tracking:** The video annotation toolbox supports both Single Object Tracking (SOT) and Multiple Object Tracking (MOT), enabling efficient tracking of objects across frames. Users can integrate their own tracking algorithms without needing specialized knowledge. **AI-Assisted Labeling:** The toolbox offers interactive semantic segmentation tools powered by class-agnostic neural networks, which can be trained on custom datasets. These neural networks are both trainable and customizable, allowing users to adapt them for various data types and use cases. **Video Tagging:** With segment tagging, users can annotate specific video sections to capture and categorize different events or activities. This simplifies the video annotation workflow and ensures that important moments are accurately labeled. **User-Friendly Interface:** The interface is highly customizable, allowing users to tailor the annotation environment to their needs. **No Conversion Required:** Users can start annotating videos immediately after uploading or by connecting their cloud storage. This eliminates the need for converting videos into image sequences, speeding up the workflow. **Automatic Management of Tracking IDs:** The toolbox automatically handles tracking IDs across frames, reducing manual intervention. This simplifies the process of tracking multiple objects and enhances workflow efficiency. **Consistent Frame Handling:** By maintaining consistency across all frames, the toolbox ensures that object tracking and tagging remain accurate over long video segments. This is critical for training neural networks, ensuring they learn from the same data that human annotators have reviewed. ## **Overview**
1. **Home button** - returns user to the main menu (Projects page) 2. [Basic interface elements](#basic-interface-elements) - basic settings, such as history of operations, theme, a hotkeys map and more useful features. 3. [Main scene & labeling scene settings & playback](#main-scene-and-labeling-scene-settings-and-playback) - annotation area for current video and its labels. 4. [Timeline and track controls](#timeline-and-track-controls) - video timeline and controls for managing tracks and frames. 5. [Instruments panel](#instruments-panel) - annotation tools used to create annotations. 6. [Objects panel](#objects-panel) - list of figures on the current video with additional information like classes and tags. 7. [Videos/Apps/Settings panel](#images-panel) - list of videos in your dataset, list of additional apps you can embed into the labeling toolbox, visualization and other settings. *** ## **Using tags in the Video Annotation Tool 2.0** ### **Global Tags** Global Tags apply to the entire video or to an object across its presence in the video. #### **Assign a global tag for an video** 1. Go to the **Videos** panel and open **Toggle tags panel**. 2. In the **Tags Available** section, select or create tags to describe the video property tag (for example, "Traffic Density" or "Weather Condition"). Choose whether the scope is global or global and frame-based when creating a tag. 3. Click the **Attach to video as a property tag** button to assign it as a global tag for the entire video.
#### **Assign a global tag for an object** 1. Click on the labeled object in the video or go to the **Objects** panel and select one (e.g., "car"). 2. In the **Tags Available** section, select or create tags to describe the object property tag throughout the video (for example, "Road Position" or "Direction"). Choose whether the scope is global or global and frame-based when creating a tag. 3. Click the **Attach to annotation object as a property tag** to link the tag to the object.
### **Frame-Based Tags** Frame Tag**s** allow you to tag specific moments or actions on individual frames. **Assign a frame tag for an object** 1. Click on the labeled object in the video or go to the **Objects** panel and select one (e.g., "bus"). 2. In the **Tags Available** section, choose a tag (e.g., "Lane Change") and click **Mark Frames** to tag the frames where the bus changes lanes. 3. Use the **Timeline** slider to navigate through the video and label specific frames with the selected tag. Once you've identified the relevant frames, the tag will be applied accordingly to the chosen frames for the object.
**Assign a frame tag for an video** 1. Go to the **Videos** panel and open **Toggle tags panel**. 2. In the **Tags Available** section, choose a tag (e.g., "Traffic Density") and click **Mark Frames** to select the frame range to which you want to apply the tag. 3. Use the **Timeline** slider to navigate through the video and label specific frames with the selected tag. Once you've identified the relevant frames, the tag will be applied accordingly to the chosen frames for the video.
## **Tracking** **Tracking** allows the system to automatically follow an object across multiple frames. 1. Select the object in the video. 2. Click **Track** (or use the **SHIFT + T** shortcut) to start tracking. The system will automatically follow the object's movement. 3. In the tracking settings, you can select the tracking method, direction, and number of frames to process. Once tracking is complete, the object will be tracked across all selected frames.
*** ## Basic interface elements The top toolbar contains options for personalizing the interface and managing data and its annotations.
**Video navigation arrows (next, previous):** Allow users to move between videos in the dataset. **Undo and redo buttons:** Undo or redo the most recent annotation action. **Select theme (dark or light):** Ability to switch between light and dark interface themes, especially useful for those who prefer to work at night. **Hotkeys:** A list of hotkeys for quick access to tools. **More options:** * **Enter fullscreen** - this option allows the user to switch the interface to fullscreen mode, maximizing the workspace area. It hides browser toolbars and other elements * **Screenshot** - the screenshot function enables users to take a snapshot of the current workspace, including the video and any annotations displayed. This can be useful for documentation, sharing progress, or reviewing annotations with team members. * **Enter restore mode** - enter restore mode provides tools to recover lost or corrupted annotations. When enabled, it offers options to revert changes to a previous state or repair specific parts of the annotation dataset. * **Restore default layout** - this function resets the interface layout to its default configuration. It is useful when the layout has been modified (e.g., panels moved or resized) and the user wants to return to the original setup. *** ## Main scene & labeling scene settings & playback This is the central area. It displays the video to be annotated, with various display controls that the user can hide or show in the panel as needed. Main scene shows the video currently being worked on. Users can interact directly with this area using the annotation tools from the sidebar.
### Annotation display settings **Opacity:** To modify the transparency of objects, hold and drag the cursor left or right. This allows for a more nuanced view of the objects' layers. Additionally, you can hold the `SHIFT` key and scroll the mouse wheel to adjust the opacity conveniently from anywhere on the screen. **Border:** Enhance the visibility of object boundaries by holding and dragging the cursor left or right. This action changes the width of the objects' borders, allowing for clearer demarcation. Useful when working with huge resolutions or with large number of small objects **Point:** Adjust the radius of object points by holding and dragging the cursor left or right. **Default color:** Paint objects with their original colors as defined in class settings. This is the default setting and helps maintain consistency and recognition. So the objects of different classes are visually distinguishable. **Randomize color:** Randomize object colors to distinguish between objects of the same class. A simple click, followed by `SHIFT+H`, randomizes the object's colors. Can be used in Instance Segmentation Computer Vision task to highlight visual distinction of all objects of the same class on an image.
### Attribute display settings for clearer context **ID:** Toggle the visibility of object IDs near the objects on the scene. This is essential for identifying and referring to specific objects. **Bindings:** Show or hide bindings near objects to understand how various elements are connected to each other. [Objects can be combined into groups](https://developer.supervisely.com/advanced-user-guide/objects-binding). **Tags:** Display tags near objects to provide additional context or categorization. **Classes:** Enable visibility of the classes assigned to each object, helping in the classification and organization of scene elements. **Author:** Display the creator's name near the objects to acknowledge object authorship. **Change Visibility Mode:** This option allows users to switch between different visibility modes, optimizing the scene display as per the user's preference. You can choose how to show the attributes: * **Always** | Users can select full tag display, which means that the information will be visible directly on Objects or Videos in the project. * **Show on hover** | Tags are only displayed when the cursor is hovered over the annotated object. * **Show when selected** | The ability to hide Tags until the Object is selected provides a cleaner look and feel to the interface and prevents information overload when working with a project.
### Advanced interaction with scene objects **Auto-select:** Automatically select objects of the current shape when hovering the cursor over them. **Show object trajectory:** Visualize the path an object has taken in the scene over time. You can customize the way the trajectory looks using the following settings: * **Width (px):** This setting controls the thickness of the trajectory line in pixels. A higher value makes the line thicker, making the trajectory more visible, while a lower value makes the line thinner. * **Draw backward frames:** This option controls how many frames in the past are drawn to represent the object's trajectory. Users can adjust the number of frames using a slider, allowing them to visualize the object's past movements over a longer or shorter period of time. * **Draw forward frames:** This option controls how many frames in the future are drawn to represent the predicted path of the object. Users can adjust the number of frames using a slider, which helps plan and visualize future motion.
### Customizing image display settings **Scene display settings:** Adjust scene display settings like brightness or contrast to suit different viewing conditions or preferences. For example, you can use them while annotation dark or low-contrast images. **Grid:** A grid helps organize the navigation on the images with high resolutions and large number of small objects.
### Visibility and image sizing The ability to hide annotation settings declutters the workspace, focusing attention on the task at hand. Real-time image resizing adapts to various project needs, ensuring optimal viewing and editing conditions. Just zoom-in or out on the images to see object details and perform precise labeling of object boundaries.
### **Playback controls** **Play/Pause buttons(ENTER)**: Standard playback controls to play or pause the video. **Play video backwards:** Plays the video from the current frame backwards, i.e. from the current frame to the previous frame, and so on to the beginning of the video. **Previous/Next frame (<- | ->):** Buttons to jump to the previous or next frame. **Previous/Next 10 frame** **(ALT + <- | ALT + ->):** Buttons to jump to the previous or next 10 frame. **Frame counter and time display**: Shows the current frame number, total number of frames, and current time position in the video (e.g., "100 / 384" or "0:04 / 0:16"). For example, `122 / 384` means you are at frame 122 out of 384 total frames. **Track length slider**: Allows you to quickly scroll through the video frames by dragging the slider left or right. Annotated segments of the video are displayed by blue color line. ### **Playback settings** **Speed:** Controls the playback speed (e.g., x2). **Skip frames:** Allows skipping a specified number of frames during navigation (e.g., +/- 10 frames). **Fast decoding mode:** Enabling this mode speeds up the video playback process. **Navigation bar settings:** Switches between different modes for the navigation bar, such as * **Figures:** show frames with objects * **Tags:** show frames with tags * **Auto:** show frames with figures/tags based current selection
*** ## Timeline and track controls It's easy to get frustrated with thousands of frames and objects labeled! The tracks panel offers controls for managing tracks, tags, and navigating through video frames. It provides a clear overview of the video's structure, answers questions about what has already been labeled, and simplifies editing tag segments and tracked objects. ![](/files/xGFS7OANHz3FU9oqEfHr) ### **Manage tracks** This dropdown list allows you to manage different tracks or objects detected in the video. A track represents a series of frames where an object is detected and labeled. By selecting options from this menu you can choose which track will be visible on the timeline. **Figures**: Filters and displays only figures detected in the video frames. **Video Tags**: Displays all tags assigned with the entire video. **Objects Tags**: Shows tags specific to objects on the frames. **Current Object Tags**: Focuses on the tags currently assigned with the selected object. **Current Tag Tags**: Filters and displays the tags that are actively being used in the current tagging session. Select one or more options to filter and view specific tags in the frame or across the video. ### **Timeline** It's a linear scale that visually displays the sequence of video frames and associated annotations. It allows users to manage annotation processes, object tracking, and other labeling tasks. The timeline contains various tracks which display data related to objects and tags. **Time scale:** A horizontal ruler that shows all video frames. Each frame is numbered, making it easy to locate specific points in time. The frame-level timeline provides more detailed control and visualization of the video on a frame-by-frame basis. The manually labeled frames are shown in gray, the frames that were automatically tracked are striped, the current frame is in a blue frame **Frame segments:** Labeled objects and tags are shown as segments on the timeline. For example, tracks for objects or tags show the start and end points of their presence in the video. **Track information display**: Displays the current segment of the video you are managing (e.g., "SEGMENT 1 / 2") and allows you to navigate between segments. * Play segment: **SHIFT + ENTER** * Previous segment: **SHIFT + <-** * Next segment: **SHIFT + ->** ### Track **Track button (SHIFT + T):** Automatically create new figures based on the selected one for the next video frames. The tracking process is started manually and performed automatically. The system analyzes the selected object or area in the frame and starts tracking its motion and changes as the video plays. **Tracking progress bar:** A progress bar appears in the interface that shows how many percent of tracking has already been done. The tracking percentage will be updated in real time as the operation progresses, and the bar will gradually fill up to reflect the progress. When the progress bar reaches 100%, it means that the tracking process has been successfully completed. The object has been tracked on all selected frames. After tracking is complete, the user can track the object if no errors are detected, or edit some frames and re-track them. **Stop tracking (SHIFT+S)**: The system stops tracking the selected object in the frame. Already created figures will remain. ### **Track settings** **Method:** Choose the object tracking algorithm. For example, "MixFormer Object Tracking" application is used to track the motion of selected objects. The predefined **Clone** command creates a duplicate (or clone) of the selected shape, object, or annotation. This clone will have the same parameters as the original, including size, position, shape, and any associated data (such as tags or metadata). **Working with clones:** A clone can be created in the same frame as the original shape, or moved to another frame to keep identical settings in different parts of the video. Once a clone is created, you can modify it independently of the original, or use it as a template to create other objects with the same settings. **Direction:** Defines the direction of tracking, such as "Forward"/"Backward". **Overwrite figures:** Auto-generated objects will be override. **Frames:** Defines the number of frames to be processed during tracking (e.g., +/- 10 frames). ### **Editing Tools** **Scissors (COMMAND + X):** Cuts figures at the current frame. **Copy (COMMAND + C):** Duplicates the selected figures. **Paste** **(COMMAND + V)**: Pastes duplicate figures. **Trash (SHIFT + D)**: Removes the selected figures. **Frame selector (SHIFT + SPACE) :** Allows you to manually select multiple frames for batch tracking, tagging and removal. *** ## **Instruments panel** [**Pan & Move Scene Tool:**](#basic-interface-elements) Quickly navigate around the image without modifying annotations. [**Select Figure**:](/labeling/labeling-tools/navigation-and-selection-tools) Select and modify existing annotations; essential for refining objects. [**Drag Figure:**](/labeling/labeling-tools/navigation-and-selection-tools) Reposition annotations without altering their size or shape. [**Point Tool:**](/labeling/labeling-tools/point-tool) Label specific points or small objects precisely. [**Bounding Box:**](/labeling/labeling-tools/bounding-box-rectangle-tool) Best for object detection tasks. [**Polyline Tool:**](/labeling/labeling-tools/polyline-tool) Annotate linear objects or edges with multiple connected line segments. [**Polygon Tool:**](/labeling/labeling-tools/polygon-tool) Ideal for irregular and complex shapes. [**Brush and Eraser Tool:**](/labeling/labeling-tools/brush-tool) Flexible for both polygonal and free-form masks. [**Mask Pen Tool:**](/labeling/labeling-tools/mask-pen-tool) Great for segmenting diverse objects with varying shapes. [**Smart Tool:**](/labeling/labeling-tools/smart-tool) Efficient for quick, AI-assisted segmentation. [**Graph (Keypoins) Tool:**](/labeling/labeling-tools/graph-keypoints-tool) For pose-estimation tasks.
*** ## Objects panel The Objects panel displays key properties and parameters of objects that can be tracked on the timeline. It is a dynamic space for showcasing and managing object tags, attributes, and metadata. Some of the features it offers include: * **Clone Objects** - Easily replicate selected objects to the next frame with a simple right arrow key press ( ->). * **Filter and Manage** - Quickly filter objects, remove all from the video, or toggle their visibility according to your needs. For example you can hide all objects except of the specific class. * **Advanced Interactions** - Select, delete, hide, merge objects, or adjust their layering. Additionally, you can modify metadata and assign or manage tags right from this window, enhancing the object's data with minimal effort.
*** ## Videos panel The videos panel provides a comprehensive view of all the videos within a selected dataset. Key functionalities include: * **Tag Management** - Clone tags from the previous video, assign new ones, or modify existing tags to maintain consistency and organization. * **Filter and Manage** - View and edit metadata details or filter through videos to find exactly what you need. * **Video Operations** - Delete videos, download them individually, or download annotations for external use. *** ## Apps panel: expand your capabilities In an ever-evolving ML landscape, the apps panel serves as a portal to a wide range of applications from the [Computer Vision Ecosystem](https://ecosystem.supervisely.com/), enhancing the functionality of your workspace. This window allows you to run and open the public or private apps and extend the Labeling Toolbox with custom UI and functionality. *** ## Settings panel The settings panel is the control center for personalizing the interface. It houses various options allowing users to tweak the interface to match their workflow, preferences, and project requirements.
*** ## **Speed up labeling with AI-assistance** Video labeling toolbox has countless features, but most importantly, it provides integrations with the best AI models that dramatically improve your labeling performance. For example: * [MixFormer](https://ecosystem.supervisely.com/apps/mixformer/serve/serve?utm_source=blog) for bounding box detection and tracking * [RITM](https://ecosystem.supervisely.com/apps/ritm-interactive-segmentation/supervisely?utm_source=blog) and [SAM](https://ecosystem.supervisely.com/apps/serve-segment-anything-model?utm_source=blog) for mask segmentation and tracking * [TAP-Net](https://ecosystem.supervisely.com/apps/serve-tapnet/tapnet/supervisely/serve?utm_source=blog) for polygon segmentation and tracking You can check this short video to get an overview on AI capabilities in the video labeling toolbox: {% embed url="" %} # Videos 3.0 **Video Annotation Tool 3.0** is designed to simplify and streamline the complex task of video annotation, which involves not only labeling multiple frames, but also tracking the relationships between them to ensure consistent object tracking and accurate labeling.
*** **Auto Tracking**: This AI-powered functionality leverages Supervisely Apps and advanced tracking models like MixFormer to automatically choose the best model for various geometry types (bounding box, skeleton, mask, etc.). Auto Tracking not only predicts labels as you move through frames but can also stop tracking when an object leaves the scene, significantly boosting labeling efficiency. **Definitions Panel**: The revamped Definitions Panel is designed to streamline video labeling by making classes quickly searchable and immediately accessible. Tags can now be assigned globally or frame-by-frame, with options to apply tags to either specific objects or entire videos. This feature allows for greater flexibility in organizing and managing video tags. **Intuitive Frame Tagging**: Tagging frames is now much simpler. Users can apply tags by checking a box on the current frame, with options to define the tag range either from the current frame to the end, or from frame 0. The tag can also be modified or removed with ease, making it straightforward to label specific ranges without unnecessary steps. **New Objects & Tags Timeline**: A major improvement in Video Toolbox 3.0, this panel displays a comprehensive overview of all objects and tags across the entire video. Users can quickly jump to specific frames, adjust tag values, and perform various actions from the context menu, while the smaller floating zoomed timeline aids in precise adjustments. **Trajectories View**: The Trajectories view visualizes object movement across frames, helping users quickly identify tracking errors, especially with static cameras. This feature is invaluable for quality control, as it shows the path of each object and makes inconsistencies in tracking easy to spot. **Quick Actions Panel**: To save time when annotating large videos, the floating quick actions panel appears beside the annotation object, enabling rapid access to functions such as moving, deleting, or adjusting bounding boxes, thereby minimizing unnecessary cursor movements. *** ## **Overview**
1. **Home button** - returns user to the main menu (Projects page) 2. [Basic interface elements](#basic-interface-elements) - basic settings, such as history of operations, theme, a hotkeys map and more useful features. 3. [Auto-Tracking](#auto-tracking) - button to start tracking and adjust tracking settings. 4. [Main scene & labeling scene settings & playback](#main-scene-and-labeling-scene-settings-and-playback) - annotation area for current video and its labels. 5. [Objects & Tags & Timeline](#objects-and-tags-and-timeline) - video timeline and overview of all objects and tags throughout the entire video. 6. [Instruments panel](#instruments-panel) - annotation tools used to create annotations. 7. [Definitions panel](#definitions-panel) - make it easy to create, manage and assign classes and tags. 8. [Videos/Apps/Settings panel](#images-panel) - list of videos in your dataset, list of additional apps you can embed into the labeling toolbox, visualization and other settings. *** ## Video Annotation Tool 3.0 Tagging and Tracking Guide This guide provides detailed instructions for using auto-tracking and tagging features in Video Annotation Tool 3.0. ### Step 1. **Run Auto-Tracking** 1. **Open the video** you want to annotate. 2. **Annotate the object** with the desired class, such as a bounding box, skeleton, or mask. 3. **Select the object** you want to track by clicking on it within the video frame. 4. **Click the Auto Tracking button** to activate the Auto Track application. By default, Auto Track will be used, but you can also select other models, like [**Serve Segment Anything**](https://app.supervisely.com/ecosystem/apps/serve-segment-anything-2) or [**MixFormer**](https://app.supervisely.com/ecosystem/apps/supervisely-ecosystem/mixformer/serve/serve), depending on your tracking needs. 5. **Configure tracking settings** by adjusting options such as: * **Auto-tracking mode**: Automatically re-tracks the object when labels change or as you move forward through frames. * **Detect off-screen**: Automatically stops tracking and removes objects when they go off-screen. * **Tracking By Detection**: Automatically finds and tracks new objects in the scene when you extend the track by clicking forward in the video. 6. **Scroll through the video** to view auto-generated label predictions and updates. If the object leaves the frame, tracking will automatically stop. ### Step 2. Configuring the Definitions Panel 1. Go to the Definitions Panel and select the necessary label classes for objects. 2. Use the search bar to quickly access the desired class. 3. Set tags as either global (for the entire video or object) or frame-based (for specific frame ranges). 4. Specify whether each tag applies to the video, object, or both. Apply global tags like "car," "pedestrian," and "traffic light," along with frame-based tags for traffic light states ("red," "green"). ### **Step 3. Applying Frame-Based Tags** 1. On the frame where you want to set a tag, check the box next to the relevant tag. 2. A window will appear where you can select the start and end points for the tag. By default, the tag will apply from the current frame to the end of the video. 3. Uncheck the box at the desired frame to end the tag range. 4. Adjust tag values or edit ranges as needed. ### Step 4. **Using the Objects & Tags Panel** 1. Go to the **Objects & Tags Panel** to get an overview of all objects and tags in the video. 2. Easily navigate to specific segments. 3. Use the minimized timeline for precise adjustment. ### **Step 5. Viewing Trajectories** 1. Activate **Trajectories** to visualize object movement across frames. 2. Review the object's movement; trajectories help identify errors, such as tracking mistakenly jumping to a different object. ## Basic interface elements The top toolbar contains options for personalizing the interface and managing data and its annotations. **Video navigation arrows (next, previous):** Allow users to move between videos in the dataset. **Undo and redo buttons:** Undo or redo the most recent annotation action. **Select theme (dark or light):** Ability to switch between light and dark interface themes, especially useful for those who prefer to work at night. **Hotkeys:** A list of hotkeys for quick access to tools. **More options:** * **Enter fullscreen** - this option allows the user to switch the interface to fullscreen mode, maximizing the workspace area. It hides browser toolbars and other elements * **Screenshot** - the screenshot function enables users to take a snapshot of the current workspace, including the video and any annotations displayed. This can be useful for documentation, sharing progress, or reviewing annotations with team members. * **Enter restore mode** - enter restore mode provides tools to recover lost or corrupted annotations. When enabled, it offers options to revert changes to a previous state or repair specific parts of the annotation dataset. * **Restore default layout** - this function resets the interface layout to its default configuration. It is useful when the layout has been modified (e.g., panels moved or resized) and the user wants to return to the original setup. *** ## Auto-Tracking This smart functionality is powered by Supervisely Apps that work on top of existing tracking apps from our [ecosystem](https://ecosystem.supervisely.com/), such as [MixFormer](https://ecosystem.supervisely.com/apps/mixformer/serve/serve). Auto Tracking intelligently selects the right model based on the type of geometry you're working with - whether it's a bounding box, skeleton, mask, or other types.\ As you move through your video or modify labels, **Auto Tracking** predicts labels automatically. But it gets better! The feature is now enhanced with the latest AI developments. For example, it can detect when an **object leaves the scene and stop tracking it**. *** **1. Auto-Tracking Mode** * **Automatic Tracking Extension**: Auto-Tracking Mode automatically extends tracking when the user reaches the last 10 frames of the current tracked segment. This feature prevents the need to manually restart tracking, ensuring seamless object tracking, especially in long or dynamic videos. * **Re-tracking on Label Changes**: When Auto-Tracking Mode is enabled, the tool will automatically re-track the object if labels are modified or as you move through frames. This is useful for dynamic scenes where the object may change shape or position, such as tracking people or vehicles. *** **2. Track Frames** * This setting allows you to specify the number of frames to track and the direction of tracking (forward or backward). * For example, setting "Track 10 frames forward" will extend tracking of the object for the next 10 frames, which is ideal for short but predictable movements, like a car moving a short distance on a road. *** **3. Interpolate Until Next Real Frame** * **Interpolation** allows the system to fill intermediate frames between keyframes, creating smoother transitions and enhancing annotation accuracy. * This feature is ideal for scenes with steady movement, such as a person walking along a straight path, where it's unnecessary to manually label every frame. *** **4. Detect Off-Screen** * **Automatic Removal of Off-Screen Objects**: If an object moves out of the camera's view, the Detect Off-Screen feature will automatically stop tracking it and remove the label from subsequent frames. * This feature is useful for videos where objects may leave the frame, such as a vehicle exiting the scene. *** **5. Tracking By Detection** * **Detecting and Tracking New Objects**: This feature enables the system to automatically detect and start tracking new objects as you extend tracking by moving forward through frames. * Tracking By Detection is ideal for crowded scenes, like busy streets, where new objects, such as pedestrians or vehicles, frequently enter the frame and require immediate tracking. *** **6. Tracking Engine** * **Selecting a Tracking Model**: Auto-Tracking supports multiple models, such as **Auto Track**, **MixFormer**, and **Serve Segment Anything**. Each model is suited to different tracking tasks - from general object tracking to precise segmentation tracking. * Users can choose the model that best fits their project needs, such as tracking complex shapes or small objects. *** ## Main scene & labeling scene settings & playback This is the central area. It displays the video to be annotated, with various display controls that the user can hide or show in the panel as needed. Main scene shows the video currently being worked on. Users can interact directly with this area using the annotation tools from the sidebar. #### Annotation display settings **Opacity:** To modify the transparency of objects, hold and drag the cursor left or right. This allows for a more nuanced view of the objects' layers. Additionally, you can hold the `SHIFT` key and scroll the mouse wheel to adjust the opacity conveniently from anywhere on the screen. **Border:** Enhance the visibility of object boundaries by holding and dragging the cursor left or right. This action changes the width of the objects' borders, allowing for clearer demarcation. Useful when working with huge resolutions or with large number of small objects **Point:** Adjust the radius of object points by holding and dragging the cursor left or right. **Default color:** Paint objects with their original colors as defined in class settings. This is the default setting and helps maintain consistency and recognition. So the objects of different classes are visually distinguishable. **Randomize color:** Randomize object colors to distinguish between objects of the same class. A simple click, followed by `SHIFT+H`, randomizes the object's colors. Can be used in Instance Segmentation Computer Vision task to highlight visual distinction of all objects of the same class on an image.
#### Attribute display settings for clearer context **ID:** Toggle the visibility of object IDs near the objects on the scene. This is essential for identifying and referring to specific objects. **Bindings:** Show or hide bindings near objects to understand how various elements are connected to each other. [Objects can be combined into groups](https://developer.supervisely.com/advanced-user-guide/objects-binding). **Tags:** Display tags near objects to provide additional context or categorization. **Classes:** Enable visibility of the classes assigned to each object, helping in the classification and organization of scene elements. **Author:** Display the creator's name near the objects to acknowledge object authorship. **Change Visibility Mode:** This option allows users to switch between different visibility modes, optimizing the scene display as per the user's preference. You can choose how to show the attributes: * **Always** | Users can select full tag display, which means that the information will be visible directly on Objects or Videos in the project. * **Show on hover** | Tags are only displayed when the cursor is hovered over the annotated object. * **Show when selected** | The ability to hide Tags until the Object is selected provides a cleaner look and feel to the interface and prevents information overload when working with a project.
#### Advanced interaction with scene objects **Auto-select:** Automatically select objects of the current shape when hovering the cursor over them. **Show object trajectory:** Visualize the path an object has taken in the scene over time. You can customize the way the trajectory looks using the following settings: * **Width (px):** This setting controls the thickness of the trajectory line in pixels. A higher value makes the line thicker, making the trajectory more visible, while a lower value makes the line thinner. * **Draw backward frames:** This option controls how many frames in the past are drawn to represent the object's trajectory. Users can adjust the number of frames using a slider, allowing them to visualize the object's past movements over a longer or shorter period of time. * **Draw forward frames:** This option controls how many frames in the future are drawn to represent the predicted path of the object. Users can adjust the number of frames using a slider, which helps plan and visualize future motion.
#### Customizing image display settings **Scene display settings:** Adjust scene display settings like brightness or contrast to suit different viewing conditions or preferences. For example, you can use them while annotation dark or low-contrast images. **Grid:** A grid helps organize the navigation on the images with high resolutions and large number of small objects.
#### Visibility and image sizing The ability to hide annotation settings declutters the workspace, focusing attention on the task at hand. Real-time image resizing adapts to various project needs, ensuring optimal viewing and editing conditions. Just zoom-in or out on the images to see object details and perform precise labeling of object boundaries.
#### **Playback controls** **Play/Pause buttons(ENTER)**: Standard playback controls to play or pause the video. **Play video backwards:** Plays the video from the current frame backwards, i.e. from the current frame to the previous frame, and so on to the beginning of the video. **Previous/Next frame (<- | ->):** Buttons to jump to the previous or next frame. **Previous/Next 10 frame** **(ALT + <- | ALT + ->):** Buttons to jump to the previous or next 10 frame. **Frame counter and time display**: Shows the current frame number, total number of frames, and current time position in the video (e.g., "100 / 384" or "0:04 / 0:16"). For example, `122 / 384` means you are at frame 122 out of 384 total frames. **Track length slider**: Allows you to quickly scroll through the video frames by dragging the slider left or right. Annotated segments of the video are displayed by blue color line. #### **Playback settings** **Speed:** Controls the playback speed (e.g., x2). **Skip frames:** Allows skipping a specified number of frames during navigation (e.g., +/- 10 frames). **Fast decoding mode:** Enabling this mode speeds up the video playback process. **Navigation bar settings:** Switches between different modes for the navigation bar, such as * **Figures:** show frames with objects * **Tags:** show frames with tags * **Auto:** show frames with figures/tags based current selection
## Objects & Tags & Timeline The **Objects & Tags Panel** provides a clear overview of all objects and tags in your video, allowing for efficient navigation and editing. Instead of relying on a single, zoomed-in timeline (which only shows about 50 frames at a time), this panel gives you a **clear overview** of all objects and tags throughout the entire video. You can easily jump to specific segments, adjust tag values, or use various actions from the context menu.
We've kept the floating zoomed timeline as well, but now it's much smaller and designed to help you focus on precision while the Objects & Tags Panel gives you the full picture. You may click on the interesting frame or use your mouse scroll. ![](https://cdn.supervisely.com/blog/video-annotation-update-2024/timeline-floating-window.png?width=800) 1. **Access the Objects & Tags Panel**: This panel displays a timeline for each object and tag, making it easy to see when and where tags have been applied across the video. 2. **Navigate and Select Objects**: Each object (e.g., "car," "bus") has an assigned ID and color-coded timeline. You can select specific objects or tags by clicking on them directly in the panel, allowing you to focus on individual elements without switching frames. 3. **Adjust Tag Ranges**: Each tag (e.g., "Road position," "Lane change") is shown on the timeline with segmented bars indicating its duration. Tags like "On-lane" or "Off-lane" help indicate changes across the timeline. * To edit a tag, click on it directly on the timeline. You can drag the edges to adjust its duration or click within the segment to modify it further. 4. **Remove Tags**: To delete a tag, click on the **three dots menu** next to the tag entry. Select **Remove Tag** from the dropdown menu, as shown in the screenshot, to delete it from the selected range. 5. **Control Visibility and Filter**: Use the **Visible/All** filter toggle at the top of the panel to control which objects and tags are visible on the timeline. This helps to declutter the view and focus on specific elements. 6. **Use Timeline for Precision**: Each object and tag timeline is color-coded, allowing for easy identification. The minimized timeline view lets you accurately pinpoint where tags start and end, making adjustments quick and precise. ## Merge Objects You can merge multiple objects into one. This is particularly useful when you have annotated the same object multiple times and want to consolidate them for better organization. All step numbers align with those on the accompanying image: 1. Select the “Merge objects” mode on the timeline.
2. Move the slider to the desired frames to view the relevant objects in the preview. 3. Select the object you wish to merge. 4. Click “Add to merger.” 5. An indicator will confirm that the object has been added.
6. Move the slider on the timeline to the frame containing the next object of interest. 7. Click “Add to merger” for this object as well. 8. Once both objects are selected, click the now-active “Merge 2 objects” button.
*** ### **Instruments panel** [**Pan & Move Scene Tool:**](https://docs.supervisely.com/labeling/labeling-toolbox/videos#basic-interface-elements) Quickly navigate around the image without modifying annotations. [**Select Figure**:](https://docs.supervisely.com/labeling/labeling-tools/navigation-and-selection-tools) Select and modify existing annotations; essential for refining objects. [**Drag Figure:**](https://docs.supervisely.com/labeling/labeling-tools/navigation-and-selection-tools) Reposition annotations without altering their size or shape. [**Point Tool:**](https://docs.supervisely.com/labeling/labeling-tools/point-tool) Label specific points or small objects precisely. [**Bounding Box:**](https://docs.supervisely.com/labeling/labeling-tools/bounding-box-rectangle-tool) Best for object detection tasks. [**Polyline Tool:**](https://docs.supervisely.com/labeling/labeling-tools/polyline-tool) Annotate linear objects or edges with multiple connected line segments. [**Polygon Tool:**](https://docs.supervisely.com/labeling/labeling-tools/polygon-tool) Ideal for irregular and complex shapes. [**Brush and Eraser Tool:**](https://docs.supervisely.com/labeling/labeling-tools/brush-tool) Flexible for both polygonal and free-form masks. [**Mask Pen Tool:**](https://docs.supervisely.com/labeling/labeling-tools/mask-pen-tool) Great for segmenting diverse objects with varying shapes. [**Smart Tool:**](https://docs.supervisely.com/labeling/labeling-tools/smart-tool) Efficient for quick, AI-assisted segmentation. [**Graph (Keypoins) Tool:**](https://docs.supervisely.com/labeling/labeling-tools/graph-keypoints-tool) For pose-estimation tasks.
*** ## Definitions Panel In Video Toolbox 3.0, the Definitions Panel has been revamped for video labeling, building on what we've previously [introduced](https://supervisely.com/blog/definitions-panel/) for image labeling. Just like with image labeling, you can now quickly search for classes and start labeling right away without needing to manually select the appropriate tool. All your classes of labels are presented clearly on your screen for fast access. For video, we've added the ability to assign tags globally or frame-by-frame. Global tags can be applied to entire videos or specific objects, while frame-based tags let you mark specific ranges. You can even configure whether a tag is assigned only to the video, only to objects, or both. Global tags are assigned as usual: to the selected annotation object or to the current video, if none selected. In previous versions, this was done by clicking the "clip" icon near the tag.
The Definitions panel provides a simple interface for creating and managing classes and tags in a project. It helps users organize and control annotations. Instead of choosing a tool first, you can now click on the desired class from the Definitions Panel. The associated tool will be automatically selected, allowing you to start labeling immediately. To start a new label, simply click on any class (including the currently selected one) in the Definitions Panel. If you want to change the class of the selected object, you can click the small icon on the right, which appears only if the new class shape matches the currently selected object. Tags are also present on the same panel. If no object is selected, image tags are shown. You can check the desired tag or hover the cursor and start typing a tag value or select it from a dropdown, which will automatically assign it. **Improved Search:** To find classes or tags more easily in a long list, click the magnifying glass icon in the top right corner of the panel. Type your query and select the desired class or tag to continue your workflow. *** ## Videos panel The videos panel provides a comprehensive view of all the videos within a selected dataset. Key functionalities include: * **Tag Management** - Clone tags from the previous video, assign new ones, or modify existing tags to maintain consistency and organization. * **Filter and Manage** - View and edit metadata details or filter through videos to find exactly what you need. * **Video Operations** - Delete videos, download them individually, or download annotations for external use. *** ## Apps panel: expand your capabilities In an ever-evolving ML landscape, the apps panel serves as a portal to a wide range of applications from the [Computer Vision Ecosystem](https://ecosystem.supervisely.com/), enhancing the functionality of your workspace. This window allows you to run and open the public or private apps and extend the Labeling Toolbox with custom UI and functionality. *** ## Settings panel The settings panel is the control center for personalizing the interface. It houses various options allowing users to tweak the interface to match their workflow, preferences, and project requirements.
--- [Next Page](/llms-full.txt/1)