Supervisely
AboutAPI ReferenceSDK Reference
  • 🤖What's Supervisely
  • 🚀Ecosystem of Supervisely Apps
  • 💡FAQ
  • 📌Getting started
    • How to import
    • How to annotate
    • How to invite team members
    • How to connect agents
    • How to train models
  • 🔁Import and Export
    • Import
      • Overview
      • Import using Web UI
      • Supported annotation formats
        • Images
          • 🤖Supervisely JSON
          • 🤖Supervisely Blob
          • COCO
          • Yolo
          • Pascal VOC
          • Cityscapes
          • Images with PNG masks
          • Links from CSV, TXT and TSV
          • PDF files to images
          • Multiview images
          • Multispectral images
          • Medical 2D images
          • LabelMe
          • LabelStudio
          • Fisheye
          • High Color Depth
        • Videos
          • Supervisely
        • Pointclouds
          • Supervisely
          • .PCD, .PLY, .LAS, .LAZ pointclouds
          • Lyft
          • nuScenes
          • KITTI 3D
        • Pointcloud Episodes
          • Supervisely
          • .PCD, .PLY, .LAS, .LAZ pointclouds
          • Lyft
          • nuScenes
          • KITTI 360
        • Volumes
          • Supervisely
          • .NRRD, .DCM volumes
          • NIfTI
      • Import sample dataset
      • Import into an existing dataset
      • Import using Team Files
      • Import from Cloud
      • Import using API & SDK
      • Import using agent
    • Migrations
      • Roboflow to Supervisely
      • Labelbox to Supervisely
      • V7 to Supervisely
      • CVAT to Supervisely
    • Export
  • 📂Data Organization
    • Core concepts
    • MLOps Workflow
    • Projects
      • Datasets
      • Definitions
      • Collections
    • Team Files
    • Disk usage & Cleanup
    • Quality Assurance & Statistics
      • Practical applications of statistics
    • Operations with Data
      • Data Filtration
        • How to use advanced filters
      • Pipelines
      • Augmentations
      • Splitting data
      • Converting data
        • Convert to COCO
        • Convert to YOLO
        • Convert to Pascal VOC
    • Data Commander
      • Clone Project Meta
  • 📝Labeling
    • Labeling Toolboxes
      • Images
      • Videos 2.0
      • Videos 3.0
      • 3D Point Clouds
      • DICOM
      • Multiview images
      • Fisheye
    • Labeling Tools
      • Navigation & Selection Tools
      • Point Tool
      • Bounding Box (Rectangle) Tool
      • Polyline Tool
      • Polygon Tool
      • Brush Tool
      • Mask Pen Tool
      • Smart Tool
      • Graph (Keypoints) Tool
      • Frame-based tagging
    • Labeling Jobs
      • Labeling Queues
      • Labeling Consensus
      • Labeling Statistics
    • Labeling with AI-Assistance
  • 🤝Collaboration
    • Admin panel
      • Users management
      • Teams management
      • Server disk usage
      • Server trash bin
      • Server cleanup
      • Server stats and errors
    • Teams & workspaces
    • Members
    • Issues
    • Guides & exams
    • Activity log
    • Sharing
  • 🖥️Agents
    • Installation
      • Linux
      • Windows
      • AMI AWS
      • Kubernetes
    • How agents work
    • Restart and delete agents
    • Status and monitoring
    • Storage and cleanup
    • Integration with Docker
  • 🔮Neural Networks
    • Overview
    • Inference & Deployment
      • Overview
      • Supervisely Serving Apps
      • Deploy & Predict with Supervisely SDK
      • Using trained models outside of Supervisely
    • Model Evaluation Benchmark
      • Object Detection
      • Instance Segmentation
      • Semantic Segmentation
      • Custom Benchmark Integration
    • Custom Model Integration
      • Overview
      • Custom Inference
      • Custom Training
    • Legacy
      • Starting with Neural Networks
      • Train custom Neural Networks
      • Run pre-trained models
  • 👔Enterprise Edition
    • Get Supervisely
      • Installation
      • Post-installation
      • Upgrade
      • License Update
    • Kubernetes
      • Overview
      • Installation
      • Connect cluster
    • Advanced Tuning
      • HTTPS
      • Remote Storage
      • Single Sign-On (SSO)
      • CDN
      • Notifications
      • Moving Instance
      • Generating Troubleshoot Archive
      • Storage Cleanup
      • Private Apps
      • Data Folder
      • Firewall
      • HTTP Proxy
      • Offline usage
      • Multi-disk usage
      • Managed Postgres
      • Scalability Tuning
  • 🔧Customization and Integration
    • Supervisely .JSON Format
      • Project Structure
      • Project Meta: Classes, Tags, Settings
      • Tags
      • Objects
      • Single-Image Annotation
      • Single-Video Annotation
      • Point Cloud Episodes
      • Volumes Annotation
    • Developer Portal
    • SDK
    • API
  • 💡Resources
    • Changelog
    • GitHub
    • Blog
    • Ecosystem
Powered by GitBook
On this page
  • Project Structure System
  • Downloaded Project Structure
  • Project Structure Eample
  • Extended Project Structure
  • Understanding Blob Files and Offsets for Optimized Project Handling

Was this helpful?

  1. Customization and Integration
  2. Supervisely .JSON Format

Project Structure

PreviousSupervisely .JSON FormatNextProject Meta: Classes, Tags, Settings

Last updated 28 days ago

Was this helpful?

In Supervisely, all data and annotations are stored inside individual projects which consist of datasets containing files and Project Meta - a collection of classes and tags.

When downloaded, each project is converted into a folder structure that includes a meta.json file containing Project Meta, and dataset folders with individual annotation files (and optionally the original data files). This organization enables seamless data transfer between Supervisely and local storage using the Supervisely Format import plugin when needed.

This structure remains the same for every type of project in Supervisely.

Project Structure System

Project Folder

On the top level we have Project folders, these are the elements visible on the main Supervisely dashboard. Inside them, they can contain only Datasets and Project Meta information, all other data has to be stored a level below in a Dataset. All datasets within a project have to contain content of the same category.

Project Meta

Project Meta contains essential information about the project, including Classes and Tags, which are defined project-wide and can be used for labeling in any dataset within the current project. It also includes the Project Type and Settings, which configure the labeling interface.

Datasets

Datasets are the second level folders inside the project, they host the individual data files and their annotations.

Items

Every data file in the project has to be stored inside a dataset. Each file as its own set of annotations.

Downloaded Project Structure

All projects downloaded from Supervisely maintain the same basic structure, with the contents varying based on which download option you chose.

Download Archive

When you select one of the download option, the system automatically creates an archive with the following name structure: project_name.tar

Downloaded Project

All projects downloaded from Supervisely have the following structure:

📂 Root folder for the project named project name:

  • 📄 meta.json file

  • 📂 Dataset folders, each named dataset_name, which contains:

    • 📂 ann folder, contains annotation files, each named source_media_file_name.json for the corresponding file

    • 📂 img (video or pointcloud) folder, contains source media

    • 📂 img_info folder, contains JSON files with representation of ImageInfo downloaded from instance

    • 📂 meta optional folder, contains corresponding JSON files with metadata for images

Project Structure Eample

The following structure is an example of a project with 2 datasets, each containing 2 images with annotations, and also meta directory with metadata for each image.

📦 project-name
 ┣ 📂 dataset-name-001
 ┃ ┣ 📂 ann
 ┃ ┃ ┣ 📄 pexels-photo-101063.png.json
 ┃ ┃ ┗ 📄 pexels-photo-103127.png.json
 ┃ ┣ 📂 img
 ┃ ┃ ┣ 🏞️ pexels-photo-101063.png
 ┃ ┃ ┗ 🏞️ pexels-photo-103127.png
 ┃ ┣ 📂 meta
 ┃ ┃ ┣ 📄 pexels-photo-101063.png.json
 ┃ ┃ ┗ 📄 pexels-photo-103127.png.json
 ┃ ┣ 📂 img_info
 ┃ ┃ ┣ 📄 pexels-photo-101063.png.json
 ┃ ┃ ┗ 📄 pexels-photo-103127.png.json
 ┣ 📂 dataset-name-002
 ┃ ┣ 📂 ann
 ┃ ┃ ┣ 📄 pexels-photo-100583.png.json
 ┃ ┃ ┗ 📄 pexels-photo-106118.png.json
 ┃ ┣ 📂 img
 ┃ ┃ ┣ 🏞️ pexels-photo-100583.png
 ┃ ┃ ┗ 🏞️ pexels-photo-106118.png
 ┃ ┗ 📂 meta
 ┃ ┃ ┣ 📄 pexels-photo-100583.png.json
 ┃ ┃ ┗ 📄 pexels-photo-106118.png.json
 ┃ ┣ 📂 img_info
 ┃ ┃ ┣ 📄 pexels-photo-100583.png.json
 ┃ ┃ ┗ 📄 pexels-photo-106118.png.json
 ┗ 📄 meta.json

Extended Project Structure

A project directory may contain the following folders or files:

  • 📂 blob optional folder, contains blob files that are used for optimized uploads of projects. These blob files are TAR archives with hundreds of thousands of small images.

  • 📄 key_id_map.json - optional file, created when annotating inside the Supervisely interface. Establishes correspondence between unique identifiers (keys and IDs) of items, objects, and frames where objects are located. The project file system stores these identifiers and keys on disk, which is necessary for navigation and for using the high-level API and applications.

    A dataset directory may contain the following folders or files:

    • 📄 blob_1_offsets.pkl optional pickle files, contain batches (lists) of BlobImageInfo objects, which represent file names and their offsets inside blob files. These files are used to add images to the project dataset based on their offsets.

Understanding Blob Files and Offsets for Optimized Project Handling

Supervisely provides a powerful optimization for projects containing a large number of small image files through its blob file system. Instead of handling thousands of individual files (which can lead to significant overhead in network transfers and filesystem operations), blob files consolidate many images into a single large binary file. This approach dramatically improves upload and download speeds, especially when dealing with datasets containing tens or hundreds of thousands of images.

Complementing the blob files are offset files with the suffix _offsets.pkl, which store metadata about each image's location within the blob. These files contain BlobImageInfo objects that define the byte range representing each image in the binary.

📂 project-name
 ┣ 📂 blob
 ┃  ┗ 📦 small_images.tar
 ┣ 📂 dataset-name-001
 ┃  ┣ 📄 small_images_offsets.pkl
 ┃  ┣ 📂 ann
 ┃  ┃  ┣ 📄 pexels-photo-101063.png.json
 ┃  ┃  ┣ 📄 small-image-0000001.png.json
 ┃  ┃  ┣ ...
 ┃  ┃  ┗ 📄 small-image-0999999.png.json
 ┃  ┗ 📂 img
 ┃     ┗ 🏞️ pexels-photo-101063.png
 ┗ 📄 meta.json

Related:

📄 obj_class_to_machine_color.json - optional file for image annotation projects. Mapping between machine colors and classes in machine mask. Could be generated by applications such as

📂 masks_human optional folder for image annotation projects, contains .png files with RGB semantic segmentation masks where every pixel has the color of the corresponding class. Could be generated by applications such as

📂 masks_machine optional folder for image annotation projects, contains .png files with semantic segmentation masks (machine annotations). This files should have the same name as the original images (may have a different extension). Could be generated by applications such as

📂 masks_instances optional folder contains BW instance segmentation masks for every object on the image. Could be generated by applications such as

To learn more about extended Supervisely format with blob files, refer to this page:

To export extended Supervisely format with the blob files and offsets, use the application. ☝️ However, other applications export projects in the Supervisely format using the traditional method, downloading each image separately.

Importing the extended Supervisely format happens automatically in applications that previously imported projects in the Supervisely format without blobs. Such as application or tool.

🔧
Export As Masks
Export As Masks
Export As Masks
Export As Masks
🤖 Supervisely Blob
Export to Supervisely format: Blob
Import Images in Supervisely Format
Auto Import
project_structure system
project_structure system