Supervisely
AboutAPI ReferenceSDK Reference
  • 🤖What's Supervisely
  • 🚀Ecosystem of Supervisely Apps
  • 💡FAQ
  • 📌Getting started
    • How to import
    • How to annotate
    • How to invite team members
    • How to connect agents
    • How to train models
  • 🔁Import and Export
    • Import
      • Overview
      • Import using Web UI
      • Supported annotation formats
        • Images
          • 🤖Supervisely JSON
          • 🤖Supervisely Blob
          • COCO
          • Yolo
          • Pascal VOC
          • Cityscapes
          • Images with PNG masks
          • Links from CSV, TXT and TSV
          • PDF files to images
          • Multiview images
          • Multispectral images
          • Medical 2D images
          • LabelMe
          • LabelStudio
          • Fisheye
          • High Color Depth
        • Videos
          • Supervisely
        • Pointclouds
          • Supervisely
          • .PCD, .PLY, .LAS, .LAZ pointclouds
          • Lyft
          • nuScenes
          • KITTI 3D
        • Pointcloud Episodes
          • Supervisely
          • .PCD, .PLY, .LAS, .LAZ pointclouds
          • Lyft
          • nuScenes
          • KITTI 360
        • Volumes
          • Supervisely
          • .NRRD, .DCM volumes
          • NIfTI
      • Import sample dataset
      • Import into an existing dataset
      • Import using Team Files
      • Import from Cloud
      • Import using API & SDK
      • Import using agent
    • Migrations
      • Roboflow to Supervisely
      • Labelbox to Supervisely
      • V7 to Supervisely
      • CVAT to Supervisely
    • Export
  • 📂Data Organization
    • Core concepts
    • MLOps Workflow
    • Projects
      • Datasets
      • Definitions
      • Collections
    • Team Files
    • Disk usage & Cleanup
    • Quality Assurance & Statistics
      • Practical applications of statistics
    • Operations with Data
      • Data Filtration
        • How to use advanced filters
      • Pipelines
      • Augmentations
      • Splitting data
      • Converting data
        • Convert to COCO
        • Convert to YOLO
        • Convert to Pascal VOC
    • Data Commander
      • Clone Project Meta
  • 📝Labeling
    • Labeling Toolboxes
      • Images
      • Videos 2.0
      • Videos 3.0
      • 3D Point Clouds
      • DICOM
      • Multiview images
      • Fisheye
    • Labeling Tools
      • Navigation & Selection Tools
      • Point Tool
      • Bounding Box (Rectangle) Tool
      • Polyline Tool
      • Polygon Tool
      • Brush Tool
      • Mask Pen Tool
      • Smart Tool
      • Graph (Keypoints) Tool
      • Frame-based tagging
    • Labeling Jobs
      • Labeling Queues
      • Labeling Consensus
      • Labeling Statistics
    • Labeling with AI-Assistance
  • 🤝Collaboration
    • Admin panel
      • Users management
      • Teams management
      • Server disk usage
      • Server trash bin
      • Server cleanup
      • Server stats and errors
    • Teams & workspaces
    • Members
    • Issues
    • Guides & exams
    • Activity log
    • Sharing
  • 🖥️Agents
    • Installation
      • Linux
      • Windows
      • AMI AWS
      • Kubernetes
    • How agents work
    • Restart and delete agents
    • Status and monitoring
    • Storage and cleanup
    • Integration with Docker
  • 🔮Neural Networks
    • Overview
    • Inference & Deployment
      • Overview
      • Supervisely Serving Apps
      • Deploy & Predict with Supervisely SDK
      • Using trained models outside of Supervisely
    • Model Evaluation Benchmark
      • Object Detection
      • Instance Segmentation
      • Semantic Segmentation
      • Custom Benchmark Integration
    • Custom Model Integration
      • Overview
      • Custom Inference
      • Custom Training
    • Legacy
      • Starting with Neural Networks
      • Train custom Neural Networks
      • Run pre-trained models
  • 👔Enterprise Edition
    • Get Supervisely
      • Installation
      • Post-installation
      • Upgrade
      • License Update
    • Kubernetes
      • Overview
      • Installation
      • Connect cluster
    • Advanced Tuning
      • HTTPS
      • Remote Storage
      • Single Sign-On (SSO)
      • CDN
      • Notifications
      • Moving Instance
      • Generating Troubleshoot Archive
      • Storage Cleanup
      • Private Apps
      • Data Folder
      • Firewall
      • HTTP Proxy
      • Offline usage
      • Multi-disk usage
      • Managed Postgres
      • Scalability Tuning
  • 🔧Customization and Integration
    • Supervisely .JSON Format
      • Project Structure
      • Project Meta: Classes, Tags, Settings
      • Tags
      • Objects
      • Single-Image Annotation
      • Single-Video Annotation
      • Point Cloud Episodes
      • Volumes Annotation
    • Developer Portal
    • SDK
    • API
  • 💡Resources
    • Changelog
    • GitHub
    • Blog
    • Ecosystem
Powered by GitBook
On this page
  • Project Structure Example
  • Main concepts
  • Format of annotation.json
  • Key id map file
  • Format of frame_pointcloud_map.json
  • Photo context image annotation file
  • SDK

Was this helpful?

  1. Customization and Integration
  2. Supervisely .JSON Format

Point Cloud Episodes

PreviousSingle-Video AnnotationNextVolumes Annotation

Last updated 1 year ago

Was this helpful?

Project Structure Example

Root folder for the project named project name

  • meta.json file

  • key_id_map.json file

  • Dataset folders, that represents single episode. Each named dataset_name, which contains:

    • annotation.json - file with whole episode annotation

    • frame_pointcloud_map.json - file with pointcloud to episode frame mapping

    • pointcloud folder, contains source point cloud files, for example frame1.pcd, frame2.pcd

    • related_images optional folder, contains photo-context data:

      • Frame folder, each named according to pointcloud (/related_images/frame1/), which contains:

        • image files (.png \ .jpg)

        • image ann files (.json) - json files, named according to image name (1.png -> 1.json)

Main concepts

Point cloud Episode Project Folder

On the top level are stored Project folders, these are the elements visible on the main Supervisely dashboard. Inside them they can contain only Datasets and Poject Meta information, all other data has to be stored a level below in a Dataset (Sequence). All datasets within a project have to contain content of the same cathegory.

Project Meta

Project Meta contains the essential information about the project - Classes and Tags. These are defined project-wide and can be used for labeling in every dataset inside the current project.

Datasets (Sequences)

Datasets are the second level folders inside the project, they host the individual data files (pointclouds), photo context (images) and their annotations.

Items (Point clouds)

Every .pcd file in sequence has to be stored inside a dataset/pointcloud folder. Each file has file name and order number, that defined inside frame_pointcloud_map.json.

Items Annotation

A sequence of frames has a list of objects that are shared between frames. For example, sequence might have 10 cars objects represented by theirs figures in 100 frames.

The list of objects is defined for the entire sequence, even if the object's figure occurs in only one frame.

Figures - represents individual cuboids, attached to one single frame and its object.

Example:
The sequence contains 3 objects: (car1, car2, pedestrian1) and 10 frames.

Each frame in the sequence might have a figure for every object.
Then it will be 30 figures (10 figures for each object per sequence; 1 figure for the object per frame).

Linking between point clouds , Objects and Figures

In Supervisely Annotation Format each point cloud is a frame. Frame contain figures and index: sequence number, that have relation to point cloud name in frame_pointcloud_map.json.

In the example below, created one Car object (with key "6663ca1d20c74bea83bd48c24568989d") for the entire sequence. (Means that this object can appear in any frame of the sequence)

Then, on frames (with indices 0 and 1) figures are placed, and each figure correspond to the Car object by the objectKey field.

Format of annotation.json

[
    {
    "description": "",
    "key": "e9f0a3ae21be41d08eec166d454562be",
    "tags": [],
    "objects": [
        {
            "key": "6663ca1d20c74bea83bd48c24568989d",
            "classTitle": "car",
            "tags": []
        }],
    "framesCount": 48,
    "frames": [
         {
            "index": 0,
            "figures": [
               {
                "key": "cb8e067dadfc423aa8575a0c4e62de33",
                "objectKey": "6663ca1d20c74bea83bd48c24568989d",
                "geometryType": "cuboid_3d",
                "geometry": {
                    "position": {
                        "x": -10.863547325134277,
                        "y": -93.57706451416016,
                        "z": -4.598618030548096
                    },
                    "rotation": {
                        "x": 0,
                        "y": 0,
                        "z": 3.250733629393711
                    },
                    "dimensions": {
                        "x": 1.978,
                        "y": 4.607,
                        "z": 1.552
                        }
                      }
                    }
            ]
         },
         {
            "index": 1,
            "figures": [               
               {
                "key": "71e0fe52dc4f4f6aaf059ad095f43c1f",
                "objectKey": "6663ca1d20c74bea83bd48c24568989d",
                "labelerLogin": "username",
                "updatedAt": "2021-11-11T17:19:11.448Z",
                "createdAt": "2021-11-11T16:53:03.670Z",
                "geometryType": "cuboid_3d",
                "geometry": {
                    "position": {
                        "x": -11.10418701171875,
                        "y": -91.33098602294922,
                        "z": -4.5446248054504395
                    },
                    "rotation": {
                        "x": 0,
                        "y": 0,
                        "z": 3.24780199600921
                    },
                    "dimensions": {
                        "x": 1.978,
                        "y": 4.607,
                        "z": 1.552
                    }
                }
              }
            ]
         }
    ]
    }
]

Optional fields and loading These fields are optional and are not needed when loading the project. The server can automatically fill in these fields while project is loading.

  • id - unique identifier of the current object

  • classId - unique class identifier of the current object

  • labelerLogin - string - the name of user who created the current figure

  • createdAt - string - date and time of figure creation

  • updatedAt - string - date and time of the last figure update

Fields definitions:

  • description - string - (optional) - this field is used to store the text to assign to the sequence.

  • key - string, unique key for a given sequence (used in key_id_map.json to get the sequence ID)

  • tags - list of strings that will be interpreted as episode tags

  • objects - list of objects that may be present on the episode

  • frames - list of frames of which the episode consists. List contains only frames with an object from the 'objects' field

    • index - integer - number of the current frame

    • figures - list of figures in the current frame.

  • framesCount - integer - total number of frames in the episode

  • geometryType - "cuboid_3d" - class shape

Fields definitions for objects field:

  • key - string - unique key for a given object (used in key_id_map.json)

  • classTitle - string - the title of a class. It's used to identify the class shape from the meta.json file

  • tags - list of strings that will be interpreted as object tags (can be empty)

Fields description for figures field:

  • key - string - unique key for a given figure (used in key_id_map.json)

  • objectKey - string - unique key to link figure to object (used in key_id_map.json)

  • geometryType - "cuboid_3d" -class shape

  • geometry - geometry of the object

Description for geometry field:

  • position 3D vector X, Y, Z values matches the axes on world coordinates, defined in global frame of reference as:

    • +x - forward in the direction of travel ego vehicle

    • +y - left

    • +z - up

  • dimensions is 3D vector with:

    • x - width

    • y - length

    • z - height

  • rotationis 3D Vector with:

    • x - pitch

    • y - roll

    • z - yaw (direction)

Rotation values bound inside [-pi ; pi ] When yaw = 0 box direction will be strict +y

Key id map file

The basic idea behind key-id-map is that it maps the unique identifiers of the object to the frame on which the shape is located. The server works with an identifier, but the file system of the loaded project stores these identifiers and object keys on disk, which is necessary for navigation and use of the high-level API and applications.

When loading a dataset (sequence), the system returns its identifier, after which it is saved to a file on disk, along with the key of the loaded sequence in key-id-map file.

When uploading objects to the server, a sequence ID is required (to determine which sequence the object belongs to), and it can be read from the key-id-map file by key. The system then returns the IDs of the successfully loaded objects.

Then, while figures uploading to the server, an object identifier is required (which loaded object the shape belongs to), which can again be read from the key-id-map file.

While annotating the episode inside Supervisely interface key-id-map file is created automatically, and will be downloaded with entire project. Json format of key_id_map.json:

{
    "tags": {},
    "objects": {
        "198f727d40c749eebcacc4aed299b39a": 20520
    },
    "figures": {
        "65f21690780e43b49863c3cbd07eab3a": 503130811
    },
    "videos": {
        "e9f0a3ae21be41d08eec166d454562be": 42656
    }
}
  • objects - dictionary, where the key is a unique string, generated inside Supervisely environment to set correspondence of current object in annotation, and values are unique integer ID corresponding to the current object

  • figures - dictionary, where the key is a unique string, generated inside Supervisely environment to set correspondence of object on current frame in annotation, and values are unique integer ID corresponding to the current frame

  • videos - dictionary, where the key is unique string, generated inside Supervisely environment to set correspondence of sequence (dataset) in annotation, and value is a unique integer ID corresponding to the current sequence

  • tags - dictionary, where the keys are unique strings, generated inside Supervisely environment to set correspondence of tag on current frame in annotation, and values are a unique integer ID corresponding to the current tag

  • Value - returned by server integer identifier while uploading object / figure / sequence / tag

Format of frame_pointcloud_map.json

This file create for mapping between pointcloud files and annotation frames in the correct order.

{
    "0" : "frame1.pcd",
    "1" : "frame2.pcd",
    "2" : "frame3.pcd" 
}

Keys - frame order number Values - point cloud name (with extension)

Photo context image annotation file

    {
        "name": "host-a005_cam4_1231201437716091006.jpeg"
        "entityId": 2359620,
        "meta": {
            "deviceId": "CAM_BACK_LEFT",
            "timestamp": "2019-01-11T03:23:57.802Z",
            "sensorsData": {
                "extrinsicMatrix": [
                    -0.8448329028461443,
                    -0.5350302199120708,
                    0.00017334762588639086,
                    -0.012363736761232369,
                    -0.0035124448582330757,
                    0.005222293412494302,
                    -0.9999801949951969,
                    -0.16621728572112304,
                    0.5350187183638307,
                    -0.8448167798004226,
                    -0.006291229448121315,
                    -0.3527897896721229
                ],
                "intrinsicMatrix": [
                    882.42699274,
                    0,
                    602.047851885,
                    0,
                    882.42699274,
                    527.99972239,
                    0,
                    0,
                    1
                ]
            }
        }
    }

Fields description:

  • name - string - Name of image file

  • Id - (OPTIONAL) - integer >= 1 ID of the photo in the system. It is not required when upload and is filled in automatically when the project is loaded.

  • entityId (OPTIONAL) - integer >= 1 ID of the Point Cloud in the system, that photo attached to. Doesn't required while uploading.

  • deviceId - string- Device ID or name.

  • timestamp - string - Time when the frame occurred in ISO 8601 format

SDK

Work with Supervisely projects involves two ways:

  1. Labeling within the interface and use of default download / upload applications.

  2. Using the API and SDK for manual labeling and data management.

$ git clone https://github.com/supervisely-ecosystem/sdk_pointcloud_episode_example.git
$ cd sdk_pointcloud_episode_example  
$ pip -r requirements.txt

This repo stores some test files, the main script main.py and requirements.txt After clone the repository, install the supervise library sdk from the requirements file.

Then, you can run the code, after specifying your token, server address and workspace id.

The PCD file format description can be found

Main idea of key fields and id you can see below in section.

You can avoid using key-id-map directly with API and SDK to create your own file structure. An example will be shown below in the .

Key - . The unique string. All key values and id's should be unique inside single project and can not be shared between frames\sequences.

sensorsData - Sensors data such as Pinhole camera model parameters. See wiki: and .

intrinsicMatrix - Array of number - 3x3 flatten matrix (dropped last zeros column) of intrinsic parameters in row-major order, also called camera matrix. It's used to denote camera calibration parameters. See .

extrinsicMatrix - Array of number - 4x3 flatten matrix (dropped last zeros column) of extrinsic parameters in row-major order, also called joint rotation-translation matrix. It's used to denote the coordinate system transformations from 3D world coordinates to 3D camera coordinates. See .

Trying API and SDK is possible in the documented step-by-step .

If you do not have the necessary credentials for connection, please contact technical support via or .

🔧
here
generated by python3 function uuid.uuid4().hex
Pinhole camera model
OpenCV docs for 3D reconstruction
Intrinsic parameters
Extrinsic_parameters
example repository
website
slack channel
Key id map file
SDK section
3D Episodes labeling interface