Supervisely
AboutAPI ReferenceSDK Reference
  • 🤖What's Supervisely
  • 🚀Ecosystem of Supervisely Apps
  • 💡FAQ
  • 📌Getting started
    • How to import
    • How to annotate
    • How to invite team members
    • How to connect agents
    • How to train models
  • 🔁Import and Export
    • Import
      • Overview
      • Import using Web UI
      • Supported annotation formats
        • Images
          • 🤖Supervisely JSON
          • 🤖Supervisely Blob
          • COCO
          • Yolo
          • Pascal VOC
          • Cityscapes
          • Images with PNG masks
          • Links from CSV, TXT and TSV
          • PDF files to images
          • Multiview images
          • Multispectral images
          • Medical 2D images
          • LabelMe
          • LabelStudio
          • Fisheye
          • High Color Depth
        • Videos
          • Supervisely
        • Pointclouds
          • Supervisely
          • .PCD, .PLY, .LAS, .LAZ pointclouds
          • Lyft
          • nuScenes
          • KITTI 3D
        • Pointcloud Episodes
          • Supervisely
          • .PCD, .PLY, .LAS, .LAZ pointclouds
          • Lyft
          • nuScenes
          • KITTI 360
        • Volumes
          • Supervisely
          • .NRRD, .DCM volumes
          • NIfTI
      • Import sample dataset
      • Import into an existing dataset
      • Import using Team Files
      • Import from Cloud
      • Import using API & SDK
      • Import using agent
    • Migrations
      • Roboflow to Supervisely
      • Labelbox to Supervisely
      • V7 to Supervisely
      • CVAT to Supervisely
    • Export
  • 📂Data Organization
    • Core concepts
    • MLOps Workflow
    • Projects
      • Datasets
      • Definitions
      • Collections
    • Team Files
    • Disk usage & Cleanup
    • Quality Assurance & Statistics
      • Practical applications of statistics
    • Operations with Data
      • Data Filtration
        • How to use advanced filters
      • Pipelines
      • Augmentations
      • Splitting data
      • Converting data
        • Convert to COCO
        • Convert to YOLO
        • Convert to Pascal VOC
    • Data Commander
      • Clone Project Meta
  • 📝Labeling
    • Labeling Toolboxes
      • Images
      • Videos 2.0
      • Videos 3.0
      • 3D Point Clouds
      • DICOM
      • Multiview images
      • Fisheye
    • Labeling Tools
      • Navigation & Selection Tools
      • Point Tool
      • Bounding Box (Rectangle) Tool
      • Polyline Tool
      • Polygon Tool
      • Brush Tool
      • Mask Pen Tool
      • Smart Tool
      • Graph (Keypoints) Tool
      • Frame-based tagging
    • Labeling Jobs
      • Labeling Queues
      • Labeling Consensus
      • Labeling Statistics
    • Labeling Performance
    • Labeling with AI-Assistance
  • 🤝Collaboration
    • Admin panel
      • Users management
      • Teams management
      • Server disk usage
      • Server trash bin
      • Server cleanup
      • Server stats and errors
    • Teams & workspaces
    • Members
    • Issues
    • Guides & exams
    • Activity log
    • Sharing
  • 🖥️Agents
    • Installation
      • Linux
      • Windows
      • AMI AWS
      • Kubernetes
    • How agents work
    • Restart and delete agents
    • Status and monitoring
    • Storage and cleanup
    • Integration with Docker
  • 🔮Neural Networks
    • Overview
    • Inference & Deployment
      • Overview
      • Supervisely Serving Apps
      • Deploy & Predict with Supervisely SDK
      • Using trained models outside of Supervisely
    • Model Evaluation Benchmark
      • Object Detection
      • Instance Segmentation
      • Semantic Segmentation
      • Custom Benchmark Integration
    • Custom Model Integration
      • Overview
      • Custom Inference
      • Custom Training
    • Legacy
      • Starting with Neural Networks
      • Train custom Neural Networks
      • Run pre-trained models
  • 👔Enterprise Edition
    • Get Supervisely
      • Installation
      • Post-installation
      • Upgrade
      • License Update
    • Kubernetes
      • Overview
      • Installation
      • Connect cluster
    • Advanced Tuning
      • HTTPS
      • Remote Storage
      • Single Sign-On (SSO)
      • CDN
      • Notifications
      • Moving Instance
      • Generating Troubleshoot Archive
      • Storage Cleanup
      • Private Apps
      • Data Folder
      • Firewall
      • HTTP Proxy
      • Offline usage
      • Multi-disk usage
      • Managed Postgres
      • Scalability Tuning
  • 🔧Customization and Integration
    • Supervisely .JSON Format
      • Project Structure
      • Project Meta: Classes, Tags, Settings
      • Tags
      • Objects
      • Single-Image Annotation
      • Single-Video Annotation
      • Point Cloud Episodes
      • Volumes Annotation
    • Developer Portal
    • SDK
    • API
  • 💡Resources
    • Changelog
    • GitHub
    • Blog
    • Ecosystem
Powered by GitBook
On this page
  • Overview
  • Format description
  • Input files structure
  • nuScenes' Annotation format
  • Useful links

Was this helpful?

  1. Import and Export
  2. Import
  3. Supported annotation formats
  4. Pointcloud Episodes

nuScenes

PreviousLyftNextKITTI 360

Last updated 4 months ago

Was this helpful?

Overview

Easily import your pointclouds with annotations in the nuScenes format.

The original nuScenes dataset is a comprehensive, large-scale public dataset designed for autonomous driving research, developed by Motional. It includes 1,000 diverse 20-second driving scenes from Boston and Singapore, featuring dense traffic and challenging conditions. With 1.4M camera images, 390k LIDAR sweeps, and 1.4M RADAR sweeps, it provides extensive multimodal sensor data, including annotations for 23 object classes with 3D bounding boxes at 2Hz and object-level attributes like visibility and activity.

When uploading as Pointcloud Episodes, pointclouds will be grouped by scene and objects will be tracked across a scene.

Format description

Supported point cloud format: .bin With annotations: yes Supported annotation format: .json Data structure: Information is provided below.

Input files structure

Example data: .

Both directory and archive are supported.

Format directory structure:

📦pointcloud_project (folder or .tar/.zip archive)
├──📂dataset1
│ ├──📂maps
│ │ ├──🏞️36092f0b03a857c6a3403e25b4b7aab3.png
│ │ └──🏞️...
│ ├──📂samples
│ │ ├──📂CAM_BACK
│ │ │ ├──🏞️n008-2018-08-01-15-16-36-0400__CAM_BACK__1533151603537558.jpg
│ │ │ └──🏞️...
│ │ ├──📂CAM_BACK_LEFT
│ │ │ ├──🏞️n008-2018-08-01-15-16-36-0400__CAM_BACK_LEFT__1533151603547405.jpg
│ │ │ └──🏞️...
│ │ ├──📂CAM_BACK_RIGHT
│ │ │ ├──🏞️n008-2018-08-01-15-16-36-0400__CAM_BACK_RIGHT__1533151605028113.jpg
│ │ │ └──🏞️...
│ │ ├──📂CAM_FRONT
│ │ │ ├──🏞️n008-2018-08-01-15-16-36-0400__CAM_FRONT__1533151615912404.jpg
│ │ │ └──🏞️...
│ │ ├──📂CAM_FRONT_LEFT
│ │ │ ├──🏞️n008-2018-08-01-15-16-36-0400__CAM_FRONT_LEFT__1533151614904799.jpg
│ │ │ └──🏞️...
│ │ ├──📂CAM_FRONT_RIGHT
│ │ │ ├──🏞️n008-2018-08-01-15-16-36-0400__CAM_FRONT_RIGHT__1533151611420482.jpg
│ │ │ └──🏞️...
│ │ ├──📂LIDAR_TOP
│ │ │ ├──📄n008-2018-08-01-15-16-36-0400__LIDAR_TOP__1533151611896734.pcd.bin
│ │ │ └──📄...
│ │ ├──📂RADAR_BACK_LEFT
│ │ │ ├──📄n008-2018-08-01-15-16-36-0400__RADAR_BACK_LEFT__1533151610450182.pcd
│ │ │ └──📄...
│ │ ├──📂RADAR_BACK_RIGHT
│ │ │ ├──📄n008-2018-08-01-15-16-36-0400__RADAR_BACK_RIGHT__1533151607538446.pcd
│ │ │ └──📄...
│ │ ├──📂RADAR_FRONT
│ │ │ ├──📄n008-2018-08-01-15-16-36-0400__RADAR_FRONT__1533151610426071.pcd
│ │ │ └──📄...
│ │ ├──📂RADAR_FRONT_LEFT
│ │ │ ├──📄n008-2018-08-01-15-16-36-0400__RADAR_FRONT_LEFT__1533151616430028.pcd
│ │ │ └──📄...
│ │ ├──📂RADAR_FRONT_RIGHT
│ │ │ ├──📄n008-2018-08-01-15-16-36-0400__RADAR_FRONT_RIGHT__1533151609574740.pcd
│ │ └─└──📄...
│ ├──📂sweeps
│ │ ├──📂CAM_BACK
│ │ │ └──🏞️...
│ │ ├──📂CAM_BACK_LEFT
│ │ │ └──🏞️...
│ │ ├──📂CAM_BACK_RIGHT
│ │ │ └──🏞️...
│ │ ├──📂CAM_FRONT
│ │ │ └──🏞️...
│ │ ├──📂CAM_FRONT_LEFT
│ │ │ └──🏞️...
│ │ ├──📂CAM_FRONT_RIGHT
│ │ │ └──🏞️...
│ │ ├──📂LIDAR_TOP
│ │ │ └──📄...
│ │ ├──📂RADAR_BACK_LEFT
│ │ │ └──📄...
│ │ ├──📂RADAR_BACK_RIGHT
│ │ │ └──📄...
│ │ ├──📂RADAR_FRONT
│ │ │ └──📄...
│ │ ├──📂RADAR_FRONT_LEFT
│ │ │ └──📄...
│ │ ├──📂RADAR_FRONT_RIGHT
│ │ └─└──📄...
│ ├──📂v1.0-mini
│ │ ├──📄attribute.json
│ │ ├──📄calibrated_sensor.json
│ │ ├──📄category.json
│ │ ├──📄ego_pose.json
│ │ ├──📄instance.json
│ │ ├──📄log.json
│ │ ├──📄map.json
│ │ ├──📄sample.json
│ │ ├──📄sample_annotation.json
│ │ ├──📄sample_data.json
│ │ ├──📄scene.json
│ │ ├──📄sensor.json
└── └──📄visibility.json

Every .bin file in a sequence has to be stored inside a LIDAR_TOP folder of 'sample' and 'sweeps' folders of dataset.

Key
Value

x

The x coordinate of the point.

y

The y coordinate of the point.

z

The z coordinate of the point.

i

Intensity of the return signal.

r

Ring index.

nuScenes' Annotation format

The nuScenes annotations are token-based. The dataset is composed of dozens of scenes, each containing a certain amount of samples (pointclouds). Samples are linked to annotation data through the use of unique identifiers. These tokens ensure that each point cloud frame is accurately associated with its corresponding annotations, camera data, log information, maps, etc. maintaining the integrity and structure of the dataset.

attribute.json

The attribute.json file contains metadata about the attributes of objects in the dataset. An attribute is a property of an instance that can change while the category remains the same. Example: a vehicle being parked/stopped/moving, and whether or not a bicycle has a rider.

{
   "token":                   <str> -- Unique record identifier.
   "name":                    <str> -- Attribute name.
   "description":             <str> -- Attribute description.
}

calibrated_sensor.json

The calibrated_sensor.json file contains calibration data for each sensor used in the dataset. Definition of a particular sensor (lidar/radar/camera) as calibrated on a particular vehicle. All extrinsic parameters are given with respect to the ego vehicle body frame. All camera images come undistorted and rectified.

{
   "token":                   <str> -- Unique record identifier.
   "sensor_token":            <str> -- Foreign key pointing to the sensor type.
   "translation":             <float> [3] -- Coordinate system origin in meters: x, y, z.
   "rotation":                <float> [4] -- Coordinate system orientation as quaternion: w, x, y, z.
   "camera_intrinsic":        <float> [3, 3] -- Intrinsic camera calibration. Empty for sensors that are not cameras.
}

category.json

The category.json file contains metadata about the categories of objects in the dataset. Taxonomy of object categories (e.g. vehicle, human). Subcategories are delineated by a period (e.g. human.pedestrian.adult).

{
   "token":                   <str> -- Unique record identifier.
   "name":                    <str> -- Category name. Subcategories indicated by period.
   "description":             <str> -- Category description.
   "index":                   <int> -- The index of the label used for efficiency reasons in the .bin label files of nuScenes-lidarseg. This field did not exist previously.
}

ego_pose.json

Ego vehicle pose at a particular timestamp. Given with respect to global coordinate system of the log's map. The ego_pose is the output of a lidar map-based localization algorithm described in our paper. The localization is 2-dimensional in the x-y plane.

{
   "token":                   <str> -- Unique record identifier.
   "translation":             <float> [3] -- Coordinate system origin in meters: x, y, z. Note that z is always 0.
   "rotation":                <float> [4] -- Coordinate system orientation as quaternion: w, x, y, z.
   "timestamp":               <int> -- Unix time stamp.
}

instance.json

The instance.json file contains metadata about each instance in the dataset. An object instance, e.g. particular vehicle. This table is an enumeration of all object instances we observed. Note that instances are not tracked across scenes, only inside a given scene.

{
   "token":                   <str> -- Unique record identifier.
   "category_token":          <str> -- Foreign key pointing to the object category.
   "nbr_annotations":         <int> -- Number of annotations of this instance.
   "first_annotation_token":  <str> -- Foreign key. Points to the first annotation of this instance.
   "last_annotation_token":   <str> -- Foreign key. Points to the last annotation of this instance.
}

log.json

The log.json file contains information about the log from which the data was extracted.

{
   "token":                   <str> -- Unique record identifier.
   "logfile":                 <str> -- Log file name.
   "vehicle":                 <str> -- Vehicle name.
   "date_captured":           <str> -- Date (YYYY-MM-DD).
   "location":                <str> -- Area where log was captured, e.g. singapore-onenorth.
}

map.json

The map.json file contains metadata about the map data that is stored as binary semantic masks from a top-down view.

{
   "token":                   <str> -- Unique record identifier.
   "log_tokens":              <str> [n] -- Foreign keys.
   "category":                <str> -- Map category, currently only semantic_prior for drivable surface and sidewalk.
   "filename":                <str> -- Relative path to the file with the map mask.
}

sample.json

The sample.json file contains metadata about each sample in the dataset. A sample is an annotated keyframe at 2 Hz. The data is collected at (approximately) the same timestamp as part of a single LIDAR sweep.

{
   "token":                   <str> -- Unique record identifier.
   "timestamp":               <int> -- Unix time stamp.
   "scene_token":             <str> -- Foreign key pointing to the scene.
   "next":                    <str> -- Foreign key. Sample that follows this in time. Empty if end of scene.
   "prev":                    <str> -- Foreign key. Sample that precedes this in time. Empty if start of scene.
}

sample_annotation.json

The sample_annotation.json file contains detailed information about each annotation in the dataset. A bounding box defining the position of an object seen in a sample. All location data is given with respect to the global coordinate system.

{
   "token":                   <str> -- Unique record identifier.
   "sample_token":            <str> -- Foreign key. NOTE: this points to a sample NOT a sample_data since annotations are done on the sample level taking all relevant sample_data into account.
   "instance_token":          <str> -- Foreign key. Which object instance is this annotating. An instance can have multiple annotations over time.
   "attribute_tokens":        <str> [n] -- Foreign keys. List of attributes for this annotation. Attributes can change over time, so they belong here, not in the instance table.
   "visibility_token":        <str> -- Foreign key. Visibility may also change over time. If no visibility is annotated, the token is an empty string.
   "translation":             <float> [3] -- Bounding box location in meters as center_x, center_y, center_z.
   "size":                    <float> [3] -- Bounding box size in meters as width, length, height.
   "rotation":                <float> [4] -- Bounding box orientation as quaternion: w, x, y, z.
   "num_lidar_pts":           <int> -- Number of lidar points in this box. Points are counted during the lidar sweep identified with this sample.
   "num_radar_pts":           <int> -- Number of radar points in this box. Points are counted during the radar sweep identified with this sample. This number is summed across all radar sensors without any invalid point filtering.
   "next":                    <str> -- Foreign key. Sample annotation from the same object instance that follows this in time. Empty if this is the last annotation for this object.
   "prev":                    <str> -- Foreign key. Sample annotation from the same object instance that precedes this in time. Empty if this is the first annotation for this object.
}

sample_data.json

The sample_data.json file contains metadata about each data sample in the dataset. A sensor data e.g. image, point cloud or radar return. For sample_data with is_key_frame=True, the time-stamps should be very close to the sample it points to. For non key-frames the sample_data points to the sample that follows closest in time.

{
   "token":                   <str> -- Unique record identifier.
   "sample_token":            <str> -- Foreign key. Sample to which this sample_data is associated.
   "ego_pose_token":          <str> -- Foreign key.
   "calibrated_sensor_token": <str> -- Foreign key.
   "filename":                <str> -- Relative path to data-blob on disk.
   "fileformat":              <str> -- Data file format.
   "width":                   <int> -- If the sample data is an image, this is the image width in pixels.
   "height":                  <int> -- If the sample data is an image, this is the image height in pixels.
   "timestamp":               <int> -- Unix time stamp.
   "is_key_frame":            <bool> -- True if sample_data is part of key_frame, else False.
   "next":                    <str> -- Foreign key. Sample data from the same sensor that follows this in time. Empty if end of scene.
   "prev":                    <str> -- Foreign key. Sample data from the same sensor that precedes this in time. Empty if start of scene.
}

scene.json

The scene.json file contains metadata about each scene in the dataset. A scene is a 20s long sequence of consecutive frames extracted from a log. Multiple scenes can come from the same log. Note that object identities (instance tokens) are not preserved across scenes.

{
   "token":                   <str> -- Unique record identifier.
   "name":                    <str> -- Short string identifier.
   "description":             <str> -- Longer description of the scene.
   "log_token":               <str> -- Foreign key. Points to log from where the data was extracted.
   "nbr_samples":             <int> -- Number of samples in this scene.
   "first_sample_token":      <str> -- Foreign key. Points to the first sample in scene.
   "last_sample_token":       <str> -- Foreign key. Points to the last sample in scene.
}

sensor.json

The sensor.json file specifies sensor types.

{
   "token":                   <str> -- Unique record identifier.
   "channel":                 <str> -- Sensor channel name.
   "modality":                <str> {camera, lidar, radar} -- Sensor modality. Supports category(ies) in brackets.
}

visibility.json

The visibility of an instance is the fraction of annotation visible in all 6 images. Binned into 4 bins 0-40%, 40-60%, 60-80% and 80-100%.

{
   "token":                   <str> -- Unique record identifier.
   "level":                   <str> -- Visibility level.
   "description":             <str> -- Description of visibility level.
}

Useful links

The nuScenes format description can be found

🔁
download ⬇️
here
nuScenes homepage
MMDetection3D Documentation on nuScenes dataset
nuScenes devkit GitHub page