Multiview videos

Annotate multiple synchronized videos from different camera angles as a unified scene with shared objects and synchronized playback.

Overview

Multiview mode allows you to work with multiple videos of the same scene captured from different cameras or angles simultaneously. All videos within one dataset form a synchronized multiview group, enabling efficient annotation of complex multi-camera setups.

This is particularly useful for:

Autonomous driving — multiple cameras mounted on a vehicle
Surveillance — security cameras covering the same area from different angles
Sports analytics — tracking athletes from multiple viewpoints
3D reconstruction — multi-camera setups for depth estimation

Key Concepts

Unified Objects

If you have a common object appearing across different videos (e.g., a car visible from front, left, and right cameras), you can annotate it as a single Supervisely object with a shared ID.

The object maintains its identity across all videos in the multiview group
When you export and re-import the project, the object is recreated as a unified entity

Synchronized Playback

All videos in a multiview group can be played back synchronously:

Navigate through frames simultaneously across all views
Configure frame offsets if videos have different starting points
Control the order of videos in the labeling interface using metadata files with videoStreamIndex (see "How to Create Multiview Project" section below)
Easily compare and analyze object behavior from multiple angles in real-time

How to Create Multiview Project

Via UI (Drag & Drop)

Prepare your data according to the required structure (see below):
- Create a folder for each multiview group (dataset)
- Place all camera videos inside the corresponding folder

📂 .
└── 📂 scene_01
    ├── 🎥 camera_left.mp4
    ├── 🎥 camera_top.mp4
    ├── 🎥 camera_front.mp4
    └── 🎥 camera_right.mp4

Go to your workspace and start from creating a new project (+ New ⇨ New Project Wizard)
Select Videos ⇨ Multi-view labeling interface and proceed
Press Import to open the import app wizard
Drag and drop your prepared folder or archive
The import app will automatically group videos by datasets

📥 Download example data

For detailed import format specification, see Multiview Import Format.

Via Python SDK

import supervisely as sly

# Initialize API
api = sly.Api.from_env()

# Create a new project with multiview settings
project = api.project.create(
    workspace_id=WORKSPACE_ID,
    name="My Multiview Project",
    type=sly.ProjectType.VIDEOS,
    change_name_if_conflict=True
)

# Update project settings to enable multiview
api.project.set_multiview_settings(project.id)

# Create a dataset (each dataset = one multiview group)
dataset = api.dataset.create(project.id, "scene_01")

# Upload videos to the dataset
video_paths = [
    "/path/to/camera_front.mp4",
    "/path/to/camera_left.mp4",
    "/path/to/camera_right.mp4"
]

for video_path in video_paths:
    api.video.upload_path(
        dataset_id=dataset.id,
        name=sly.fs.get_file_name_with_ext(video_path),
        path=video_path
    )

Open the project in the Supervisely UI to start annotating in multiview mode.

Optional: Specify Video Order

To control the order of videos in the multiview labeling interface, you can create a metadata JSON file for each video with the suffix .meta.json. This file should include the videoStreamIndex to define its position. For example, for a video named camera_front.mp4, create a file named camera_front.mp4.meta.json with the following content:

{
  "videoStreamIndex": 0
}

📂 project_name
    ┣ 📂 dataset_01
    ┃  ┣ 🎥 camera_front.mp4
    ┃  ┣ 🎥 camera_front.mp4.meta.json  # ⬅︎ meta file with videoStreamIndex
    ┃  ┣ 🎥 camera_left.mp4
    ┃  ┣ 🎥 camera_left.mp4.meta.json  # ⬅︎ meta file with videoStreamIndex
    ┃  ┣ 🎥 camera_right.mp4
    ┃  ┗ 🎥 camera_right.mp4.meta.json  # ⬅︎ meta file with videoStreamIndex
    ┗ 📂 dataset_03
       ┣ 🎥 camera_front.mp4
       ┣ 🎥 camera_left.mp4
       ┗ 🎥 camera_right.mp4

Labeling in Multiview Mode

Interface Overview

The multiview labeling interface includes the following key elements:

Video panels & Synchronized timeline — each video is displayed in its own panel with a shared timeline for synchronized navigation:

Objects & Tags panel — shows objects across all videos and video-specific tags:

Multiview settings — configure frame offsets to align videos temporally to ensure synchronized playback. You can specify number of frames or time in milliseconds for each video:

Annotating Objects

Note: When annotating in multiview mode, you can create unified objects across videos, but tags remain video-specific.

Create an object on any video using annotation tools (rectangle, polygon, etc.)
The same object can be annotated on other videos in the group
Objects with the same ID are linked across all videos

Video-specific Tags

Unlike objects, tags apply only within a specific video:

When you tag a figure, frame, or video, that tag is associated only with that particular video
Tags are displayed only on the video where they were created
This allows for view-specific annotations (e.g., "occluded" tag on one camera angle)

Auto-tracking

Multiview mode supports automatic object tracking:

Annotate the first frame on one or more videos
Select the object you want to track
Click Track (or use Shift + T shortcut)
Configure tracking settings:
- Select tracking model
- Choose direction (forward/backward)
- Set number of frames
The tracker will follow the object across frames

Tracking is applied per-video. You may need to track the same object separately on each camera view.

Export

Use the Export Videos in Supervisely Format app to export your multiview project:

All videos are exported with their annotations in Supervisely format
Object relationships across videos are preserved via shared object keys
Metadata files contain videoStreamIndex for maintaining video order
Project meta include settings for multiview configuration

The exported structure can be re-imported to recreate the exact same multiview setup.

Use Cases

Use Case

Description

Autonomous Driving

Annotate objects visible from front, rear, and side cameras as unified entities

Surveillance

Track people or vehicles across multiple security cameras

Sports Analytics

Follow athletes from different camera angles for comprehensive analysis

Retail Analytics

Monitor customer behavior from multiple store cameras

3D Reconstruction

Annotate corresponding points across stereo or multi-camera setups

Useful Links

PreviousMultiview images NextFisheye

Last updated 1 day ago