Videos

Label hours-long videos without cutting them into images. In your browser, with multi-track timeline, built-in object tracking and segments tagging tools.

Advanced Object Tracking: The video annotation toolbox supports both Single Object Tracking (SOT) and Multiple Object Tracking (MOT), enabling efficient tracking of objects across frames. Users can integrate their own tracking algorithms without needing specialized knowledge.

AI-Assisted Labeling: The toolbox offers interactive semantic segmentation tools powered by class-agnostic neural networks, which can be trained on custom datasets. These neural networks are both trainable and customizable, allowing users to adapt them for various data types and use cases.

Video Tagging: With segment tagging, users can annotate specific video sections to capture and categorize different events or activities. This simplifies the video annotation workflow and ensures that important moments are accurately labeled.

User-Friendly Interface: The interface is highly customizable, allowing users to tailor the annotation environment to their needs.

No Conversion Required: Users can start annotating videos immediately after uploading or by connecting their cloud storage. This eliminates the need for converting videos into image sequences, speeding up the workflow.

Automatic Management of Tracking IDs: The toolbox automatically handles tracking IDs across frames, reducing manual intervention. This simplifies the process of tracking multiple objects and enhances workflow efficiency.

Consistent Frame Handling: By maintaining consistency across all frames, the toolbox ensures that object tracking and tagging remain accurate over long video segments. This is critical for training neural networks, ensuring they learn from the same data that human annotators have reviewed.

Overview

Home button — returns user to the main menu (Projects page)
Basic interface elements — basic settings, such as history of operations, theme, a hotkeys map and more useful features.
Main scene & labeling scene settings & playback — annotation area for current video and its labels.
Timeline and track controls — video timeline and controls for managing tracks and frames.
Definitions panel — make it easy to create and manage classes and tags.
Instruments panel — annotation tools used to create annotations.
Objects panel — list of figures on the current video with additional information like classes and tags.
Videos/Apps/Settings panel — list of videos in your dataset, list of additional apps you can embed into the labeling toolbox, visualization and other settings.

Basic interface elements

The top toolbar contains options for personalizing the interface and managing data and its annotations.

Video navigation arrows (next, previous): Allow users to move between videos in the dataset.

Undo and redo buttons: Undo or redo the most recent annotation action.

Select theme (dark or light): Ability to switch between light and dark interface themes, especially useful for those who prefer to work at night.

Hotkeys: A list of hotkeys for quick access to tools.

More options:

Enter fullscreen - this option allows the user to switch the interface to fullscreen mode, maximizing the workspace area. It hides browser toolbars and other elements
Screenshot - the screenshot function enables users to take a snapshot of the current workspace, including the video and any annotations displayed. This can be useful for documentation, sharing progress, or reviewing annotations with team members.
Enter restore mode - enter restore mode provides tools to recover lost or corrupted annotations. When enabled, it offers options to revert changes to a previous state or repair specific parts of the annotation dataset.
Restore default layout - this function resets the interface layout to its default configuration. It is useful when the layout has been modified (e.g., panels moved or resized) and the user wants to return to the original setup.

Main scene & labeling scene settings & playback

This is the central area. It displays the video to be annotated, with various display controls that the user can hide or show in the panel as needed.

Main scene shows the video currently being worked on. Users can interact directly with this area using the annotation tools from the sidebar.

Annotation display settings

Opacity: To modify the transparency of objects, hold and drag the cursor left or right. This allows for a more nuanced view of the objects' layers. Additionally, you can hold the SHIFT key and scroll the mouse wheel to adjust the opacity conveniently from anywhere on the screen.

Border: Enhance the visibility of object boundaries by holding and dragging the cursor left or right. This action changes the width of the objects' borders, allowing for clearer demarcation. Useful when working with huge resolutions or with large number of small objects

Point: Adjust the radius of object points by holding and dragging the cursor left or right.

Default color: Paint objects with their original colors as defined in class settings. This is the default setting and helps maintain consistency and recognition. So the objects of different classes are visually distinguishable.

Randomize color: Randomize object colors to distinguish between objects of the same class. A simple click, followed by SHIFT+H, randomizes the object's colors. Can be used in Instance Segmentation Computer Vision task to highlight visual distinction of all objects of the same class on an image.

Attribute display settings for clearer context

ID: Toggle the visibility of object IDs near the objects on the scene. This is essential for identifying and referring to specific objects.

Bindings: Show or hide bindings near objects to understand how various elements are connected to each other. Objects can be combined into groups.

Tags: Display tags near objects to provide additional context or categorization.

Classes: Enable visibility of the classes assigned to each object, helping in the classification and organization of scene elements.

Author: Display the creator's name near the objects to acknowledge object authorship.

Change Visibility Mode: This option allows users to switch between different visibility modes, optimizing the scene display as per the user's preference. You can choose how to show the attributes:

Always | Users can select full tag display, which means that the information will be visible directly on Objects or Videos in the project.
Show on hover | Tags are only displayed when the cursor is hovered over the annotated object.
Show when selected | The ability to hide Tags until the Object is selected provides a cleaner look and feel to the interface and prevents information overload when working with a project.

Advanced interaction with scene objects

Auto-select: Automatically select objects of the current shape when hovering the cursor over them.

Show object trajectory: Visualize the path an object has taken in the scene over time.

You can customize the way the trajectory looks using the following settings:

Width (px): This setting controls the thickness of the trajectory line in pixels. A higher value makes the line thicker, making the trajectory more visible, while a lower value makes the line thinner.
Draw backward frames: This option controls how many frames in the past are drawn to represent the object's trajectory. Users can adjust the number of frames using a slider, allowing them to visualize the object's past movements over a longer or shorter period of time.
Draw forward frames: This option controls how many frames in the future are drawn to represent the predicted path of the object. Users can adjust the number of frames using a slider, which helps plan and visualize future motion.

Customizing image display settings

Scene display settings: Adjust scene display settings like brightness or contrast to suit different viewing conditions or preferences. For example, you can use them while annotation dark or low-contrast images.

Grid: A grid helps organize the navigation on the images with high resolutions and large number of small objects.

Visibility and image sizing

The ability to hide annotation settings declutters the workspace, focusing attention on the task at hand.

Real-time image resizing adapts to various project needs, ensuring optimal viewing and editing conditions. Just zoom-in or out on the images to see object details and perform precise labeling of object boundaries.

Playback controls

Play/Pause buttons(ENTER): Standard playback controls to play or pause the video.

Play video backwards: Plays the video from the current frame backwards, i.e. from the current frame to the previous frame, and so on to the beginning of the video.

Previous/Next frame (<- | ->): Buttons to jump to the previous or next frame.

Previous/Next 10 frame (ALT + <- | ALT + ->): Buttons to jump to the previous or next 10 frame.

Frame counter and time display: Shows the current frame number, total number of frames, and current time position in the video (e.g., "100 / 384" or "0:04 / 0:16"). For example, 122 / 384 means you are at frame 122 out of 384 total frames.

Track length slider: Allows you to quickly scroll through the video frames by dragging the slider left or right. Annotated segments of the video are displayed by blue color line.

Playback settings

Speed: Controls the playback speed (e.g., x2).

Skip frames: Allows skipping a specified number of frames during navigation (e.g., +/- 10 frames).

Fast decoding mode: Enabling this mode speeds up the video playback process.

Navigation bar settings: Switches between different modes for the navigation bar, such as

Figures: show frames with objects
Tags: show frames with tags
Auto: show frames with figures/tags based current selection

Timeline and track controls

It’s easy to get frustrated with thousands of frames and objects labeled!

The tracks panel offers controls for managing tracks, tags, and navigating through video frames. It provides a clear overview of the video's structure, answers questions about what has already been labeled, and simplifies editing tag segments and tracked objects.

Manage tracks

This dropdown list allows you to manage different tracks or objects detected in the video. A track represents a series of frames where an object is detected and labeled. By selecting options from this menu you can choose which track will be visible on the timeline.

Figures: Filters and displays only figures detected in the video frames.

Video Tags: Displays all tags assigned with the entire video.

Objects Tags: Shows tags specific to objects on the frames.

Current Object Tags: Focuses on the tags currently assigned with the selected object.

Current Tag Tags: Filters and displays the tags that are actively being used in the current tagging session.

Select one or more options to filter and view specific tags in the frame or across the video.

Timeline

It's a linear scale that visually displays the sequence of video frames and associated annotations. It allows users to manage annotation processes, object tracking, and other labeling tasks. The timeline contains various tracks which display data related to objects and tags.

Time scale: A horizontal ruler that shows all video frames. Each frame is numbered, making it easy to locate specific points in time. The frame-level timeline provides more detailed control and visualization of the video on a frame-by-frame basis.

The manually labeled frames are shown in gray, the frames that were automatically tracked are striped, the current frame is in a blue frame

Frame segments: Labeled objects and tags are shown as segments on the timeline. For example, tracks for objects or tags show the start and end points of their presence in the video.

Track information display: Displays the current segment of the video you are managing (e.g., "SEGMENT 1 / 2") and allows you to navigate between segments.

Play segment: SHIFT + ENTER
Previous segment: SHIFT + <-
Next segment: SHIFT + ->

Track

Track button (SHIFT + T): Automatically create new figures based on the selected one for the next video frames.

The tracking process is started manually and performed automatically. The system analyzes the selected object or area in the frame and starts tracking its motion and changes as the video plays.

Tracking progress bar: A progress bar appears in the interface that shows how many percent of tracking has already been done.

The tracking percentage will be updated in real time as the operation progresses, and the bar will gradually fill up to reflect the progress.

When the progress bar reaches 100%, it means that the tracking process has been successfully completed. The object has been tracked on all selected frames.

After tracking is complete, the user can track the object if no errors are detected, or edit some frames and re-track them.

Stop tracking (SHIFT+S): The system stops tracking the selected object in the frame. Already created figures will remain.

Track settings

Method: Choose the object tracking algorithm. For example, "MixFormer Object Tracking" application is used to track the motion of selected objects.

The predefined Clone command creates a duplicate (or clone) of the selected shape, object, or annotation. This clone will have the same parameters as the original, including size, position, shape, and any associated data (such as tags or metadata).

Working with clones:

A clone can be created in the same frame as the original shape, or moved to another frame to keep identical settings in different parts of the video. Once a clone is created, you can modify it independently of the original, or use it as a template to create other objects with the same settings.

Direction: Defines the direction of tracking, such as "Forward"/"Backward".

Overwrite figures: Auto-generated objects will be override.

Frames: Defines the number of frames to be processed during tracking (e.g., +/- 10 frames).

Editing Tools

Scissors (COMMAND + X): Cuts figures at the current frame.

Copy (COMMAND + C): Duplicates the selected figures.

Paste (COMMAND + V): Pastes duplicate figures.

Trash (SHIFT + D): Removes the selected figures.

Frame selector (SHIFT + SPACE) : Allows you to manually select multiple frames for batch tracking, tagging and removal.

Definitions panel

The Definitions panel provides a simple interface for creating and managing classes and tags in a project. It helps users organize and control annotations.

Instead of choosing a tool first, you can now click on the desired class from the Definitions Panel. The associated tool will be automatically selected, allowing you to start labeling immediately. To start a new label, simply click on any class (including the currently selected one) in the Definitions Panel.

If you want to change the class of the selected object, you can click the small icon on the right, which appears only if the new class shape matches the currently selected object. Tags are also present on the same panel. If no object is selected, image tags are shown. You can check the desired tag or hover the cursor and start typing a tag value or select it from a dropdown, which will automatically assign it.

Improved Search: To find classes or tags more easily in a long list, click the magnifying glass icon in the top right corner of the panel. Type your query and select the desired class or tag to continue your workflow.

Classes

Class may be assigned only to an object and represent a clear category to which the object in the image belongs. For example, a classification of vehicles in an image might include the classes "car", "truck", and "bus".

Definitions panel displays the list of annotation classes (e.g., "person", "road sign", "vehicle"). Each class has a unique name, color and shape to identify between different types of objects.

Add new class definitions using the add new class definition option.
Organize classes in the definitions list to improve navigation.

Video property tags

This section is for tags that describe general properties of the video as a whole.
No tags here indicates that no tags have been added to this section.
These tags can include attributes like the video’s location, weather conditions, or general scene characteristics that are constant throughout the video.

Video frame tags

This section is for tags that apply to specific frames in the video.
Tags here help to identify or categorize specific moments or frames within the video.
X On Screen indicates tags currently visible on any frame.
X Total indicates tags created for any frame in the entire video..
Video frame tags can be used for frame-specific annotations, like marking a moment when a skier jumps or a snowboarder falls.

Object Property Tags

This section is for tags that describe general properties of an object in the video as a whole.
No tags here indicates that no tags have been added to this section.
These tags can include attributes such as the object's type, color, size, or other characteristics that remain common across the video.

Example: snowboarder

Object type: human; clothing: winter suit, helmet; snowboard color: black: equipment: snowboard

Object Frame Tags

This section is for tags that apply to specific frames in the video and describe changes or actions related to an object.
Tags here help identify or categorize specific actions or occurrences for an object at specific frames.
X On Screen indicates tags that are currently visible on any frame for the object.
X Total indicates tags created for the object across all frames in the video.
Object frame tags can be used for frame-specific annotations, such as marking when an object moves, changes state, or interacts with another object (for example, when a person picks up an object or a vehicle starts moving).

Example: snowboarder performing a trick

Frame 00:15: snowboarder begins a jump off the ramp; frame 00:17: performs a 360-degree spin; frame 00:19: lands on the slope

Tags available

This section lists all the available tags that can be used in the whole video.
Search for tags... input field allows the user to search for a specific tag if the list is extensive.
In the example provided, there is 1 tag available: status-segment...: A tag that appears to be related to the status or segmentation of objects or frames. It is indicated by a blue icon.
Link icon (🔗): Allows the user to link the tag to specific frames or objects.
Mark Frames button: Probably used to mark or assign this tag to a range of frames in the video.

Instruments panel

Pan & Move Scene Tool: Quickly navigate around the image without modifying annotations.

Select Figure: Select and modify existing annotations; essential for refining objects.

Drag Figure: Reposition annotations without altering their size or shape.

Point Tool: Label specific points or small objects precisely.

Bounding Box: Best for object detection tasks.

Polyline Tool: Annotate linear objects or edges with multiple connected line segments.

Polygon Tool: Ideal for irregular and complex shapes.

Brush and Eraser Tool: Flexible for both polygonal and free-form masks.

Mask Pen Tool: Great for segmenting diverse objects with varying shapes.

Smart Tool: Efficient for quick, AI-assisted segmentation.

Graph (Keypoins) Tool: For pose-estimation tasks.

Objects panel

The Objects panel displays key properties and parameters of objects that can be tracked on the timeline. It is a dynamic space for showcasing and managing object tags, attributes, and metadata. Some of the features it offers include:

Clone Objects - Easily replicate selected objects to the next frame with a simple right arrow key press ( ->).
Filter and Manage - Quickly filter objects, remove all from the video, or toggle their visibility according to your needs. For example you can hide all objects except of the specific class.
Advanced Interactions - Select, delete, hide, merge objects, or adjust their layering. Additionally, you can modify metadata and assign or manage tags right from this window, enhancing the object's data with minimal effort.

Videos panel

The videos panel provides a comprehensive view of all the videos within a selected dataset. Key functionalities include:

Tag Management - Clone tags from the previous video, assign new ones, or modify existing tags to maintain consistency and organization.
Filter and Manage - View and edit metadata details or filter through videos to find exactly what you need.
Video Operations - Delete videos, download them individually, or download annotations for external use.

Apps panel: expand your capabilities

In an ever-evolving ML landscape, the apps panel serves as a portal to a wide range of applications from the Computer Vision Ecosystem, enhancing the functionality of your workspace. This window allows you to run and open the public or private apps and extend the Labeling Toolbox with custom UI and functionality.

Settings panel

The settings panel is the control center for personalizing the interface. It houses various options allowing users to tweak the interface to match their workflow, preferences, and project requirements.

How to use tags in Video Annotation Tool

Step 1. Use the Definitions panel to search for or create tags:

Below the classes are sections for object property tags and object frame tags. These sections are used to tag entire objects (such as a skier or snowboarder) or specific frames for specific attributes or actions.
The tags available section displays all tags that can be used in the video annotation. You can create new tags, search for tags using the search bar, or select directly from the available list.
Video Property Tags: This section, found in the right panel, allows you to assign tags that describe general properties of the entire video, such as "on the move." Use the + button next to this section to add tags that describe overarching elements of the video, like the weather, location, or ongoing action throughout the entire clip.
Video Frame Tags: This section, also in the right panel, helps in tagging specific moments or events within the video. If an event (like a skier jumping) occurs at a particular frame, you can tag that frame using this section.

Step 2. Applying tags to Frames:

To apply a tag to a specific frame, click on the MARK FRAMES button next to the tag in the tags available section. This action allows you to select frames to which you want to apply the selected tag.

Navigation and Tagging Controls:

The bottom of the interface contains playback controls and a timeline slider.
Use these controls to navigate through the video frames (e.g., go frame by frame, play, pause) and select the frames you want to tag.
The "TAG," "UNTAG," and "CHANGE VALUE" buttons below the timeline slider allow you to apply, remove, or modify tags for the selected frames or objects.

Filter and Manage Tracks:

The Manage tracks dropdown menu lets you control which tags are visible and which types of tags you want to manage. You can filter by different tag categories, such as "Figures," "Video Tags," "Object Tags," etc., to simplify your view and focus on specific elements.
The Filter icon next to each section helps narrow down what is displayed, making it easier to manage large amounts of tagged data.

Practical Example: Tagging a skier's motion

Select the Skier Class: Ensure that the "skier" class is selected in the Definitions panel.
Choose a Tag: Decide which tag to apply (e.g., "on the move").
Mark Frames: Click on the MARK FRAMES button next to the tag "on the move."
Select Frames: Use the video timeline slider to navigate to the frames where the skier is moving and apply the tag.
Confirm Tagging: The selected frames will now be tagged with "on the move," and you can see the tag applied in the Video Frame Tags or Object Frame Tags sections.

Speed up labeling with AI-assistance

Video labeling toolbox has countless features, but most importantly, it provides integrations with the best AI models that dramatically improve your labeling performance. For example: