Videos 2.0
Label hours-long videos without cutting them into images. In your browser, with multi-track timeline, built-in object tracking and segments tagging tools.
Last updated
Label hours-long videos without cutting them into images. In your browser, with multi-track timeline, built-in object tracking and segments tagging tools.
Last updated
Advanced Object Tracking: The video annotation toolbox supports both Single Object Tracking (SOT) and Multiple Object Tracking (MOT), enabling efficient tracking of objects across frames. Users can integrate their own tracking algorithms without needing specialized knowledge.
AI-Assisted Labeling: The toolbox offers interactive semantic segmentation tools powered by class-agnostic neural networks, which can be trained on custom datasets. These neural networks are both trainable and customizable, allowing users to adapt them for various data types and use cases.
Video Tagging: With segment tagging, users can annotate specific video sections to capture and categorize different events or activities. This simplifies the video annotation workflow and ensures that important moments are accurately labeled.
User-Friendly Interface: The interface is highly customizable, allowing users to tailor the annotation environment to their needs.
No Conversion Required: Users can start annotating videos immediately after uploading or by connecting their cloud storage. This eliminates the need for converting videos into image sequences, speeding up the workflow.
Automatic Management of Tracking IDs: The toolbox automatically handles tracking IDs across frames, reducing manual intervention. This simplifies the process of tracking multiple objects and enhances workflow efficiency.
Consistent Frame Handling: By maintaining consistency across all frames, the toolbox ensures that object tracking and tagging remain accurate over long video segments. This is critical for training neural networks, ensuring they learn from the same data that human annotators have reviewed.
Home button — returns user to the main menu (Projects page)
Basic interface elements — basic settings, such as history of operations, theme, a hotkeys map and more useful features.
Main scene & labeling scene settings & playback — annotation area for current video and its labels.
Timeline and track controls — video timeline and controls for managing tracks and frames.
Instruments panel — annotation tools used to create annotations.
Objects panel — list of figures on the current video with additional information like classes and tags.
Videos/Apps/Settings panel — list of videos in your dataset, list of additional apps you can embed into the labeling toolbox, visualization and other settings.
Global Tags apply to the entire video or to an object across its presence in the video.
Go to the Videos panel and open Toggle tags panel.
In the Tags Available section, select or create tags to describe the video property tag (for example, "Traffic Density" or "Weather Condition"). Choose whether the scope is global or global and frame-based when creating a tag.
Click the Attach to video as a property tag button to assign it as a global tag for the entire video.
Click on the labeled object in the video or go to the Objects panel and select one (e.g., "car").
In the Tags Available section, select or create tags to describe the object property tag throughout the video (for example, "Road Position" or "Direction"). Choose whether the scope is global or global and frame-based when creating a tag.
Click the Attach to annotation object as a property tag to link the tag to the object.
Frame Tags allow you to tag specific moments or actions on individual frames.
Assign a frame tag for an object
Click on the labeled object in the video or go to the Objects panel and select one (e.g., "bus").
In the Tags Available section, choose a tag (e.g., "Lane Change") and click Mark Frames to tag the frames where the bus changes lanes.
Use the Timeline slider to navigate through the video and label specific frames with the selected tag. Once you've identified the relevant frames, the tag will be applied accordingly to the chosen frames for the object.
Assign a frame tag for an video
Go to the Videos panel and open Toggle tags panel.
In the Tags Available section, choose a tag (e.g., "Traffic Density") and click Mark Frames to select the frame range to which you want to apply the tag.
Use the Timeline slider to navigate through the video and label specific frames with the selected tag. Once you've identified the relevant frames, the tag will be applied accordingly to the chosen frames for the video.
Tracking allows the system to automatically follow an object across multiple frames.
Select the object in the video.
Click Track (or use the SHIFT + T shortcut) to start tracking. The system will automatically follow the object's movement.
In the tracking settings, you can select the tracking method, direction, and number of frames to process. Once tracking is complete, the object will be tracked across all selected frames.
The top toolbar contains options for personalizing the interface and managing data and its annotations.
Video navigation arrows (next, previous): Allow users to move between videos in the dataset.
Undo and redo buttons: Undo or redo the most recent annotation action.
Select theme (dark or light): Ability to switch between light and dark interface themes, especially useful for those who prefer to work at night.
Hotkeys: A list of hotkeys for quick access to tools.
More options:
Enter fullscreen - this option allows the user to switch the interface to fullscreen mode, maximizing the workspace area. It hides browser toolbars and other elements
Screenshot - the screenshot function enables users to take a snapshot of the current workspace, including the video and any annotations displayed. This can be useful for documentation, sharing progress, or reviewing annotations with team members.
Enter restore mode - enter restore mode provides tools to recover lost or corrupted annotations. When enabled, it offers options to revert changes to a previous state or repair specific parts of the annotation dataset.
Restore default layout - this function resets the interface layout to its default configuration. It is useful when the layout has been modified (e.g., panels moved or resized) and the user wants to return to the original setup.
This is the central area. It displays the video to be annotated, with various display controls that the user can hide or show in the panel as needed.
Main scene shows the video currently being worked on. Users can interact directly with this area using the annotation tools from the sidebar.
Opacity: To modify the transparency of objects, hold and drag the cursor left or right. This allows for a more nuanced view of the objects' layers. Additionally, you can hold the SHIFT key and scroll the mouse wheel to adjust the opacity conveniently from anywhere on the screen.
Border: Enhance the visibility of object boundaries by holding and dragging the cursor left or right. This action changes the width of the objects' borders, allowing for clearer demarcation. Useful when working with huge resolutions or with large number of small objects
Point: Adjust the radius of object points by holding and dragging the cursor left or right.
Default color: Paint objects with their original colors as defined in class settings. This is the default setting and helps maintain consistency and recognition. So the objects of different classes are visually distinguishable.
Randomize color: Randomize object colors to distinguish between objects of the same class. A simple click, followed by SHIFT+H, randomizes the object's colors. Can be used in Instance Segmentation Computer Vision task to highlight visual distinction of all objects of the same class on an image.
ID: Toggle the visibility of object IDs near the objects on the scene. This is essential for identifying and referring to specific objects.
Bindings: Show or hide bindings near objects to understand how various elements are connected to each other. Objects can be combined into groups.
Tags: Display tags near objects to provide additional context or categorization.
Classes: Enable visibility of the classes assigned to each object, helping in the classification and organization of scene elements.
Author: Display the creator's name near the objects to acknowledge object authorship.
Change Visibility Mode: This option allows users to switch between different visibility modes, optimizing the scene display as per the user's preference. You can choose how to show the attributes:
Always | Users can select full tag display, which means that the information will be visible directly on Objects or Videos in the project.
Show on hover | Tags are only displayed when the cursor is hovered over the annotated object.
Show when selected | The ability to hide Tags until the Object is selected provides a cleaner look and feel to the interface and prevents information overload when working with a project.
Auto-select: Automatically select objects of the current shape when hovering the cursor over them.
Show object trajectory: Visualize the path an object has taken in the scene over time.
You can customize the way the trajectory looks using the following settings:
Width (px): This setting controls the thickness of the trajectory line in pixels. A higher value makes the line thicker, making the trajectory more visible, while a lower value makes the line thinner.
Draw backward frames: This option controls how many frames in the past are drawn to represent the object's trajectory. Users can adjust the number of frames using a slider, allowing them to visualize the object's past movements over a longer or shorter period of time.
Draw forward frames: This option controls how many frames in the future are drawn to represent the predicted path of the object. Users can adjust the number of frames using a slider, which helps plan and visualize future motion.
Scene display settings: Adjust scene display settings like brightness or contrast to suit different viewing conditions or preferences. For example, you can use them while annotation dark or low-contrast images.
Grid: A grid helps organize the navigation on the images with high resolutions and large number of small objects.
The ability to hide annotation settings declutters the workspace, focusing attention on the task at hand.
Real-time image resizing adapts to various project needs, ensuring optimal viewing and editing conditions. Just zoom-in or out on the images to see object details and perform precise labeling of object boundaries.
Play/Pause buttons(ENTER): Standard playback controls to play or pause the video.
Play video backwards: Plays the video from the current frame backwards, i.e. from the current frame to the previous frame, and so on to the beginning of the video.
Previous/Next frame (<- | ->): Buttons to jump to the previous or next frame.
Previous/Next 10 frame (ALT + <- | ALT + ->): Buttons to jump to the previous or next 10 frame.
Frame counter and time display: Shows the current frame number, total number of frames, and current time position in the video (e.g., "100 / 384" or "0:04 / 0:16"). For example, 122 / 384 means you are at frame 122 out of 384 total frames.
Track length slider: Allows you to quickly scroll through the video frames by dragging the slider left or right. Annotated segments of the video are displayed by blue color line.
Speed: Controls the playback speed (e.g., x2).
Skip frames: Allows skipping a specified number of frames during navigation (e.g., +/- 10 frames).
Fast decoding mode: Enabling this mode speeds up the video playback process.
Navigation bar settings: Switches between different modes for the navigation bar, such as
Figures: show frames with objects
Tags: show frames with tags
Auto: show frames with figures/tags based current selection
It’s easy to get frustrated with thousands of frames and objects labeled!
The tracks panel offers controls for managing tracks, tags, and navigating through video frames. It provides a clear overview of the video's structure, answers questions about what has already been labeled, and simplifies editing tag segments and tracked objects.
This dropdown list allows you to manage different tracks or objects detected in the video. A track represents a series of frames where an object is detected and labeled. By selecting options from this menu you can choose which track will be visible on the timeline.
Figures: Filters and displays only figures detected in the video frames.
Video Tags: Displays all tags assigned with the entire video.
Objects Tags: Shows tags specific to objects on the frames.
Current Object Tags: Focuses on the tags currently assigned with the selected object.
Current Tag Tags: Filters and displays the tags that are actively being used in the current tagging session.
Select one or more options to filter and view specific tags in the frame or across the video.
It's a linear scale that visually displays the sequence of video frames and associated annotations. It allows users to manage annotation processes, object tracking, and other labeling tasks. The timeline contains various tracks which display data related to objects and tags.
Time scale: A horizontal ruler that shows all video frames. Each frame is numbered, making it easy to locate specific points in time. The frame-level timeline provides more detailed control and visualization of the video on a frame-by-frame basis.
The manually labeled frames are shown in gray, the frames that were automatically tracked are striped, the current frame is in a blue frame
Frame segments: Labeled objects and tags are shown as segments on the timeline. For example, tracks for objects or tags show the start and end points of their presence in the video.
Track information display: Displays the current segment of the video you are managing (e.g., "SEGMENT 1 / 2") and allows you to navigate between segments.
Play segment: SHIFT + ENTER
Previous segment: SHIFT + <-
Next segment: SHIFT + ->
Track button (SHIFT + T): Automatically create new figures based on the selected one for the next video frames.
The tracking process is started manually and performed automatically. The system analyzes the selected object or area in the frame and starts tracking its motion and changes as the video plays.
Tracking progress bar: A progress bar appears in the interface that shows how many percent of tracking has already been done.
The tracking percentage will be updated in real time as the operation progresses, and the bar will gradually fill up to reflect the progress.
When the progress bar reaches 100%, it means that the tracking process has been successfully completed. The object has been tracked on all selected frames.
After tracking is complete, the user can track the object if no errors are detected, or edit some frames and re-track them.
Stop tracking (SHIFT+S): The system stops tracking the selected object in the frame. Already created figures will remain.
Method: Choose the object tracking algorithm. For example, "MixFormer Object Tracking" application is used to track the motion of selected objects.
The predefined Clone command creates a duplicate (or clone) of the selected shape, object, or annotation. This clone will have the same parameters as the original, including size, position, shape, and any associated data (such as tags or metadata).
Working with clones:
A clone can be created in the same frame as the original shape, or moved to another frame to keep identical settings in different parts of the video. Once a clone is created, you can modify it independently of the original, or use it as a template to create other objects with the same settings.
Direction: Defines the direction of tracking, such as "Forward"/"Backward".
Overwrite figures: Auto-generated objects will be override.
Frames: Defines the number of frames to be processed during tracking (e.g., +/- 10 frames).
Scissors (COMMAND + X): Cuts figures at the current frame.
Copy (COMMAND + C): Duplicates the selected figures.
Paste (COMMAND + V): Pastes duplicate figures.
Trash (SHIFT + D): Removes the selected figures.
Frame selector (SHIFT + SPACE) : Allows you to manually select multiple frames for batch tracking, tagging and removal.
Pan & Move Scene Tool: Quickly navigate around the image without modifying annotations.
Select Figure: Select and modify existing annotations; essential for refining objects.
Drag Figure: Reposition annotations without altering their size or shape.
Point Tool: Label specific points or small objects precisely.
Bounding Box: Best for object detection tasks.
Polyline Tool: Annotate linear objects or edges with multiple connected line segments.
Polygon Tool: Ideal for irregular and complex shapes.
Brush and Eraser Tool: Flexible for both polygonal and free-form masks.
Mask Pen Tool: Great for segmenting diverse objects with varying shapes.
Smart Tool: Efficient for quick, AI-assisted segmentation.
Graph (Keypoins) Tool: For pose-estimation tasks.
The Objects panel displays key properties and parameters of objects that can be tracked on the timeline. It is a dynamic space for showcasing and managing object tags, attributes, and metadata. Some of the features it offers include:
Clone Objects - Easily replicate selected objects to the next frame with a simple right arrow key press ( ->).
Filter and Manage - Quickly filter objects, remove all from the video, or toggle their visibility according to your needs. For example you can hide all objects except of the specific class.
Advanced Interactions - Select, delete, hide, merge objects, or adjust their layering. Additionally, you can modify metadata and assign or manage tags right from this window, enhancing the object's data with minimal effort.
The videos panel provides a comprehensive view of all the videos within a selected dataset. Key functionalities include:
Tag Management - Clone tags from the previous video, assign new ones, or modify existing tags to maintain consistency and organization.
Filter and Manage - View and edit metadata details or filter through videos to find exactly what you need.
Video Operations - Delete videos, download them individually, or download annotations for external use.
In an ever-evolving ML landscape, the apps panel serves as a portal to a wide range of applications from the Computer Vision Ecosystem, enhancing the functionality of your workspace. This window allows you to run and open the public or private apps and extend the Labeling Toolbox with custom UI and functionality.
The settings panel is the control center for personalizing the interface. It houses various options allowing users to tweak the interface to match their workflow, preferences, and project requirements.
Video labeling toolbox has countless features, but most importantly, it provides integrations with the best AI models that dramatically improve your labeling performance. For example:
You can check this short video to get an overview on AI capabilities in the video labeling toolbox:
