Labeling with AI-Assistance
Disrupt accustomed approaches and boost both labeling performance and quality with the help of interactive smart tools.
What is Smart Tool?
Smart Tool is a powerful interactive segmentation tool. This powerful tool uses the latest artificial intelligence (AI) technologies to create highly accurate object masks, significantly speeding up the annotation process. With seamless integration of several state-of-the-art neural network models, including Segment Anything 2 (SAM 2), ClickSEG and others, Smart Tool easily adapts to a wide range of use cases and domains.
Availability of SmartTool
Access to Smart Tool features depends on your Supervisely plan. In the Community version you get access to an already running Smart Tool that works with the ClickSEG or Segment Anything 2 (SAM 2) model.
Step-by-Step Usage Guide
Step 1. Open the Labeling Interface
Navigate to your project and open an Image or Video Labeling Tool.
Locate and select the Smart Tool from the toolbar on the left side of the interface.
Step 2. Select the Neural Network Model
Click on the Model ID button in the left toolbar.
Choose a neural network model, such as Segment Anything 2 (SAM 2), or ClickSEG, depending on your project requirements. The selected model will be activated for segmentation tasks.
Step 3. Create a Bounding Box
Identify the object you want to segment in the image.
Click and drag to draw a bounding box around the object.
Ensure that the box includes the object with a 10% padding for better segmentation accuracy.
Step 4. Adjust the Mask with Points
Place positive points (🟢) on areas of the object you want to include in the mask.
Place negative points (🔴) on areas you want to exclude (e.g., background or overlapping objects).
Adjust the mask by adding or moving points until the segmentation aligns with the object’s boundaries.
Press the SPACE key to finalize the segmentation mask. The mask will be saved as a labeled object in the project.
Step 5. Refine the Segmentation (Optional)
Use the Brush, Eraser, or Pen tools for manual adjustments:
Brush: Add to the mask by painting over missing areas.
Eraser: Remove unwanted parts of the mask.
Pen: Draw precise boundaries around the object.
Press and hold the SHIFT key while using the tools to make additional edits.
Switch Models for Improved Results (Optional)
If the current model doesn’t produce satisfactory results, return to the Model ID button.
Select a different model and repeat the segmentation steps.
Experiment with multiple models to achieve the best performance for your specific dataset.
Video Tutorial
Master the Smart Tool with our detailed 5-minute video tutorial.
SmartTool Deployment
To successfully deploy and run SmartTool models on your own server, you must have a GPU-enabled compute agent. This ensures optimal performance and efficiency of the tool.
We also provide detailed guide on how to deploy your own computer agent to make setting up and using SmartTool easier.
Note: Using a GPU Compute Agent is an important requirement for SmartTool models to function correctly.
Working with Models
Pre-trained Models: Supervisely provides access to popular models like ClickSEG, RITM, and Segment Anything, ensuring a seamless user experience.
Custom Models: Tailor models to your unique data directly on the platform, with no coding required. Example use cases include agricultural image segmentation and road crack detection.
Enterprise Customization: Private instance administrators can configure preferred models via an intuitive administration panel, enabling enterprise-level flexibility.
Learn More
In Supervisely, you have the capability to train the Smart Tool specifically for your data directly within the platform, and the best part is, that no coding is required! We're actively experimenting with this feature and are impressed by the results. Feel free to explore other blog posts dedicated to this topic.
Working with Custom Data
Apart from SmartTool, we have access to a variety of applications tailored for different data handling purposes:
SmartTool: Primarily used for image annotation and labeling.
Apply NN to Images Project: Enables the application of neural networks to image projects directly.
NN Image Labeling: Any NN can be integrated into Labeling interface if it has properly implemented serving app (for example: Serve YOLOv5).
Moreover, we provide various training applications:
These tools use training of specific models like RITM, YOLOv8, UNet, and MMDetection, optimizing the image annotation process across diverse data types.
Different modalities
Another substantial thing about neural networks is that it's easy to adapt it to different modalities. That means, that the Smart Tools not only work on images, but on sequential frames, such as videos or multi-slice medial imaging and even 3D point clouds with more than two dimensions!
Please check the those awesome tutorials and guides:
Last updated
