Graph (Keypoints) Tool
Last updated
Last updated
The Graph (Keypoints) Tool is designed for annotating key points on images and videos, providing an accurate way to analyze poses of objects such as humans and animals. This tool is particularly useful for tasks related to pose estimation, including movement tracking, gesture analysis, and behavior understanding.
To better understand how to use the Graph (Keypoints) Tool for pose estimation, watch our video tutorial. For animal pose estimation, you can also check out the video: "ViTPose — How to Use the Best Pose Estimation Model on Animals | Computer Vision Tutorial". These tutorials provide straightforward instructions on using the Keypoints Tool for both human and animal pose estimation.
Follow this step-by-step guide to learn how to use the Graph (Keypoints) Tool effectively for pose annotation:
On the Project page, navigate to the Definitions section.
Click + New Class and enter a name, select Keypoints shape and choose color.
Define the key points and connections:
Add key points that represent the object's pose (e.g., head, torso, limbs for a human skeleton).
Use the Add Node button to place keypoints on an example image, and connect them with the Add Edge button to create a complete skeleton structure.
Assign descriptive names to each point or edge and click Save to finalize the class template.
Your defined keypoint class will be saved and ready to use in annotation tasks.
Open an image or video in the Annotation Toolbox.
Select the pre-defined keypoint template created in the previous step.
Place the points on the image according to the object's pose. To do this, click once on the edges of the object in the image and map them to the keypoints in the template.
Correct the existing points using Drag a point to move, Disable/Enable point and Remove point buttons so that they repeat the subject pose.
The ViTPose model provides AI-powered assistance for keypoint annotation, enabling automatic pose estimation for animals and humans.
Go to Ecosystem page and find the app Serve ViTPose. Also, you can run it from the Neural Networks page from the category Images → Pose Estimation (Keypoints) → Serve.
Select the ViTPose+ pre-trained model for animal pose estimation and the type of animal you're interested in, press Serve button and wait for the model to deploy.
If you select a model for animal pose estimation, you will also see a list of supported animal species and basic information about the pitfalls of animal pose estimation.
Run NN Image Labeling app, connect to ViTPose, create a bounding box with the animal's type name, and click on Apply the model to ROI.
For the animal pose estimation task, you need to create a bounding box with the class name, which is presented in the list of supported animal species; keypoints skeleton class with the name {yourclassname}keypoints will be created. Otherwise, the keypoints skeleton class with the name animalkeypoints will be created.
Correct keypoints. If there is only a part of the target object on the image, then you can increase the point threshold in app settings to get rid of unnecessary points. Correct the existing points using Drag a point to move, Disable/Enable point, Remove point buttons so that they repeat the animal's pose.
Pro Tip: Increase the point threshold in app settings to avoid unnecessary key points if only part of the object is visible.
An efficient way to automate pose annotation is by combining YOLOv8 for object detection with ViTPose for pose estimation. This combination significantly reduces manual work and speeds up the annotation process. Steps to use the autolabeling pipeline:
Run the YOLOv8 app for detection.
Go to the Ecosystem page, find and run the Serve YOLOv8 app under Images → Object Detection → Serve.
Deploy the app as a REST API service and select a pre-trained model based on your task requirements.
Apply detection and pose estimation models to images project:
Use the Apply Detection and Pose Estimation Models to Images Project app found in the Ecosystem or Images → Pose Estimation (Keypoints) → Inference interfaces.
Configure the settings, and this app will automatically apply the detection and pose estimation models to your images.
Outcome: This combination allows you to quickly identify objects and automatically add accurate pose keypoints, even for complex or blurry objects.
After applying AI-assisted annotation, you may need to fine-tune keypoints manually:
Drag Points to adjust their position.
Enable/Disable Points to toggle visibility as needed.
Remove Points if the AI has incorrectly placed them.
Place predefined template
Left Mouse Click
Toggle keypoint visibility
Command (⌘) + Left Mouse Click
Remove keypoint
Shift + Left Mouse Click
Zoom with Mouse wheel. Hold 
to move scene.