Labeling Jobs
Labeling jobs is a powerful tool for efficiently organizing and distributing data annotation tasks within a team. It is designed to ensure that annotators work on well-defined and manageable portions of the dataset, follow consistent guidelines, and contribute to the overall success of the annotation project while maintaining data quality and accuracy. It is a critical component of effective team coordination in data annotation efforts.
Learn more about Labeling Jobs in Mastering Labeling Jobs: Your Ultimate Guide.
Annotation Quality Control: ✅ Includes inspections and options for re-labeling when necessary.
Task Distribution: Prevents overlap by assigning jobs to specific team members.
Consistent Guidelines: 📜 Eliminates subjective errors through clear technical requirements and restrictions.
Real-Time Monitoring: 🕒 Tracks progress and allows iterations to improve results.
Acess Control: 🔒 Limits access to specific datasets as needed.
Roles in Labeling Jobs
Roles define team members’ responsibilities and permissions within the annotation process. Each member is assigned a role on the Members page, and a single user can have different roles across teams.
📋 Manager
Responsibilities: 🛠️
Creates and manages Labeling Jobs.
Provides detailed task descriptions in the Info section.
Monitors progress and analyzes team performance statistics.
Assigns tasks to Annotators and Reviewers.
Permissions: 🔑
Full access to manage the annotation process and review statistics.
✍️ Annotator (Labeler)
Responsibilities: 🛠️
Labels assigned data following predefined instructions.
Submits completed jobs for review.
Permissions: 🔑
Access limited to the Labeling Jobs page.
Cannot create classes, use other applications, or Neural Networks.
🔍 Reviewer
Responsibilities: 🛠️
Reviews annotations and sends jobs for re-labeling if needed.
Can create Labeling Jobs and analyze statistics.
Permissions: 🔑
Access to review and finalize jobs or send them for re-annotation.
Using Labeling Jobs: Step-By-Step Guide
Before you start
Make sure that you have:
dedicated Team assigned with proper roles
necessary projects for annotation with relevant properties (clasess/figures/tags) defined
Step 1. Create a Labeling Job
Navigate to the Labeling Jobs tab. 🗂️
Assign a title and provide a detailed description for Annotators.
Use Markdown or Labeling Guides for detailed instructions with images or videos.
Assign Reviewers and Annotators.
Select the data to annotate.
Define available classes, tags, and any restrictions (e.g., max figures per item).
Filter data by criteria like tags or item count. By default, the entire dataset is selected for annotation.
Complete setup to redirect to the Labeling Jobs page, where tasks are listed for each Annotator.
Step 2. Complete the Job
Annotators should:
Read technical specifications.
Use the annotation toolbox to label data with predefined classes.
Confirm the task is completed and submit it for review.
Once submitted, the job is marked as completed and removed from the Annotator’s list.
Step 3. Review and Quality Control
Reviewers follow these steps:
Read technical specifications.
Review annotations in the annotation toolbox. 🛠️
Accept (👍) or Reject (👎) the work.
Finalize the Labeling Job if all data is reviewed.
Rejected annotations are sent for re-annotation in a new Labeling Job. 🚀
Labeling Job Statistics
Managers and Reviewers can click the Stats button to view performance statistics for each Labeling Job.
Video Tutorial
Watch our concise video tutorial 🎥 to unlock the full potential of collaboration using Labeling Jobs on the Supervisely platform.
Labeling Queues
Labeling Queues is a systematic method for distributing and managing labeling process in a team, where labeling tasks are grouped into queues and sequentially distributed among annotators.
Labeling Consensust
Consensus labeling is an annotation approach where multiple annotators jointly label the same set of images independently.
Labeling Statistics
Understanding your data is extremely important if you want to have precise annotations and consistent training data.
Last updated