Labeling Jobs
Last updated
Was this helpful?
Last updated
Was this helpful?
Labeling jobs is a powerful tool for efficiently organizing and distributing data annotation tasks within a . It is designed to ensure that annotators work on well-defined and manageable portions of the , follow consistent guidelines, and contribute to the overall success of the annotation project while maintaining data quality and accuracy. It is a critical component of effective team coordination in data annotation efforts.
Annotation Quality Control: ✅ Includes inspections and options for re-labeling when necessary.
Task Distribution: Prevents overlap by assigning jobs to specific team members.
Consistent Guidelines: 📜 Eliminates subjective errors through clear technical requirements and restrictions.
Real-Time Monitoring: 🕒 Tracks progress and allows iterations to improve results.
Acess Control: 🔒 Limits access to specific datasets as needed.
Roles define team members’ responsibilities and permissions within the annotation process. Each member is assigned a role on the Members page, and a single user can have different roles across teams.
Responsibilities: 🛠️
Creates and manages Labeling Jobs.
Provides detailed task descriptions in the Info section.
Monitors progress and analyzes team performance statistics.
Assigns tasks to Annotators and Reviewers.
Permissions: 🔑
Full access to manage the annotation process and review statistics.
Responsibilities: 🛠️
Labels assigned data following predefined instructions.
Submits completed jobs for review.
Permissions: 🔑
Access limited to the Labeling Jobs page.
Cannot create classes, use other applications, or Neural Networks.
Responsibilities: 🛠️
Reviews annotations and sends jobs for re-labeling if needed.
Can create Labeling Jobs and analyze statistics.
Permissions: 🔑
Access to review and finalize jobs or send them for re-annotation.
Navigate to the Labeling Jobs tab. 🗂️
Assign a title and provide a detailed description for Annotators.
Use Markdown or Labeling Guides for detailed instructions with images or videos.
Assign Reviewers and Annotators.
Select the data to annotate.
Define available classes, tags, and any restrictions (e.g., max figures per item).
Filter data by criteria like tags or item count. By default, the entire dataset is selected for annotation.
Complete setup to redirect to the Labeling Jobs page, where tasks are listed for each Annotator.
Annotators should:
Read technical specifications.
Use the annotation toolbox to label data with predefined classes.
Confirm the task is completed and submit it for review.
Once submitted, the job is marked as completed and removed from the Annotator’s list.
Reviewers follow these steps:
Read technical specifications.
Review annotations in the annotation toolbox. 🛠️
Accept (👍) or Reject (👎) the work.
Finalize the Labeling Job if all data is reviewed.
Rejected annotations are sent for re-annotation in a new Labeling Job. 🚀
Managers and Reviewers can click the Stats button to view performance statistics for each Labeling Job.
Watch our concise video tutorial 🎥 to unlock the full potential of collaboration using Labeling Jobs on the Supervisely platform.
Labeling Queues
Labeling Queues is a systematic method for distributing and managing labeling process in a team, where labeling tasks are grouped into queues and sequentially distributed among annotators.
Labeling Consensust
Consensus labeling is an annotation approach where multiple annotators jointly label the same set of images independently.
Labeling Statistics
Understanding your data is extremely important if you want to have precise annotations and consistent training data.