What is data annotation?
Data annotation is the process of marking raw data with the information a model is expected to learn. The term is often used interchangeably with labeling. Annotation covers everything from a single tag on a document to a detailed outline of every object in a video frame. The output is a set of data labels paired with the original files, stored in an annotation format the training code can read.
The common annotation types are the following.
- Classification tags assign one or more categories to a whole item, such as “urgent” on a support ticket.
- Bounding boxes are rectangles drawn around objects in an image, the standard input for object detection.
- Polygons trace the exact outline of an irregular object, such as a damaged area on a pallet.
- Segmentation masks assign every pixel to a class, which image segmentation models need.
- Text spans mark the start and end of an entity or clause, such as a termination date in a contract.
How data annotation works
An annotation project starts with a guideline document that defines each label and shows the edge cases. Annotators work through items in a labeling tool. They may be in-house staff, a vendor team (see outsourced labeling), or a model making first-pass suggestions that humans correct. A second person reviews a sample of the work, and disagreement rates are tracked. Model-assisted labeling, where an early model proposes boxes or tags and people fix them, is much faster than starting from a blank image.
Data annotation in practice
A regional food distributor wants a camera system that flags damaged cases as they move down a conveyor. The team collects ten thousand images, writes a guideline that separates crushed corners from cosmetic scuffs, and has warehouse supervisors draw polygons around the damage. The first review pass finds that two supervisors treated dented lids differently. The guideline is updated and those images are redone before training starts. Without that review, the model would have learned a rule nobody intended.
What to watch for
- Pick the simplest annotation type the use case needs, because polygons and masks cost several times more per image than boxes.
- Measure agreement between annotators before trusting the dataset.
- Keep annotated data under the same access controls as the source data, since photos and documents often contain personal information.
- Plan for ongoing annotation, since production data changes and the model needs fresh labels to stay accurate.