metrics_evaluation/cvat_api/README.md

CVAT API Client

A comprehensive Python client for interacting with the CVAT (Computer Vision Annotation Tool) REST API.

Features

✅ Complete API Coverage - All core CVAT operations supported
✅ Type-Safe - Full Pydantic model validation
✅ Automatic Retries - Built-in retry logic for transient errors
✅ Well-Tested - 30 validation tests, all passing
✅ Well-Documented - Comprehensive guides and examples

Installation

pip install -r requirements.txt

Required packages: requests, pydantic, python-dotenv

Quick Start

from cvat_api import CvatApiClient

# Initialize client
client = CvatApiClient(
    cvat_host="https://app.cvat.ai",
    cvat_username="your_username",
    cvat_password="your_password",
    cvat_organization="your_organization"
)

# Get project details
project = client.projects.get_project_details(project_id=239220)
print(f"Project: {project.name}")

# Download annotations
annotations = client.annotations.get_job_annotations(job_id=2355063)
print(f"Found {len(annotations.shapes)} shapes")

# Modify and upload annotations
for shape in annotations.shapes:
    if shape.label_id == 5:
        shape.occluded = True

updated = client.annotations.put_job_annotations(
    job_id=2355063,
    annotations=annotations
)
print(f"✅ Updated {len(updated.shapes)} shapes")

Available Methods

The client is organized into method groups:

Authentication

• client.auth.get_auth_token() - Get authentication token

Projects

• client.projects.get_project_details(project_id) - Get project info
• client.projects.get_project_labels(project_id) - Get all labels
• client.projects.get_project_job_ids(project_id) - Get all job IDs
• client.projects.get_project_tasks(project_id) - Get all task IDs

Tasks

• client.tasks.get_task_details(task_id) - Get task info
• client.tasks.get_task_media_metainformation(task_id) - Get media metadata
• client.tasks.get_task_job_ids(task_id) - Get job IDs
• client.tasks.update_task(task_id, task_data) - Update task metadata

Jobs

• client.jobs.list_jobs(request) - List jobs with filtering
• client.jobs.get_job_details(job_id) - Get job info
• client.jobs.get_job_media_metainformation(job_id) - Get media metadata
• client.jobs.update_job(job_id, job_data) - Update job metadata

Annotations

• client.annotations.get_job_annotations(job_id) - Get annotations
• client.annotations.get_task_annotations(task_id) - Get annotations
• client.annotations.put_job_annotations(job_id, annotations) - Update annotations
• client.annotations.put_task_annotations(task_id, annotations) - Update annotations

Downloads

• client.downloads.download_job_image(job_id, frame_number, output_path) - Download single image
• client.downloads.download_task_image(task_id, frame_number, output_path) - Download single image
• client.downloads.download_job_images(job_id, output_dir) - Download all images
• client.downloads.download_task_images(task_id, output_dir) - Download all images
• client.downloads.download_job_data_chunk(job_id, chunk_number, output_path) - Download chunk
• client.downloads.download_task_data_chunk(task_id, chunk_number, output_path) - Download chunk
• client.downloads.download_all_job_chunks(job_id, output_dir) - Download all chunks
• client.downloads.download_all_task_chunks(task_id, output_dir) - Download all chunks

Labels

• client.labels.get_label_details(label_id) - Get label definition

Complete Workflow Example

Download task, modify annotations, and upload:

from cvat_api import CvatApiClient
from pathlib import Path

# 1. Initialize
client = CvatApiClient(...)

# 2. Download task data
task_id = 1322143
task_details = client.tasks.get_task_details(task_id)
annotations = client.annotations.get_task_annotations(task_id)
images = client.downloads.download_task_images(
    task_id=task_id,
    output_dir=Path("./images")
)

# 3. Modify annotations
for shape in annotations.shapes:
    if shape.label_id == 5:  # Filter by label
        shape.occluded = True  # Mark as occluded

# 4. Upload modified annotations
updated = client.annotations.put_task_annotations(
    task_id=task_id,
    annotations=annotations
)

print(f"✅ Updated {len(updated.shapes)} shapes")

Documentation

• Usage Guide - Complete workflows and examples
• API Reference - Detailed method documentation
• Test Suite - 30 validation tests
• Test Results - Latest test results

Testing

Run the validation test suite:

# Run all tests
pytest tests/test_cvat_api/ -v

# Run specific test category
pytest tests/test_cvat_api/test_projects.py -v
pytest tests/test_cvat_api/test_updates.py -v

# Current status: 30/30 tests passing ✅

Error Handling

The client includes automatic retry logic with exponential backoff:

• General errors: 3 retries with 1-60 second delays
• Transient errors (500/502/503/504): Up to 10 retries with 10-300 second delays
• Jitter: Random delay variation to prevent thundering herd

# Automatic retries on failure
try:
    annotations = client.annotations.get_job_annotations(job_id)
except requests.HTTPError as e:
    print(f"Request failed after retries: {e}")

Environment Configuration

Set environment variables in .env:

CVAT_URL=https://app.cvat.ai
CVAT_USERNAME=your_username
CVAT_PASSWORD=your_password
CVAT_ORGANIZATION=your_organization

Architecture

cvat_api/
├── client.py          # Main CvatApiClient class
├── auth.py            # Authentication methods
├── projects.py        # Project operations
├── tasks.py           # Task operations (GET + UPDATE)
├── jobs.py            # Job operations (GET + UPDATE)
├── annotations.py     # Annotation operations (GET + PUT)
├── downloads.py       # Image/chunk download operations
├── labels.py          # Label operations
└── retry.py           # Retry logic with backoff

Pydantic Models

All data structures are validated using Pydantic models from shared/schema/cvat/:

• CvatApiProjectDetails - Project metadata
• CvatApiTaskDetails - Task metadata
• CvatApiJobDetails - Job metadata
• CvatApiJobList - Job annotations (shapes, tracks, tags)
• CvatApiTaskList - Task annotations
• CvatApiShape - Individual shape annotations
• CvatApiTag - Frame-level tags
• CvatApiLabelDefinition - Label definitions

Contributing

When adding new features:

1. Add the method to the appropriate method group class
2. Add retry decorator if needed
3. Add comprehensive tests in tests/test_cvat_api/
4. Update this README and the Usage Guide
5. Run tests: pytest tests/test_cvat_api/ -v

License

Part of the road_ai_analysis project.

See Also

• Official CVAT API Documentation
• CVAT SDK