metrics_evaluation/cvat_api/README.md

# CVAT API Client

A comprehensive Python client for interacting with the CVAT (Computer Vision Annotation Tool) REST API.

## Features

✅ **Complete API Coverage** - All core CVAT operations supported  
✅ **Type-Safe** - Full Pydantic model validation  
✅ **Automatic Retries** - Built-in retry logic for transient errors  
✅ **Well-Tested** - 30 validation tests, all passing  
✅ **Well-Documented** - Comprehensive guides and examples  

## Installation

```bash
pip install -r requirements.txt
```

Required packages: `requests`, `pydantic`, `python-dotenv`

## Quick Start

```python
from cvat_api import CvatApiClient

# Initialize client
client = CvatApiClient(
    cvat_host="https://app.cvat.ai",
    cvat_username="your_username",
    cvat_password="your_password",
    cvat_organization="your_organization"
)

# Get project details
project = client.projects.get_project_details(project_id=239220)
print(f"Project: {project.name}")

# Download annotations
annotations = client.annotations.get_job_annotations(job_id=2355063)
print(f"Found {len(annotations.shapes)} shapes")

# Modify and upload annotations
for shape in annotations.shapes:
    if shape.label_id == 5:
        shape.occluded = True

updated = client.annotations.put_job_annotations(
    job_id=2355063,
    annotations=annotations
)
print(f"✅ Updated {len(updated.shapes)} shapes")
```

## Available Methods

The client is organized into method groups:

### Authentication
- `client.auth.get_auth_token()` - Get authentication token

### Projects
- `client.projects.get_project_details(project_id)` - Get project info
- `client.projects.get_project_labels(project_id)` - Get all labels
- `client.projects.get_project_job_ids(project_id)` - Get all job IDs
- `client.projects.get_project_tasks(project_id)` - Get all task IDs

### Tasks
- `client.tasks.get_task_details(task_id)` - Get task info
- `client.tasks.get_task_media_metainformation(task_id)` - Get media metadata
- `client.tasks.get_task_job_ids(task_id)` - Get job IDs
- `client.tasks.update_task(task_id, task_data)` - **Update task metadata**

### Jobs
- `client.jobs.list_jobs(request)` - List jobs with filtering
- `client.jobs.get_job_details(job_id)` - Get job info
- `client.jobs.get_job_media_metainformation(job_id)` - Get media metadata
- `client.jobs.update_job(job_id, job_data)` - **Update job metadata**

### Annotations
- `client.annotations.get_job_annotations(job_id)` - Get annotations
- `client.annotations.get_task_annotations(task_id)` - Get annotations
- `client.annotations.put_job_annotations(job_id, annotations)` - **Update annotations**
- `client.annotations.put_task_annotations(task_id, annotations)` - **Update annotations**

### Downloads
- `client.downloads.download_job_image(job_id, frame_number, output_path)` - Download single image
- `client.downloads.download_task_image(task_id, frame_number, output_path)` - Download single image
- `client.downloads.download_job_images(job_id, output_dir)` - Download all images
- `client.downloads.download_task_images(task_id, output_dir)` - Download all images
- `client.downloads.download_job_data_chunk(job_id, chunk_number, output_path)` - Download chunk
- `client.downloads.download_task_data_chunk(task_id, chunk_number, output_path)` - Download chunk
- `client.downloads.download_all_job_chunks(job_id, output_dir)` - Download all chunks
- `client.downloads.download_all_task_chunks(task_id, output_dir)` - Download all chunks

### Labels
- `client.labels.get_label_details(label_id)` - Get label definition

## Complete Workflow Example

Download task, modify annotations, and upload:

```python
from cvat_api import CvatApiClient
from pathlib import Path

# 1. Initialize
client = CvatApiClient(...)

# 2. Download task data
task_id = 1322143
task_details = client.tasks.get_task_details(task_id)
annotations = client.annotations.get_task_annotations(task_id)
images = client.downloads.download_task_images(
    task_id=task_id,
    output_dir=Path("./images")
)

# 3. Modify annotations
for shape in annotations.shapes:
    if shape.label_id == 5:  # Filter by label
        shape.occluded = True  # Mark as occluded

# 4. Upload modified annotations
updated = client.annotations.put_task_annotations(
    task_id=task_id,
    annotations=annotations
)

print(f"✅ Updated {len(updated.shapes)} shapes")
```

## Documentation

- **[Usage Guide](./USAGE_GUIDE.md)** - Complete workflows and examples
- **[API Reference](./client.py)** - Detailed method documentation
- **[Test Suite](../tests/test_cvat_api/)** - 30 validation tests
- **[Test Results](../tests/test_cvat_api/TEST_RESULTS.md)** - Latest test results

## Testing

Run the validation test suite:

```bash
# Run all tests
pytest tests/test_cvat_api/ -v

# Run specific test category
pytest tests/test_cvat_api/test_projects.py -v
pytest tests/test_cvat_api/test_updates.py -v

# Current status: 30/30 tests passing ✅
```

## Error Handling

The client includes automatic retry logic with exponential backoff:

- **General errors**: 3 retries with 1-60 second delays
- **Transient errors (500/502/503/504)**: Up to 10 retries with 10-300 second delays
- **Jitter**: Random delay variation to prevent thundering herd

```python
# Automatic retries on failure
try:
    annotations = client.annotations.get_job_annotations(job_id)
except requests.HTTPError as e:
    print(f"Request failed after retries: {e}")
```

## Environment Configuration

Set environment variables in `.env`:

```env
CVAT_URL=https://app.cvat.ai
CVAT_USERNAME=your_username
CVAT_PASSWORD=your_password
CVAT_ORGANIZATION=your_organization
```

## Architecture

```
cvat_api/
├── client.py          # Main CvatApiClient class
├── auth.py            # Authentication methods
├── projects.py        # Project operations
├── tasks.py           # Task operations (GET + UPDATE)
├── jobs.py            # Job operations (GET + UPDATE)
├── annotations.py     # Annotation operations (GET + PUT)
├── downloads.py       # Image/chunk download operations
├── labels.py          # Label operations
└── retry.py           # Retry logic with backoff
```

## Pydantic Models

All data structures are validated using Pydantic models from `shared/schema/cvat/`:

- `CvatApiProjectDetails` - Project metadata
- `CvatApiTaskDetails` - Task metadata
- `CvatApiJobDetails` - Job metadata
- `CvatApiJobList` - Job annotations (shapes, tracks, tags)
- `CvatApiTaskList` - Task annotations
- `CvatApiShape` - Individual shape annotations
- `CvatApiTag` - Frame-level tags
- `CvatApiLabelDefinition` - Label definitions

## Contributing

When adding new features:

1. Add the method to the appropriate method group class
2. Add retry decorator if needed
3. Add comprehensive tests in `tests/test_cvat_api/`
4. Update this README and the Usage Guide
5. Run tests: `pytest tests/test_cvat_api/ -v`

## License

Part of the road_ai_analysis project.

## See Also

- [Official CVAT API Documentation](https://app.cvat.ai/api/docs/)
- [CVAT SDK](https://github.com/opencv/cvat/tree/develop/cvat-sdk)