Spaces:
Build error
Build error
metadata
title: IIIF Studio
emoji: 📜
colorFrom: blue
colorTo: yellow
sdk: docker
app_port: 7860
pinned: false
IIIF Studio
A generic platform for generating AI-augmented scholarly editions from digitized heritage documents — medieval manuscripts, incunabula, cartularies, archives, charters, papyri. Any document type, any era, any language.
IIIF Studio ingests images from any IIIF-compliant server, analyzes them with multimodal AI (Google Gemini, Mistral), and produces structured scholarly data: diplomatic OCR, layout detection, translations, commentaries, and iconographic analysis — all exportable as ALTO XML, METS, and IIIF Presentation 3.0 manifests.
Images are never stored locally. The platform streams them from origin servers using the IIIF Image API, storing only the AI-generated metadata (~5 KB per page instead of ~50 MB).
Features
- IIIF-native architecture — images streamed from origin servers (Gallica, BnF, Bodleian, etc.) with tiled deep zoom via OpenSeadragon
- Multi-provider AI — Google AI Studio, Vertex AI, Mistral AI. Model selected per corpus, auto-detected from environment
- Profile-driven analysis — 4 built-in corpus profiles (medieval illuminated, medieval textual, early modern print, modern handwritten), each with tailored prompts and active layers
- Structured output — layout regions with bounding boxes, diplomatic OCR, translations (FR/EN), scholarly and public commentary, iconographic analysis, uncertainty tracking
- Standards-compliant export — IIIF Presentation 3.0 manifests (with Image Service for tiled zoom), ALTO XML, METS XML, ZIP bundles
- Human-in-the-loop — editorial correction interface with versioned history and rollback
- Full-text search — accent-insensitive search across OCR text, translations, and iconographic tags
Architecture
┌─────────────────────────────────────────────────────────────────────────────────┐
│ IIIF IMAGE SERVERS │
│ Gallica · BnF · Bodleian · Europeana · ... │
│ (origin — images are never copied) │
└──────────┬────────────────────┬─────────────────────────────┬───────────────────┘
│ │ │
│ info.json │ /full/!1500,1500/ │ /full/max/
│ + tiles │ 0/default.jpg │ 0/default.jpg
│ │ (1500px for AI) │
│ │ │
┌──────────▼──────────┐ ┌──────▼───────────┐ ┌──────────────▼──────────────────┐
│ │ │ │ │ │
│ FRONTEND (SPA) │ │ BACKEND (API) │ │ EXPORT GENERATORS │
│ React + Vite │ │ FastAPI │ │ │
│ │ │ │ │ IIIF Manifest 3.0 │
│ ┌─────────────────┐ │ │ ┌──────────────┐ │ │ (with Image Service refs) │
│ │ OpenSeadragon │ │ │ │ Ingestion │ │ │ │
│ │ IIIF tiled zoom │ │ │ │ │ │ │ METS XML │
│ │ (info.json → │ │ │ │ manifest URL │ │ │ (IIIF URLs, not file paths) │
│ │ deep zoom) │ │ │ │ → detect svc │ │ │ │
│ └────────┬─────────┘ │ │ │ → store meta │ │ │ ALTO XML │
│ │ │ │ └──────┬───────┘ │ │ (text geometry per page) │
│ ┌────────▼─────────┐ │ │ │ │ │ │
│ │ Region overlays │ │ │ ┌─────▼────────┐│ │ ZIP bundle │
│ │ (bbox from │ │ │ │ AI Pipeline ││ │ (manifest + METS + ALTO) │
│ │ master.json, │ │ │ │ ││ └─────────────────────────────────┘
│ │ scaled to │ │ │ │ fetch 1500px││
│ │ canvas coords) │ │ │ │ in memory ││ ┌──────────────────────┐
│ └──────────────────┘ │ │ │ │ ││ │ │
│ │ │ │ ▼ ││ │ AI PROVIDERS │
│ ┌──────────────────┐ │ │ │ send bytes │├───────►│ │
│ │ Pages │ │ │ │ to AI ││ │ Google Gemini │
│ │ Home · Reader │ │ │ │ │ ││◄───────│ Vertex AI │
│ │ Editor · Admin │ │ │ │ ▼ ││ JSON │ Mistral AI │
│ └────────┬─────────┘ │ │ │ discard img ││ │ │
│ │ │ │ │ keep JSON ││ │ (auto-detected from │
│ │ REST API │ │ │ scale bbox ││ │ environment vars) │
│ │ /api/v1/* │ │ └─────┬───────┘│ └──────────────────────┘
└──────────┼───────────┘ │ │ │
│ │ ┌─────▼────────┐│
│ │ │ Response ││
│ │ │ Parser ││
└─────────────┤ │ ││
│ │ raw JSON ││
│ │ → layout ││
│ │ → OCR ││
│ │ → regions ││
│ └─────┬───────┘│
│ │ │
│ ┌─────▼────────┐│
│ │ Master ││
│ │ Writer ││
│ │ ││
│ │ ai_raw.json ││ ┌───────────────────────────────┐
│ │ master.json ││ │ │
│ └─────┬───────┘│ │ LOCAL STORAGE │
│ │ │ │ │
└───────┼─────────┘ │ SQLite (corpus, pages, │
│ │ manuscripts, jobs, models) │
└──────────────► │
│ data/corpora/{slug}/pages/ │
│ {folio}/master.json │
│ {folio}/ai_raw.json │
│ {folio}/alto.xml │
│ │
│ ~5 KB per page (JSON only) │
│ NO image binaries │
└───────────────────────────────┘
PIPELINE FLOW (per page):
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ 1.INGEST │───►│ 2.DETECT │───►│ 3.FETCH │───►│ 4.AI │───►│ 5.PARSE │
│ │ │ │ │ │ │ │ │ │
│ manifest │ │ IIIF svc │ │ 1500px │ │ send │ │ layout │
│ URL │ │ URL + │ │ JPEG in │ │ image + │ │ regions │
│ │ │ canvas │ │ memory │ │ prompt │ │ OCR │
│ │ │ dims │ │ (discard │ │ to │ │ bbox │
│ │ │ │ │ after) │ │ provider │ │ │
└──────────┘ └──────────┘ └──────────┘ └──────────┘ └────┬─────┘
│
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌────▼─────┐
│ 8.EXPORT │◄───│ 7.REVIEW │◄───│ 6.WRITE │◄──────────────────│ 5b.SCALE │
│ │ │ │ │ │ │ │
│ IIIF 3.0 │ │ human │ │ ai_raw + │ │ bbox │
│ ALTO XML │ │ correct │ │ master │ │ deriv → │
│ METS XML │ │ validate │ │ .json │ │ canvas │
│ ZIP │ │ version │ │ + ALTO │ │ coords │
└──────────┘ └──────────┘ └──────────┘ └──────────┘
Tech stack
| Layer | Technology |
|---|---|
| Backend | Python 3.11+, FastAPI, Uvicorn |
| Database | SQLite via SQLAlchemy 2.0 async + aiosqlite |
| Validation | Pydantic v2 |
| AI providers | Google Gemini (google-genai SDK), Mistral AI |
| Image viewer | OpenSeadragon (IIIF tiled zoom) |
| Frontend | React 18, TypeScript, Vite, Tailwind CSS, React Router |
| Exports | lxml (ALTO/METS XML), IIIF Presentation 3.0 |
| Deployment | Docker (HuggingFace Spaces) |
Quick start
Docker (recommended)
git clone https://github.com/maribakulj/IIIF-Studio.git && cd IIIF-Studio
# Configure at least one AI provider key
cp .env.example .env
# Edit .env and add your API key(s)
# Build and run
docker compose -f infra/docker-compose.yml up --build
# Open http://localhost:7860
Local development
# Backend
cd backend
pip install -e ".[dev]"
uvicorn app.main:app --reload --port 7860
# Frontend (separate terminal)
cd frontend
npm install
npm run dev
The API is available at http://localhost:7860/api/v1/. Interactive Swagger docs at http://localhost:7860/docs.
Usage workflow
- Create a corpus — select a profile matching your document type
- Ingest pages — provide a IIIF manifest URL, direct image URLs, or upload files
- Select an AI model — choose a provider and model from the detected options
- Run the pipeline — AI analyzes each page: layout detection, OCR, translation, commentary
- Review and correct — use the Editor to validate, correct OCR, adjust regions
- Export — download IIIF manifest, ALTO XML, METS XML, or a ZIP bundle
Corpus profiles
Profiles control which analysis layers are active, which prompt templates are used, and what uncertainty thresholds apply.
| Profile | Script | Languages | Key layers |
|---|---|---|---|
medieval-illuminated |
Caroline | Latin, French | OCR, translation, iconography, commentary, material notes |
medieval-textual |
Gothic | Latin, French | OCR, translation, scholarly commentary |
early-modern-print |
French, Latin | OCR, summary | |
modern-handwritten |
Cursive | French | OCR, summary |
Custom profiles can be added as JSON files in the profiles/ directory with matching prompt templates in prompts/.
AI providers
The backend auto-detects available providers from environment variables. No global selector — the model is chosen per corpus from the admin interface.
| Provider | Environment variable | Notes |
|---|---|---|
| Google AI Studio | GOOGLE_AI_STUDIO_API_KEY |
Free tier, good for development |
| Vertex AI (service account) | VERTEX_SERVICE_ACCOUNT_JSON |
Institutional deployments |
| Mistral AI | MISTRAL_API_KEY |
Alternative provider |
At least one key is required for the pipeline to function. Keys must never appear in code, commits, or Docker images.
API reference
All endpoints are prefixed with /api/v1/. Full OpenAPI docs available at /docs.
Corpus management
| Method | Endpoint | Description |
|---|---|---|
GET |
/corpora |
List all corpora |
POST |
/corpora |
Create a corpus (slug + title + profile) |
GET |
/corpora/{id} |
Get a corpus |
DELETE |
/corpora/{id} |
Delete a corpus (cascades) |
GET |
/corpora/{id}/manuscripts |
List manuscripts in a corpus |
Ingestion
| Method | Endpoint | Description |
|---|---|---|
POST |
/corpora/{id}/ingest/iiif-manifest |
Ingest from a IIIF manifest URL |
POST |
/corpora/{id}/ingest/iiif-images |
Ingest from direct image URLs |
POST |
/corpora/{id}/ingest/files |
Upload image files |
AI pipeline
| Method | Endpoint | Description |
|---|---|---|
GET |
/providers |
List detected AI providers |
GET |
/providers/{type}/models |
List models for a provider |
PUT |
/corpora/{id}/model |
Set AI model for a corpus |
POST |
/corpora/{id}/run |
Run pipeline on all pages |
POST |
/pages/{id}/run |
Run pipeline on a single page |
GET |
/jobs/{id} |
Check job status |
POST |
/jobs/{id}/retry |
Retry a failed job |
Pages and content
| Method | Endpoint | Description |
|---|---|---|
GET |
/pages/{id} |
Page metadata |
GET |
/pages/{id}/master-json |
Full page master (canonical JSON) |
GET |
/pages/{id}/layers |
List annotation layers |
POST |
/pages/{id}/corrections |
Apply editorial corrections |
GET |
/pages/{id}/history |
Version history |
GET |
/search?q= |
Full-text search across all pages |
Export
| Method | Endpoint | Description |
|---|---|---|
GET |
/manuscripts/{id}/iiif-manifest |
IIIF Presentation 3.0 manifest |
GET |
/manuscripts/{id}/mets |
METS XML |
GET |
/pages/{id}/alto |
ALTO XML |
GET |
/manuscripts/{id}/export.zip |
ZIP bundle (manifest + METS + ALTO) |
Data model
Each analyzed page produces a master.json — the canonical source of truth for all exports.
PageMaster
├── image → IIIF service URL, canvas dimensions, provenance
├── layout → regions with bounding boxes [x, y, w, h] in absolute pixels
├── ocr → diplomatic text, confidence, uncertain segments
├── translation → French, English
├── summary → short + detailed
├── commentary → public, scholarly, sourced claims with certainty levels
├── extensions → profile-specific data (iconography, materiality, etc.)
├── processing → provider, model, prompt version, timestamp
└── editorial → status (machine_draft → validated → published), version
Bounding boxes follow the convention [x, y, width, height] in absolute pixels of the original image. Coordinates are automatically scaled from AI analysis space to full canvas dimensions.
IIIF-native image handling
IIIF Studio operates in two modes:
IIIF-native mode (default for manifest/URL ingestion)
- Images are never downloaded or stored locally
- At ingestion: IIIF Image Service URL and canvas dimensions are extracted from the manifest
- At analysis: a 1500px derivative is fetched in memory via the IIIF Image API (
{service}/full/!1500,1500/0/default.jpg), sent to the AI, then discarded - In the viewer: OpenSeadragon loads
info.jsonfrom the IIIF server for native tiled deep zoom - Storage per page: ~5 KB (JSON metadata only)
File upload mode (for non-IIIF sources)
- Uploaded images are stored locally in
data/corpora/{slug}/ - Derivatives (1500px) and thumbnails (256px) are created on disk
- Storage per page: ~50 MB (images + JSON)
Project structure
IIIF-Studio/
├── backend/
│ ├── app/
│ │ ├── main.py # FastAPI entry point
│ │ ├── config.py # Pydantic settings from env vars
│ │ ├── api/v1/ # REST endpoints
│ │ ├── models/ # SQLAlchemy ORM models
│ │ ├── schemas/ # Pydantic v2 schemas (canonical)
│ │ └── services/
│ │ ├── ai/ # Provider factory, analyzer, prompt loader
│ │ ├── ingest/ # IIIF fetcher, service detection
│ │ ├── image/ # Normalizer (in-memory + legacy disk)
│ │ └── export/ # ALTO, METS, IIIF manifest generators
│ ├── tests/ # 585 tests (pytest + pytest-asyncio)
│ └── pyproject.toml
├── frontend/
│ ├── src/
│ │ ├── App.tsx # React Router (/, /admin, /reader, /editor)
│ │ ├── lib/api.ts # Typed API client
│ │ ├── pages/ # Home, Reader, Editor, Admin
│ │ └── components/ # Viewer (OpenSeadragon), retro UI system
│ └── package.json
├── profiles/ # 4 corpus profile JSON files
├── prompts/ # 9 prompt templates organized by profile
├── Dockerfile # Multi-stage build (Node + Python)
├── infra/docker-compose.yml # Local development
└── .env.example # Environment variable template
Testing
cd backend
pip install -e ".[dev]"
pytest tests/ -v --cov=app
Expected result: 585 passed, 3 skipped.
All AI calls are mocked in tests — no API keys required to run the test suite.
Deployment
HuggingFace Spaces
This repository is configured for HuggingFace Spaces with Docker SDK on port 7860. AI keys are stored as Space secrets (Settings → Repository secrets).
The CI pipeline (.github/workflows/) runs tests on every push and auto-deploys to HuggingFace Spaces on merge to main.
Self-hosted
docker build -t iiif-studio .
docker run -p 7860:7860 \
-e GOOGLE_AI_STUDIO_API_KEY=your_key \
-v ./data:/app/data \
iiif-studio