Update model card: Q6_K (fixes tool_call corruption), full-precision training, corrected specs
e6d5daf verified | license: apache-2.0 | |
| base_model: unsloth/Qwen3-4B-Instruct-2507 | |
| tags: [recommendation, media-server, plex, jellyfin, structured-output, qwen3] | |
| language: [en] | |
| library_name: transformers | |
| pipeline_tag: text-generation | |
| # Marquee | |
| **v1.1** · contract v1 · base Qwen3-4B-Instruct-2507 | |
| Turns one item in a media library into Plex-style rows of other items in **that same | |
| library**. Built for self-hosted media servers. | |
| ``` | |
| IN: "The Sopranos" | |
| OUT: | |
| Shows like The Sopranos Gomorrah, Boardwalk Empire, The Wire, ZeroZeroZero | |
| More from Tim Van Patten Game of Thrones, Black Mirror, Boardwalk Empire | |
| More with Edie Falco Nurse Jackie, Oz | |
| Shot by Alik Sakharov Game of Thrones, House of Cards | |
| ``` | |
| ## Read this before downloading the weights | |
| **The model alone does nothing useful.** It decides *which* rows to show, their order, | |
| and what to call them. It does not choose what goes in them -- a retriever does, by | |
| joining your library on shared cast, crew and franchise, and by embedding similarity. | |
| The model never even sees item ids. It answers with **index references** into | |
| candidates it was handed, so it cannot invent a title that isn't in your library. | |
| Hallucination isn't low here, it's structurally impossible. | |
| ```bash | |
| pip install marquee-ai | |
| ``` | |
| ## Download | |
| ```bash | |
| hf download KernelMedia/marquee-ai --local-dir marquee | |
| cd marquee && ollama create marquee -f Modelfile | |
| ``` | |
| ~3.6 GB: the model plus a catalog of ~195,000 films and shows with full cast and crew. | |
| ## Requirements | |
| | | | | |
| |---|---| | |
| | VRAM | **~4.7 GB** (Q6_K at 6144 context). Runs on a 6GB card, or CPU. | | |
| | RAM | ~2.5 GB for the 195k-title catalog index | | |
| | Disk | ~3.3 GB model + ~240 MB catalog + ~300 MB embeddings | | |
| | Latency | ~1s per request on an RTX 3060; cached responses are instant | | |
| ## Quickstart | |
| ```bash | |
| marquee serve --library catalog.jsonl --port 8080 | |
| ``` | |
| That's the whole setup -- catalog and embeddings ship precomputed. | |
| ## API | |
| ### `POST /recommend` | |
| ```bash | |
| curl -X POST localhost:8080/recommend -H 'Content-Type: application/json' \ | |
| -d '{"title": "The Sopranos", "row_size": 20}' | |
| ``` | |
| ```json | |
| { | |
| "seed": {"id": "wd:Q23628", "title": "The Sopranos", "year": 1999, | |
| "media_type": "tv", "rating": 0.0, "owned": true}, | |
| "rows": [ | |
| {"type": "thematic", "title": "Shows like The Sopranos", | |
| "items": [ | |
| {"id": "wd:Q...", "title": "Gomorrah", "year": 2014, | |
| "media_type": "tv", "owned": true}, | |
| {"id": "wd:Q...", "title": "Boardwalk Empire", "year": 2010, | |
| "media_type": "tv", "owned": true} | |
| ]}, | |
| {"type": "director", "title": "More from Tim Van Patten", | |
| "items": [ | |
| {"id": "wd:Q...", "title": "Game of Thrones", "year": 2011, | |
| "media_type": "tv", "owned": true} | |
| ]} | |
| ] | |
| } | |
| ``` | |
| Body: `seed_id` **or** `title`, plus optional `year`, `media_type`, `row_size`, | |
| `owned_only`, `semantic_k`. Rows arrive in a fixed order -- thematic, franchise, | |
| director, cast, crew. | |
| An ambiguous title returns **409** with the candidates rather than guessing. A catalog | |
| this size holds many remakes sharing a bare title: | |
| ```json | |
| {"error": "22 items match 'Macbeth'; pass seed_id, or narrow with year/media_type", | |
| "matches": [ | |
| {"id": "wd:Q15934383", "title": "Macbeth", "year": 2015, "media_type": "movie"}, | |
| {"id": "wd:Q2573008", "title": "Macbeth", "year": 1951, "media_type": "movie"} | |
| ]} | |
| ``` | |
| `{"title": "Macbeth", "year": 2015}` or `{"title": "Macbeth (2015)"}` resolves it. | |
| ### `GET /metadata` and `POST /metadata/batch` | |
| Full catalog record for one item, or up to 500 in one call. Pure index lookups -- no | |
| model, no GPU. Look up by `id`, `tmdb_id`, `imdb_id` (all exact), or `title` (fuzzy, | |
| can be ambiguous like `/recommend`). | |
| ```bash | |
| curl 'localhost:8080/metadata?tmdb_id=335984' | |
| ``` | |
| ```json | |
| { | |
| "id": "wd:Q21500755", "title": "Blade Runner 2049", "media_type": "movie", | |
| "year": 2017, "genres": ["cyberpunk", "dystopian film", "neo-noir"], | |
| "people": [{"name": "Denis Villeneuve", "role": "director"}, | |
| {"name": "Roger Deakins", "role": "cinematographer"}], | |
| "summary": "Blade Runner 2049 is a 2017 American science fiction film...", | |
| "franchise": "Blade Runner", "runtime_min": 164, | |
| "country": "United Kingdom", "language": "English", | |
| "external": {"wikidata": "Q21500755", "tmdb": "335984", "imdb": "tt1856101"} | |
| } | |
| ``` | |
| ```bash | |
| curl -X POST localhost:8080/metadata/batch -H 'Content-Type: application/json' -d '{ | |
| "queries": [{"tmdb_id": 335984}, {"imdb_id": "tt0141842"}] | |
| }' | |
| ``` | |
| Every query gets a result in the order sent -- found, ambiguous with candidates, or | |
| not found -- so you can zip the response against your input. | |
| ### `GET /search?q=` and `GET /person?name=` | |
| `/search` matches titles **and** people in one call -- "Scorsese" isn't a title but has | |
| 109 credits. `/person` returns everything someone worked on, films and shows together, | |
| ranked by a popularity-weighted score, deduplicated across roles. Both are index | |
| lookups, no model call. | |
| ```bash | |
| curl 'localhost:8080/person?name=Roger%20Deakins&limit=3' | |
| ``` | |
| ```json | |
| {"person": "Roger Deakins", "count": 3, "credits": [ | |
| {"id": "wd:...", "title": "The Shawshank Redemption", "year": 1994, | |
| "score": 8.66, "roles": ["cinematographer"]} | |
| ]} | |
| ``` | |
| `score` is what ranking actually uses -- popularity-weighted, so a well-known film | |
| outranks an obscure one even without a raw rating to sort by. | |
| ### `GET /health` and `DELETE /cache` | |
| ```json | |
| {"status": "ok", "contract_version": 1, | |
| "model_self_test": {"passed": true, "detail": "contract v1 OK (3/3 clean)"}, | |
| "library_items": 194509, "owned": 194509, "requestable": 0} | |
| ``` | |
| `status` is `degraded` if the self-test fails -- model and package disagree, or the | |
| chat template is wrong. Recommendations are cached per seed; call `DELETE /cache` | |
| after a library rescan or you'll serve rows referring to items you no longer have. | |
| ## Restricting results to your own library (optional) | |
| By default it recommends from the whole catalog. To mark what's actually on your | |
| server, so a UI can show a Request button on the rest: | |
| ```bash | |
| marquee ingest --server library.jsonl --catalog catalog.jsonl \ | |
| --out lib.jsonl --mode requestable | |
| marquee serve --library lib.jsonl | |
| ``` | |
| Your export needs one line per item; `tmdb_id` and `media_type` are enough: | |
| ```json | |
| {"id": "myserver-04471", "tmdb_id": 335984, "media_type": "movie"} | |
| ``` | |
| ## Evaluation | |
| 50 unseen holdout examples, run through plain `transformers.generate` with **no | |
| grammar constraint**, on the unmerged adapter -- so this measures the weights, not a | |
| decoder forcing shape, and not the quantization the GGUF ships with. | |
| | Metric | Base Qwen3-4B | **Marquee** | | |
| |---|---|---| | |
| | JSON parses | -- | **100%** | | |
| | Fully contract-clean | -- | **100%** | | |
| | Hallucinated ids | -- | **0.0%** | | |
| | Row-order violations | -- | **0%** | | |
| That number does not carry over unchanged to the shipped GGUF: merging a LoRA adapter | |
| and quantizing it are both lossy steps this harness doesn't exercise. Measured | |
| end-to-end through actual Ollama serving (20 held-out seeds, real greedy decoding) -- | |
| **Q6_K: 19/20 clean.** See Limitations below for what pushed us to Q6_K over a smaller | |
| quant. | |
| **What this is, honestly:** the fine-tune buys format discipline, not taste. The | |
| retriever picks every item; the model decides which rows to show and writes one row | |
| title. It does not make recommendations smarter on its own -- that lives in the | |
| retriever's embeddings and joins. | |
| ## Training | |
| - **Base**: Qwen3-4B-Instruct-2507, full-precision LoRA (bf16 base, not QLoRA) r=16, | |
| lr 2e-4, 1 epoch, 3,684 examples | |
| - **Data**: generated by Qwen3-8B-AWQ over the catalog below, normalised by a | |
| deterministic repair pass. 0.4% rejected. | |
| - **Hardware**: one RTX 3060 12GB, ~2h48m, 10.2 GB peak VRAM | |
| - **Targets are ~30 tokens**: the model emits index references, not item lists | |
| - **Why full precision, not QLoRA**: a QLoRA adapter is calibrated against its 4-bit | |
| base's numerical behaviour. Merging it into full-precision weights for GGUF export | |
| bakes in a dequantized representation the adapter never saw, which destabilized this | |
| model badly enough to periodically emit `<tool_call>` instead of JSON. Training | |
| directly on real bf16 weights removes that mismatch. | |
| ## Limitations | |
| - **Thematic rows are the weakest part.** Built from `bge-small` embeddings blended | |
| with keyword and genre overlap; ~31% of the catalog has no plot summary to embed, | |
| so those titles lean on genre/subject tags alone. | |
| - **Franchise coverage is thin (~2%)** compared to a TMDB-sourced catalog -- series | |
| membership is sparse in the source data. | |
| - **Same-medium only.** A show returns shows, a film returns films. | |
| - **Library-size sensitive.** A director with one other credit present gets no row. | |
| Deliberate -- no padding. | |
| - **Movies and TV only.** English metadata. | |
| - **No poster/image data.** The catalog is text and structured facts only; nothing in | |
| it or the API returns image URLs. Pull artwork from your media server, or from | |
| TMDB/Fanart yourself using the `external.tmdb`/`external.imdb` ids in `/metadata`. | |
| ## ⚠️ GGUF: the chat template is not optional | |
| Ollama will not use the Jinja template embedded in GGUF metadata and falls back to a | |
| bare `{{ .Prompt }}` passthrough, so the model never receives the `<|im_start|>` | |
| markers it was trained on and reverts to base-model behaviour. The supplied | |
| `Modelfile` carries an explicit ChatML `TEMPLATE` block. **Do not remove it.** | |
| Two things degrade output independently, and both are already handled in the shipped | |
| Modelfile/GGUF: | |
| - `temperature` must stay low -- 0.1 gave 8/8 clean over 8 test seeds without a | |
| grammar, 0.3 gave 5/8. The Modelfile ships 0.1. | |
| - **Don't requantize below Q6_K.** llama.cpp's Q4_K_M rounding alone -- independent of | |
| temperature, independent of the LoRA-merge issue above -- was enough to collapse | |
| this model into repeating `<tool_call>` tokens instead of JSON on most requests | |
| (1/20 clean vs Q6_K's 19/20, same weights, same greedy decoding, only the quant | |
| level changed). Q6_K is the smallest quant confirmed clean. | |
| Verify any deployment by calling it **without** a JSON schema. A working fine-tune | |
| emits valid contract JSON unprompted. `marquee serve` runs this check at startup. | |
| ## Data | |
| The catalog is built entirely from **Wikidata** (CC0) and **Wikipedia** (plot | |
| summaries, embedded then discarded -- never redistributed as text, so CC BY-SA's | |
| share-alike terms are never triggered). No TMDB data is shipped or was used in | |
| training. | |
| **Not used:** TMDB (API terms prohibit ML training and third-party redistribution of | |
| cached content), IMDb (non-transferable, non-sublicensable licence), Rotten Tomatoes | |
| (no public API). | |
| ## Licence | |
| Apache-2.0, inherited from Qwen3-4B-Instruct-2507. Catalog: CC0. | |