Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,52 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
tags:
|
| 3 |
+
- filename-parsing
|
| 4 |
+
- media-server
|
| 5 |
+
- token-classification
|
| 6 |
+
license: mit
|
| 7 |
+
---
|
| 8 |
+
|
| 9 |
+
# lain-parser
|
| 10 |
+
|
| 11 |
+
Filename parser models for [Lain](https://github.com/enrell/lain), a self-hosted
|
| 12 |
+
media server. This repo is the distribution channel for the `lain-parser`
|
| 13 |
+
service: an immutable store of encoder snapshots and exported model artifacts,
|
| 14 |
+
plus the installer.
|
| 15 |
+
|
| 16 |
+
## Layout
|
| 17 |
+
|
| 18 |
+
- `manifest.json` — the release channel. `default` names the model new installs
|
| 19 |
+
get; every entry carries sha256s, the encoder snapshot it pairs with, the
|
| 20 |
+
decoder window and the eval summary.
|
| 21 |
+
- `models/<name>/` — one directory per released artifact: `model.int8.pt`
|
| 22 |
+
(service artifact), `config.json`, `model.fpgw` + `model.fpgw.json` (the Go
|
| 23 |
+
port format). Directories are immutable — an update adds a new directory and
|
| 24 |
+
moves the `default` pointer, never rewrites a released file.
|
| 25 |
+
- `snapshots/<name>/` — the character-encoder snapshot a model was trained on
|
| 26 |
+
(`anifilebert-dapt512-long`: AniFileBERT + 40 epochs of continued masked-LM on
|
| 27 |
+
an unlabeled filename corpus).
|
| 28 |
+
- `install.sh`, `requirements.txt`, `app/` — the service installer and the
|
| 29 |
+
vendored inference subset.
|
| 30 |
+
|
| 31 |
+
## Install / update
|
| 32 |
+
|
| 33 |
+
```sh
|
| 34 |
+
hf download Enrell/lain-parser --include "install.sh requirements.txt app/**" \
|
| 35 |
+
--local-dir lain-parser
|
| 36 |
+
bash lain-parser/install.sh # installs or updates to manifest default
|
| 37 |
+
bash lain-parser/install.sh --model e5-s2 # pin a specific version
|
| 38 |
+
```
|
| 39 |
+
|
| 40 |
+
The installer downloads the pinned snapshot + model, verifies every file's
|
| 41 |
+
sha256 against the manifest, test-parses before touching the running unit, then
|
| 42 |
+
restarts `lain-parser.service`. Old artifacts are left in place for rollback.
|
| 43 |
+
Source and full provenance:
|
| 44 |
+
[filename-parser](https://github.com/enrell/filename-parser) `packaging/lain-parser/`.
|
| 45 |
+
|
| 46 |
+
## Current model
|
| 47 |
+
|
| 48 |
+
`e5-s2` — span-head parser over a DAPT-512-long AniFileBERT encoder with Unicode
|
| 49 |
+
class features and a semi-CRF loss (28.5M params, 38 MiB int8). Development-grade
|
| 50 |
+
evaluation: 137/196 exact on a hash-locked real-name suite (+6.5 pp over the
|
| 51 |
+
previous recipe); 2 false emissions on 30 gold-abstain rows; ~54 ms/row on a
|
| 52 |
+
2-core CPU.
|