| # juurio — API & CLI contract |
|
|
| Open-vocabulary Finnish morphological transforms over raw text. You hand it a sentence and an |
| operation (`past`, `passive`, `plural`, …); it parses, transforms the finite main clause, and |
| returns the result or an explicit abstention. The differentiator is **OOV/loan words**: words the |
| exception lexicon does not cover (`striimata`, `tubettaja`) are inflected by small neural realizers. |
|
|
| ## Architecture (the "split thesis") |
|
|
| Three components, each owning what it is best at: |
|
|
| | Concern | Component | |
| |---|---| |
| | structure / scope (which clause, which subject/object) | neural UD dependency parser (Giella/Stanza-free) | |
| | idiosyncrasy-free morphology (known words) | net + lexicon exception table (byte-exact to the Giella FST) | |
| | open-vocabulary morphology (OOV/loan words) | tiny lemma-anchored neural realizers | |
|
|
| The serving runtime needs only torch/numpy/transformers — no Stanza, no Giella FST, no Voikko (those |
| are used only to build the models and data). The rule engine owns all non-local structure (agreement propagation, object |
| case, subject drop); the neural realizers are called **per token**, only when the lexicon has no |
| entry for the cell. |
|
|
| ## Scope boundary (a feature, not a limitation) |
|
|
| Two entry points with deliberately different promises: |
|
|
| - **`transform(op, text)` — finite-clause transforms** that preserve (or deliberately rewrite) the |
| clause's grammar: negation, tense/mood, passive, imperative, question, number agreement, possessive |
| on the subject head. Output is a grammatical sentence or an explicit abstain. |
| - **`realize_np(np, …)` — phrase-level nominal morphology**: inflect an NP into a Case/Number/Px(+ |
| clitic). This is where *arbitrary case* lives, because case is **syntactically governed** — making |
| it a clause op would produce ungrammatical sentences (`Talo on iso` → `Talossa on iso`). |
| |
| Anything **semantic** — e.g. existential/locative framing (`Taloissani on koira`), which introduces |
| and reorders arguments — is **out of scope** for both; it would be a separate future *generator* op, |
| not a morphological transform. Keeping these apart is what lets each give a clean guarantee. |
| |
| ## Python API |
| |
| ```python |
| from juurio import Transformer, transform, SUPPORTED_OPS, OP_CATALOG, REASONS, ABSTAIN_REASONS |
| |
| t = Transformer() # loads parser + models once (heavy; reuse it) |
| r = t.transform("past", "Koira haukkuu pihalla.") |
| r.ok # True |
| r.text # 'Koira haukkui pihalla.' |
| r.reason # 'ok' |
| |
| # one-shot convenience (reuses a process-wide Transformer): |
| transform("plural", "Striimaaja nukkuu.").text # 'Striimaajat nukkuvat.' (OOV noun, neural) |
| ``` |
| |
| `import juurio` is cheap; torch + the neural parser/realizers load lazily on the first |
| `Transformer()` / `transform()` (no Stanza/Giella at serving time). |
| |
| ### `Transformer.transform(op, text, lemma=None, sent_index=0) -> Result` |
| |
| - **op** — one of `SUPPORTED_OPS` (else `ValueError`). |
| - **text** — raw text; a leading dialogue dash/quote is handled and re-prepended. |
| - **lemma** — override the parser's lemma for the main verb. Use when a loan stem is mis-lemmatized |
| (`striimaa` → parser says `striä`); a confidence warning is raised automatically otherwise. |
| - **sent_index** — which parsed sentence to operate on (default 0). |
| |
| ### `Result` |
| |
| | field | meaning | |
| |---|---| |
| | `text` | transformed sentence, or `None` if abstained | |
| | `ok` | `True` iff a form was produced | |
| | `reason` | one of `REASONS` (`"ok"` on success, else in `ABSTAIN_REASONS`) | |
| | `trace` | provenance dict (see below) — `juurio transform --trace` prints it | |
| | `warnings` | low-confidence-lemma notes + **neural realization** notices (model guess, unverified against the lexicon) | |
|
|
| `trace` fields: `op`, `root_verb`, `parser_lemma`, `lemma_source` (`override`/`parser`); and, when |
| relevant, `neural_oov` (forms the realizer produced — model guesses), `oov_unrealizable` / |
| `oov_unverified` (words it could not/did-not-trust). `realize_np` traces `util`, `case`, `number`, |
| `px`, `clitic`, `head`. Inspect on the CLI with `--trace`, or as JSON with `--json`. |
|
|
| ### Performance |
|
|
| Models load once per process; the first call also loads the neural parser (the cold cost). |
| Indicative numbers (`juurio bench`, CPU parser + GPU realizer): |
|
|
| | | time | |
| |---|---| |
| | `Transformer()` construct | ~2 s | |
| | first `transform` (cold — parser load) | ~9 s | |
| | `transform` (warm) | ~70 ms | |
| | `realize_np` (warm) | ~30 ms | |
|
|
| Reuse one `Transformer` (or the module-level `transform`/`realize_np`, which cache a singleton). |
|
|
| ## Scope contract |
|
|
| `transform` operates on a **single finite Act/Ind main clause** (≈≤12 words, one finite verb). |
| Anything else **abstains** (`ok=False`) — it never guesses. Reason codes: |
|
|
| | reason | meaning | |
| |---|---| |
| | `ok` | a transformation was produced | |
| | `parse_empty` | the parser returned no usable sentence at `sent_index` | |
| | `no_clause` | no finite Act/Ind main clause (subordinate-only, fragment, or already-negated) | |
| | `out_of_scope` | exceeds scope limits (too long / >1 finite verb / coordination) | |
| | `op_not_applicable` | op does not apply to this clause's base state (e.g. `past` on a past clause) | |
| | `oov_unrealizable` | an open-class word needed inflecting but **neither** the lexicon **nor** the neural realizer could produce the target form | |
| | `oov_unverified` | the neural realizer produced a candidate for an OOV word but it failed validation (stem-inconsistent or near-tie) and was **suppressed**, not emitted | |
| | `conflicting_features` | `inflect` got two valid but contradictory inputs (spec `mon gen` + `number="sing"`; `proper=True` + a degree; `like=` + an explicit case/number/degree; a single-word control on a phrase) | |
| | `unanalyzable_model` | `inflect(like=…)` could not analyze the model word into a nominal case/number cell to copy | |
|
|
| **Neural OOV is surfaced, never silent.** When a form is produced by the neural realizer (not the |
| lexicon), it appears in `trace["neural_unverified"]` and a `warnings` entry. Treat it as a model guess. |
|
|
| ## Operations |
|
|
| `SUPPORTED_OPS` is the flat tuple; `OP_CATALOG` groups them with descriptions. Summary: |
|
|
| - **finite verb** — `neg`, `past`, `pres`, `cond`, `perfect`, `potential`, `passive` |
| - **compound tenses** (analytic olla auxiliary + the perfect participle) — `pluperfect` |
| (`olin sanonut`), `cond_perfect` (`olisin sanonut`), `pot_perfect` (`lienen sanonut`); OOV verbs |
| reuse the trained `perfect` participle + the lexicon's `olla` auxiliary |
| - **composites** (applied atomically) — `neg+past`, `neg+cond`, `neg+perfect`, `passive+past`, |
| `passive+cond`, `passive+perfect`, `neg+passive`, `neg+passive+past`, `neg+passive+cond`, |
| `neg+passive+perfect` |
| - **clause** — `quest` (polar question) |
| - **imperative paradigm** (drop subject, total object → Nom; negative forces partitive; trailing |
| punctuation preserved) — `imperative` (Sg2, OOV-capable), `imperative_pl2/sg3/pl1/pl3`, and the |
| negative `älä`-series `neg_imperative` (Sg2, OOV-capable), `neg_imperative_pl2/sg3/pl1/pl3`. |
| **Scope:** applies only when the command's addressee is in the clause — a subjectless/already-imperative |
| source (`osta talo.`) or a 2nd-person subject (`sinä/te`); a 3rd/1st-person subject abstains |
| (`out_of_scope`), since turning *"the man buys a house"* into a command would invent an addressee. |
| - **number** (re-agrees subject NP, modifiers, finite verb, copula predicate) — `plural`, `singular` |
| - **degree** (predicate adjective) — `comp`, `super` |
| - **enclitics** (on the finite verb) — `clitic_kin`, `clitic_kaan`, `clitic_han`, `clitic_pa` |
| - **possessive suffix** on the subject head, preserving its case/number — `poss_1sg`, `poss_2sg`, |
| `poss_3`, `poss_1pl`, `poss_2pl` (`talo`→`taloni`, `talossa`→`talossani`; OOV `striimaaja`→`striimaajani`) |
|
|
| `juurio list-ops` prints the live catalog. |
|
|
| ### Combined nominal morphology |
|
|
| Nominal ops compose with `+`: a `+`-join of **nominal atoms** (`plural`/`singular`, `poss_*`, |
| `clitic_*`) is applied to the subject head as **one realization** with the final |
| Number+Case+Px, then the clitic is attached — deterministic surface order `stem+case+number+Px+clitic`. |
| This is correct-by-construction (you cannot re-pluralize the already-realized string `taloni`). |
| Number changes also re-agree modifiers and the finite verb. |
|
|
| ``` |
| poss_1sg+clitic_kin Koira haukkuu pihalla. -> Koiranikin haukkuu pihalla. |
| plural+poss_1sg Koira haukkuu pihalla. -> Koirani haukkuvat pihalla. |
| plural+poss_1sg+clitic_kin Koira haukkuu pihalla. -> Koiranikin haukkuvat pihalla. |
| plural+poss_1sg Striimaaja nukkuu. -> Striimaajani nukkuvat. (OOV head, neural) |
| ``` |
|
|
| Each dimension may appear at most once. `+`-composed **verb** ops (`neg+passive+past`) are separate, |
| pre-registered atoms in the catalog. Mixed verb+nominal composition is not supported. |
|
|
| ## Phrase-level nominal morphology: `realize_np` |
| |
| Finnish case is **syntactically governed**, so arbitrary case is *not* a clause transform (a |
| subject-case change would be ungrammatical). Instead it is a phrase utility, separate from |
| `transform`, which keeps grammaticality guarantees: |
| |
| ```python |
| from juurio import realize_np |
| realize_np("talo", case="Ine", number="Plur", px="1Sg").text # 'taloissani' |
| realize_np("iso talo", case="Ine", number="Plur").text # 'isoissa taloissa' |
| realize_np("Helsinki", case="Ine").text # 'Helsingissä' |
| realize_np("striimaaja", case="Ade", number="Plur", px="1Sg", clitic="kin").text # 'striimaajillanikin' (OOV) |
| ``` |
| |
| `realize_np(np, case, number, px, clitic)` inflects an NP — a bare lemma or a short `adj* noun` |
| phrase — into the target Case/Number with an optional possessive and enclitic, agreeing modifiers, |
| and per-token lexicon→neural fallback (surface order `stem+case+number+Px+clitic`). It returns a `Result` |
| (`ok` with the NP, or an abstain reason). This is the showcase for open-vocabulary nominal |
| morphology; `transform` is for grammaticality-preserving clause operations. Sentence-level locatives |
| (`Taloissani on koira`) are a separate future generator op, not a case transform. |
| |
| ## Phrase-level verb morphology: `realize_verbform` |
| |
| The non-finite counterpart of `realize_np`: inflect a VERB into a **participle** or **infinitive**. |
| The four participles inflect like adjectives in case+number; the 2nd/3rd infinitives are case-marked. |
| A second open-vocabulary surface — the lexicon handles known verbs, a small **verbform realizer** |
| inflects OOV/loan verbs (`striimata`), flagged `unverified`. |
| |
| ```python |
| from juurio import realize_verbform # form ∈ {va, nut, tava, tu, inf2, inf2_pass, inf3, ma} |
| realize_verbform("lukea", form="va").text # 'lukeva' (present active participle) |
| realize_verbform("lukea", form="va", case="Ine").text # 'lukevassa' |
| realize_verbform("lukea", form="tu").text # 'luettu' (past passive participle) |
| realize_verbform("mennä", form="inf3", case="Ade").text # 'menemällä' (3rd inf adessive) |
| realize_verbform("mennä", form="inf3", case="Ine").text # 'menemässä' |
| realize_verbform("tehdä", form="ma", case="Ine").text # 'tekemässä' (agent participle) |
| realize_verbform("tehdä", form="inf2_pass").text # 'tehtäessä' (passive 2nd inf, temporal) |
| realize_verbform("kuulla", form="tu", case="Par", px="3").text # 'kuultuansa' (temporal: 'after hearing') |
| realize_verbform("striimata", form="va").text # 'striimaava' (OOV verb, neural) |
| ``` |
| |
| `va`/`nut`/`tava`/`tu` = present/past active/passive participle; `inf2`/`inf3` = 2nd/3rd infinitive; |
| `inf2_pass` = passive 2nd infinitive (`-ttaessa` temporal, "while X is being done"; inessive-only, |
| case-invariant); `ma` = agent participle (-ma/-mä, "the X that someone does", inflects in case/number). |
| `px` (one of `None`/`1Sg`/`2Sg`/`3`/`1Pl`/`2Pl`) attaches a possessive suffix — participles inflect |
| like nominals, so `tu`+`Par`+`px` realizes the **temporal construction of anteriority** (`kuultuaan` / |
| the `-nsa` variant `kuultuansa` = "after hearing"). For OOV verbs the bare participle comes from the |
| verbform realizer and the case+Px from the noun realizer (both neural stages flagged unverified). |
| Returns a `Result` (`ok` with the form, or `oov_unrealizable`/`oov_unverified` when the morphology |
| engine cannot produce/trust it). Like `realize_np`, this is a phrase utility, not a clause transform. |
| |
| ## Friendly inflection: `inflect` |
| |
| The ergonomic single-word front door over `realize_np` — human-readable case/number/degree names, a |
| proper-noun toggle, and **inflect-by-example**. Returns a `Result` (never a bare string: abstention |
| stays first-class). |
| |
| ```python |
| from juurio import inflect |
| inflect("talo", "gen").text # 'talon' (alias: gen/genitive/genetiivi) |
| inflect("talo", "mon gen").text # 'talojen' (combined spec, taivutin-style) |
| inflect("talo", case="inessive", number="plural").text # 'taloissa' |
| inflect("talo", px="1sg").text # 'taloni' |
| inflect("kaunis", "gen", degree="comp").text # 'kauniimman' (degree => adjective path) |
| inflect("meri", "gen").text # 'meren' (common noun) |
| inflect("Meri", "gen", proper=True).text # proper-noun reading (≠ 'meren') |
| inflect("makaronilaatikko", like="talojen").text # 'makaronilaatikkojen' (copy a model word's cell) |
| inflect("iso talo", "mon gen").text # 'isojen talojen' (phrase -> realize_np) |
| ``` |
| |
| `inflect(word, case="nom", number="sing", *, degree="pos", px="none", clitic=None, proper=None, |
| pos=None, like=None)`: |
| |
| - **case** — a case alias, *or* a combined spec (`"mon gen"`, `"plural genitive comparative"`). |
| Accepts English abbrev/full and Finnish (`gen`/`genitive`/`genetiivi`, `mon`/`monikko`). |
| - **number / degree** — aliases too; an explicit value that contradicts the combined spec abstains |
| (`conflicting_features`). |
| - **px / clitic** — possessive suffix (`1sg`…/`None`) and enclitic, as in `realize_np`. `px=None` is |
| accepted (= no suffix). Rejected on a single adjective target. |
| - **proper** — `True`/`False` forces PROPN/NOUN; `None` (default) infers PROPN from a leading capital. |
| - **pos** — explicit `'noun'`/`'proper'`/`'adjective'` (wins over inference). PRON/NUM are not yet a |
| first-class `pos`. A degree implies the adjective path when `pos` is unset. |
| - **like** — a model word; its analyzed case (+number, +degree if adjectival) is copied onto `word`. |
| Mutually exclusive with `case`/`number`/`degree`; an unanalyzable model abstains |
| (`unanalyzable_model`). |
|
|
| An **unknown alias** raises `ValueError` (a typo fails fast); a **contradiction** abstains |
| (`conflicting_features`). A single token is realized **parser-free** (so `proper`/`pos` are honored); |
| a whitespace phrase delegates to `realize_np` (which applies `px`/`clitic` to the head but rejects the |
| single-word controls `proper`/`pos`/`degree`). An empty `word` raises `ValueError`. |
|
|
| ## Analysis: `analyze` / `lemmatize` |
|
|
| The inverse of generation (what a generate-only inflector cannot do): the serving neural parser |
| exposed directly. |
|
|
| ```python |
| from juurio import analyze, lemmatize, TokenAnalysis |
| sents = analyze("Koirat haukkuvat pihalla.") # list[list[TokenAnalysis]] (one list per sentence) |
| tok = sents[0][0] |
| tok.lemma, tok.upos, tok.feats # 'koira', 'NOUN', {'Case': 'Nom', 'Number': 'Plur'} |
| tok.asdict() # plain dict (id/form/lemma/upos/head/deprel/feats) |
| lemmatize("Koirat haukkuvat pihalla.") # ['koira', 'haukkua', 'piha', '.'] (flat) |
| ``` |
|
|
| `TokenAnalysis` is a frozen record (`id`, `form`, `lemma`, `upos`, `head` — 1-based, 0=root —, |
| `deprel`, `feats` as a read-only UD mapping). `analyze` keeps sentence boundaries; `lemmatize` |
| returns a flat lemma list. |
|
|
| ## CLI |
|
|
| Each library function has a like-named command (the CLI is a thin wrapper over the same |
| `Transformer`); `realize_np`/`realize_verbform` keep the hyphenated spelling. |
|
|
| ``` |
| # one command per library function |
| juurio transform <op> "<text>" [--lemma L] [--trace] [--json] # transform() |
| juurio inflect "<word>" [spec…] [--case C] [--number N] [--degree D] [--px P] [--clitic K] [--pos P] [--like W] [--proper] [--json] # inflect() |
| juurio analyze "<text>" [--json] # analyze() |
| juurio lemmatize "<text>" [--json] # lemmatize() |
| juurio realize-np "<np>" [--case Ine] [--number Plur] [--px 1Sg] [--clitic kin] [--json] # realize_np() |
| juurio realize-verbform "<verb>" [--form va|nut|tava|tu|inf2|inf2_pass|inf3|ma] [--case Ine] [--number Plur] [--px 3] # realize_verbform() |
| |
| # infrastructure (no library counterpart) |
| juurio list-ops |
| juurio batch <op> [--lemma L] [--json] < lines # one op for every stdin line |
| juurio batch [--json] < tsv # per-line "op<TAB>text[<TAB>lemma]" |
| juurio demo # curated tour (incl. OOV) — also a smoke test |
| juurio repl # interactive shell; model loaded once |
| ``` |
|
|
| `inflect` accepts a human case/number/degree phrase positionally (`juurio inflect talo mon gen` |
| → `talojen`, `juurio inflect kaunis gen --degree comp`), or as `--case/--number/--degree` flags; |
| a positional phrase wins over the flags. `--proper`/`--no-proper` force/forbid a proper-noun reading. |
|
|
| `juurio repl` is a read-eval loop over every feature (model loaded once): `inflect talo mon gen`, |
| `inflect makaronilaatikko like=talojen`, `transform passive Mies osti talon.`, `analyze …`, |
| `lemmatize …`, `np …`, `verb …`, `ops`, `help`. Options are `key=value` (`px=1sg proper=true |
| pos=adjective clitic=kin like=talojen`). |
|
|
| Also `python -m juurio …`. Without `--json`, prints the result (warnings appended after `⚠`) and |
| sends abstentions to stderr as `[abstain: <reason>] <input>`. With `--json`, one JSON object per |
| line (`text/ok/reason/warnings/trace`). **Exit status:** `0` if every transform produced a form, |
| `2` if any abstained — so scripts can branch on it. |
|
|
| ```console |
| $ juurio transform plural "Striimaaja nukkuu." |
| Striimaajat nukkuvat. ⚠ neural realization (unverified by FST): ['striimaaja->Striimaajat (NOUN)'] |
| ``` |
|
|
| ## Quality |
|
|
| The package is designed to **abstain rather than guess**: out-of-scope inputs return an explicit |
| abstention (`ok=False`) with a reason code instead of a speculative form, and neural OOV realizations |
| are always surfaced (in `trace["neural_unverified"]` and `warnings`) rather than emitted silently. Known |
| words are inflected to be byte-exact to the reference FST (through the exception lexicon); OOV/loan words are inflected by the neural |
| realizers, with stem-inconsistent or near-tie candidates suppressed (`oov_unverified`) rather than |
| returned. |
|
|
| Run `juurio demo` for a curated tour (including OOV cases) that doubles as a smoke test. |
|
|
