gector-bert-fr / language-rules.json

gector-bert-fr 1.0.0: first public release (Apache-2.0, base Geotrend/bert-base-fr-cased), run 2026-11-07-1303

b564447 verified about 8 hours ago

988 Bytes

{
  "version": 1,
  "language": "fr",
  "_comment": "Per-bundle language facts (see C# LanguageRules / ADR-004). stripTrailingTerminator true: the FR bundle follows the same BERT/WordPiece recipe as Italian (brittle to a trailing period) — validate at eval and flip if the FR bundle proves robust. clauseConjunctions: connectors after a comma = independent-clause boundary for GectorAnalyzer.Correct's clause split (see the EN/IT files). 'parce'/'puisque' are the single-token heads of 'parce que'/'puisque' (the splitter matches one word). 'or' is the French conjunction. No elision block: French real elisions (l', d', j', qu') must NOT be treated as glued-article typos; leave the tokenizer default alone. Validate against lang/fr/gold comma_context at eval.",
  "stripTrailingTerminator": true,
  "clauseConjunctions": ["mais", "et", "ou", "car", "donc", "or", "puis", "alors", "ensuite", "pourtant", "cependant", "toutefois", "néanmoins", "sinon", "tandis", "parce", "puisque"]
}

	{
	"version": 1,
	"language": "fr",
	"_comment": "Per-bundle language facts (see C# LanguageRules / ADR-004). stripTrailingTerminator true: the FR bundle follows the same BERT/WordPiece recipe as Italian (brittle to a trailing period) — validate at eval and flip if the FR bundle proves robust. clauseConjunctions: connectors after a comma = independent-clause boundary for GectorAnalyzer.Correct's clause split (see the EN/IT files). 'parce'/'puisque' are the single-token heads of 'parce que'/'puisque' (the splitter matches one word). 'or' is the French conjunction. No elision block: French real elisions (l', d', j', qu') must NOT be treated as glued-article typos; leave the tokenizer default alone. Validate against lang/fr/gold comma_context at eval.",
	"stripTrailingTerminator": true,
	"clauseConjunctions": ["mais", "et", "ou", "car", "donc", "or", "puis", "alors", "ensuite", "pourtant", "cependant", "toutefois", "néanmoins", "sinon", "tandis", "parce", "puisque"]
	}