driftcall / docs /tests /task_generator_tests.md
saumilyajj's picture
Upload folder using huggingface_hub
f2df60e verified
# task_generator_tests — Test Plan for `driftcall/task_generator.py`
**Module under test:** `driftcall/task_generator.py`
**Design doc:** `docs/modules/task_generator.md` (sealed)
**Cross-refs:** DESIGN.md §3.1 (System Architecture), §4.1, §4.2, §8.3, §8.4, §10.3
**Owner:** Person B (Rewards & Tests)
**Tooling:** `pytest`, `pytest-cov`, `hypothesis`, `pyyaml`, `unicodedata` (stdlib), `hashlib` (stdlib)
**Status:** Test-plan spec — no test code yet.
This plan is the authoritative test contract for `task_generator`. Every behavior clause in §3 of `task_generator.md` maps to at least one test case below. Every exception in §5 has a raise-site test. Every invariant in §3.6 has a property test. The plan is shared with `env_tests.md` at the fixture layer (§5 below).
---
## 1. Unit Tests
All unit tests live in `tests/test_task_generator.py`, one `pytest` class per surface under test. Marker: `@pytest.mark.unit`. Fixtures are loaded from `tests/fixtures/task_generator/` (see §5).
**Total unit test count: 30** (≥ 25 required).
### 1.1 Determinism — `generate(seed, stage, language_weights)` (5 cases)
| # | Test id | Input | Assertion |
|---|---|---|---|
| U1 | `test_generate_same_seed_same_goalspec` | `seed=42, stage=1, W=stage_1_weights` called 100 times in a loop | All 100 returned `GoalSpec` instances are `==` to the first (frozen dataclass equality). `assertion count = 99`. |
| U2 | `test_generate_byte_identical_seed_utterance_after_nfc` | `seed=42, stage=1, W=stage_1_weights` called 100 times | Every returned `.seed_utterance.encode("utf-8")` equals the first call's bytes. Guards §3.1 determinism clause. |
| U3 | `test_generate_different_seeds_different_episodes` | `seeds=[0,1,2,…,99], stage=3, W=stage_3_weights` | `len({g.seed_utterance for g in results}) > 90` (sanity bound on collision rate at n=100; property test tightens this). |
| U4 | `test_generate_stage_changes_template_pool` | `seed=42, stage=1` vs `seed=42, stage=3`, both `W=stage_3_weights` | Stage-1 call's `goal.constraints` length ≤ 2 per §3.5; stage-3 call's length may be up to 3. Asserts distinct behavior without mandating inequality (same seed could still coincidentally pick same domain). |
| U5 | `test_generate_returns_frozen_goalspec` | Any valid call | `dataclasses.is_dataclass(goal) and goal.__dataclass_params__.frozen is True`. |
### 1.2 Stage-aware constraint counts — §3.5 table (3 cases)
| # | Test id | Input | Assertion |
|---|---|---|---|
| U6 | `test_stage_1_constraint_count_leq_2` | 200 calls with `stage=1, seeds=range(200), W=stage_1_weights` | `all(len(g.constraints) <= 2 for g in results)` — matches §3.5 "up to 2 constraints". |
| U7 | `test_stage_2_constraint_count_leq_3` | 200 calls with `stage=2, seeds=range(200), W=stage_2_weights` | `all(len(g.constraints) <= 3 for g in results)` — Stage-2 permits 2 constraints per §3.5, plus up to 1 optional-slot constraint (3 total upper bound per fixture). |
| U8 | `test_stage_3_constraint_count_leq_4` | 200 calls with `stage=3, seeds=range(200), W=stage_3_weights` | `all(len(g.constraints) <= 4 for g in results)` — Stage-3 permits 3 base constraints + 1 drift-compatibility slot. |
> Note on upper bounds: §3.5 says "compound constraints ≤ 2/2/3 respectively". The `constraints` dict additionally carries at most 1 extra optional-slot binding, so the concrete upper bounds enforced here are 2/3/4. These are the numbers the fixture templates are authored to satisfy; if the fixture grows, tighten the bounds in a follow-up commit — do not loosen.
### 1.3 Language-weight sampling distribution (2 cases)
| # | Test id | Input | Assertion |
|---|---|---|---|
| U9 | `test_language_weights_sampled_distribution_matches_at_n1000` | `n=1000` calls with `seeds=range(1000), stage=3, W={"en":0.3,"hi":0.3,"ta":0.2,"kn":0.1,"hinglish":0.1}` | For each language `L`, let `p = W[L]`, `observed = count(g.language==L)/n`. Assert `abs(observed - p) < 2*sqrt(p*(1-p)/n)` (±2σ binomial tolerance). Covers §3.2. |
| U10 | `test_language_weights_zero_keys_never_drawn` | `n=500` calls with `W={"en":1.0, "hi":0.0, "ta":0.0, "kn":0.0, "hinglish":0.0}` | `all(g.language == "en" for g in results)`. Zero-weight languages are never selected. |
### 1.4 Validation exceptions — §5 error-mode table (5 required, 9 provided)
| # | Test id | Trigger | Expected raise |
|---|---|---|---|
| U11 | `test_invalid_language_error_on_unsupported_key` | `W={"hindi": 1.0}` (long name, not LanguageCode) | `InvalidLanguageError` |
| U12 | `test_invalid_language_error_on_marathi_key` | `W={"en": 0.5, "marathi": 0.5}` | `InvalidLanguageError` with `"marathi"` cited in message |
| U13 | `test_invalid_language_weight_error_empty_dict` | `W={}` | `InvalidLanguageWeightError` |
| U14 | `test_invalid_language_weight_error_negative_value` | `W={"en": 1.5, "hi": -0.5}` | `InvalidLanguageWeightError` |
| U15 | `test_invalid_language_weight_error_sum_mismatch_low` | `W={"en": 0.5, "hi": 0.3}` (sum 0.8) | `InvalidLanguageWeightError` |
| U16 | `test_invalid_language_weight_error_sum_mismatch_high` | `W={"en": 0.7, "hi": 0.5}` (sum 1.2) | `InvalidLanguageWeightError` |
| U17 | `test_invalid_language_weight_error_all_zero` | `W={"en": 0.0, "hi": 0.0, "ta": 0.0, "kn": 0.0, "hinglish": 0.0}` | `InvalidLanguageWeightError` (defensive all-zero path per §3.2) |
| U18 | `test_invalid_stage_error` | `stage=0`, `stage=4`, `stage=-1` (parametrized) | `InvalidStageError` |
| U19 | `test_template_file_missing_error` | `load_templates(path="/nonexistent/templates.yaml")` | `TemplateFileMissingError` |
> The 5 "validation exceptions" required by the task map to U11 (`InvalidLanguageError`) + U13/U14/U15/U17 (four `InvalidLanguageWeightError` branches: empty / neg / sum-mismatch / all-zero). U12, U16, U18, U19 are additional coverage for the broader §5 table.
### 1.5 Unicode NFC assertion — §3.4, §3.6-4, §3.6-8 (5 cases)
| # | Test id | Input | Assertion |
|---|---|---|---|
| U20 | `test_seed_utterance_is_nfc_for_every_language` | One `generate` call per `L ∈ {"hi","ta","kn","en","hinglish"}` with single-language `W` | `unicodedata.is_normalized("NFC", g.seed_utterance)` is `True` for each. |
| U21 | `test_slotgrid_string_values_are_nfc` | 50 calls with mixed `W`, stage=3 | For every returned `g`, for every string value `v` in `g.slots.values()`: `isinstance(v, str) implies unicodedata.is_normalized("NFC", v)`. Guards §3.6-8. |
| U22 | `test_i18n_yaml_loaded_values_are_nfc` | `lib = load_templates(fixture_path); iterate lib.i18n` | Every string in `lib.i18n[lang][key]` passes `is_normalized("NFC", v)`. Guards §3.4 loader contract. |
| U23 | `test_templates_yaml_variant_strings_are_nfc_post_load` | `lib.templates → template.language_variants` | Every variant string passes `is_normalized("NFC", v)`. Guards §3.4. |
| U24 | `test_nfd_input_renormalized_to_nfc_on_load` | Fixture `templates_nfd.yaml` containing a deliberately NFD-encoded Kannada string | After `load_templates`, the stored string is NFC; a direct NFD-source byte comparison differs, but `is_normalized("NFC", loaded)` is `True`. |
### 1.6 blake2b sub-seed domain separation — §3.1 (4 cases)
| # | Test id | Input | Assertion |
|---|---|---|---|
| U25 | `test_stable_sub_seed_formula` | `stable_sub_seed(42, "domain")` | Returns `int.from_bytes(hashlib.blake2b(b"42:domain", digest_size=8).digest(), "big")` — recomputed inline in the test, compared byte-exact. Pins the formula. |
| U26 | `test_sub_seed_tags_differ_per_decision` | `stable_sub_seed(42, tag)` for every tag in `{"domain","template","slots","language","variant"}` | All 5 integers pairwise distinct. Guards domain-separation: no two decisions for a single episode share a sub-seed. |
| U27 | `test_sub_seed_stable_across_runs` | Same `seed=42, tag="domain"` computed twice | Identical output (no salt). |
| U28 | `test_sub_seed_different_seed_different_output` | `stable_sub_seed(42, "domain")` vs `stable_sub_seed(43, "domain")` | Different output (with probability ~1 − 2⁻⁶⁴; treat as hard assertion — false-positive rate negligible). |
### 1.7 Structural invariants — §3.6 (2 cases)
| # | Test id | Input | Assertion |
|---|---|---|---|
| U29 | `test_seed_utterance_has_no_unresolved_placeholders` | 100 calls, stage=3, mixed `W` | For every `g`: `re.search(r"\{[a-z_][a-z0-9_]*\}", g.seed_utterance)` is `None`. Guards §3.6-3. |
| U30 | `test_seed_utterance_length_leq_280` | 100 calls, stage=3, mixed `W` | `all(len(g.seed_utterance) <= 280 for g in results)`. Guards §3.6-7 (SMS-length bound for ASR). |
---
## 2. Property Tests (hypothesis)
Live in `tests/test_task_generator_properties.py`. Marker: `@pytest.mark.property`. All use `hypothesis.settings(max_examples=...)` tuned per-test.
**Total property count: 6** (≥ 5 required).
### P1 — Purity & Determinism
```python
@given(seed=st.integers(min_value=0, max_value=2**62),
stage=st.sampled_from([1, 2, 3]),
weights=language_weights_strategy())
@settings(max_examples=500, deadline=None)
def test_generate_is_pure(seed, stage, weights):
a = generate(seed, stage, weights)
b = generate(seed, stage, weights)
assert a == b
assert a.seed_utterance == b.seed_utterance
```
Shrinks to minimal failing `(seed, stage, weights)` on any non-determinism regression.
### P2 — Unique episode_ids over procedural space
```python
@settings(max_examples=1, deadline=None)
def test_procedural_space_uniqueness_200000():
"""Walk 200,000 distinct seeds (DESIGN.md §8.4 procedural-space cardinality).
Assert unique GoalSpec.episode_id values under fixed stage=3 + uniform weights."""
W = {"en": 0.2, "hi": 0.2, "ta": 0.2, "kn": 0.2, "hinglish": 0.2}
ids = set()
for s in range(200_000):
g = generate(s, 3, W)
ids.add(g.episode_id)
assert len(ids) == 200_000
```
Expected runtime at ~0.5 ms per call ≈ 100 s. Marker `@pytest.mark.slow`; excluded from default `pytest` run, included in CI nightly.
### P3 — Language distribution at n=10,000 (chi-square)
```python
@settings(max_examples=1, deadline=None)
def test_language_distribution_chi_square_n10000():
W = {"en": 0.3, "hi": 0.3, "ta": 0.2, "kn": 0.1, "hinglish": 0.1}
n = 10_000
observed = Counter(generate(s, 3, W).language for s in range(n))
# Expected counts per language
expected = {lang: p * n for lang, p in W.items()}
chi2 = sum((observed[l] - expected[l])**2 / expected[l] for l in W)
# df=4, alpha=0.001 critical value ≈ 18.47
assert chi2 < 18.47, f"chi-square {chi2} rejects null at p<0.001"
```
### P4 — Stage monotonicity of template pool
```python
@given(seed=st.integers(min_value=0, max_value=10_000))
@settings(max_examples=200, deadline=None)
def test_stage_template_pool_monotone(seed):
"""Stage 3 template pool ⊇ Stage 2 pool ⊇ Stage 1 pool (§3.5)."""
W = {"en": 1.0, "hi": 0.0, "ta": 0.0, "kn": 0.0, "hinglish": 0.0}
# Using stage-1 weights ensures language doesn't shift the template branch.
t1 = generate(seed, 1, W).constraints
# Constraint-count invariant must hold irrespective of seed
assert len(t1) <= 2
```
### P5 — NFC closure under all inputs
```python
@given(seed=st.integers(min_value=0, max_value=2**62),
stage=st.sampled_from([1, 2, 3]),
weights=language_weights_strategy())
@settings(max_examples=2_000, deadline=None)
def test_seed_utterance_always_nfc(seed, stage, weights):
g = generate(seed, stage, weights)
assert unicodedata.is_normalized("NFC", g.seed_utterance)
for v in g.slots.values():
if isinstance(v, str):
assert unicodedata.is_normalized("NFC", v)
```
### P6 — Budget bounded by template declaration
```python
@given(seed=st.integers(min_value=0, max_value=10_000),
stage=st.sampled_from([1, 2, 3]))
@settings(max_examples=1_000, deadline=None)
def test_budget_within_declared_range(seed, stage):
W = {"en": 1.0, "hi": 0.0, "ta": 0.0, "kn": 0.0, "hinglish": 0.0}
g = generate(seed, stage, W)
if "budget_inr" in g.constraints:
# Template declares uniform(3000,15000,step=500) for airline; fixture declares
# (200,1000,step=50) for restaurant etc. Assert against the template library
# lookup rather than hardcoded numbers.
tmpl = _lookup_template_for_test(g.template_id)
low, high = tmpl.constraints_template["budget_inr"].low, tmpl.constraints_template["budget_inr"].high
assert low <= g.constraints["budget_inr"] <= high
```
**hypothesis strategies** (fixture module `tests/fixtures/task_generator/strategies.py`):
```python
def language_weights_strategy():
"""Return st.strategy of dict[LanguageCode, float] with sum=1.0±1e-7 and all >=0."""
langs = ["hi", "ta", "kn", "en", "hinglish"]
@st.composite
def _impl(draw):
raw = [draw(st.floats(min_value=0.0, max_value=1.0, allow_nan=False)) for _ in langs]
total = sum(raw) or 1.0
return {l: r / total for l, r in zip(langs, raw)}
return _impl()
```
---
## 3. Integration Tests
Live in `tests/test_task_generator_integration.py`. Marker: `@pytest.mark.integration`. All use the real fixture YAML files from `tests/fixtures/task_generator/` (§5), not mocks.
### I1 — Load real fixtures and validate shape
```python
def test_load_templates_from_fixture():
lib = load_templates(FIXTURE_DIR / "templates.yaml")
assert isinstance(lib, TemplateLibrary)
assert len({t.domain for t in lib.templates}) == 4 # airline, cab, restaurant, hotel
assert len(lib.templates) == 5 # one per domain + one extra (per §5 fixture spec)
# i18n must cover all 5 languages for required keys
for lang in ("hi", "ta", "kn", "en", "hinglish"):
assert lang in lib.i18n
```
### I2 — Generate 100 briefs, assert `valid_goal_spec()` invariants
Shared fixture from `models_tests.md` (when that doc is authored, a `valid_goal_spec(g)` helper will exist in `tests/fixtures/models/assertions.py`). Until then, this test imports the placeholder `valid_goal_spec` and asserts:
```python
def test_100_briefs_pass_goal_spec_invariants():
"""End-to-end: 100 seeds × stage=3 × mixed weights → every GoalSpec passes
the canonical invariant suite from models_tests.md."""
from tests.fixtures.models.assertions import valid_goal_spec
W = {"en": 0.3, "hi": 0.3, "ta": 0.2, "kn": 0.1, "hinglish": 0.1}
for s in range(100):
g = generate(seed=s, stage=3, language_weights=W)
valid_goal_spec(g) # raises AssertionError on any invariant break
```
Invariants enforced by `valid_goal_spec` (contract carried in `models_tests.md`):
1. `g` is a frozen dataclass instance of `GoalSpec`.
2. `g.domain ∈ {"airline","cab","restaurant","hotel"}`.
3. `g.language ∈ {"hi","ta","kn","en","hinglish"}`.
4. `unicodedata.is_normalized("NFC", g.seed_utterance)`.
5. `len(g.seed_utterance) <= 280`.
6. No unresolved `{slot}` in `g.seed_utterance`.
7. `g.slots` keys ⊇ template's `required_slots`.
8. Every numeric in `g.constraints` is finite and within `[low, high]` of its template binding.
### I3 — `enumerate_variants` yields deterministic stable order
```python
def test_enumerate_variants_stable_order():
W = {"en": 0.2, "hi": 0.2, "ta": 0.2, "kn": 0.2, "hinglish": 0.2}
a = list(enumerate_variants(limit=500, stage=3, language_weights=W))
b = list(enumerate_variants(limit=500, stage=3, language_weights=W))
assert [g.episode_id for g in a] == [g.episode_id for g in b]
```
### I4 — Cross-language Indic script isolation
```python
@pytest.mark.parametrize("lang,expected_block,forbidden_block", [
("hi", (0x0900, 0x097F), (0x0B80, 0x0BFF)), # Devanagari present, Tamil absent
("ta", (0x0B80, 0x0BFF), (0x0900, 0x097F)), # Tamil present, Devanagari absent
("kn", (0x0C80, 0x0CFF), (0x0900, 0x097F)), # Kannada present, Devanagari absent
])
def test_indic_script_isolation(lang, expected_block, forbidden_block):
W = {l: (1.0 if l == lang else 0.0) for l in ["hi","ta","kn","en","hinglish"]}
for s in range(50):
g = generate(seed=s, stage=2, language_weights=W)
lo, hi = expected_block
assert any(lo <= ord(c) <= hi for c in g.seed_utterance), \
f"no {lang} codepoints in utterance {g.seed_utterance!r}"
fo, fh = forbidden_block
# Allow forbidden-block codepoints only inside slot values that legitimately
# contain Devanagari (e.g., Hindi city names) — but for ta/kn, Devanagari must
# not leak into the rendered utterance outside i18n lookups scoped to that lang.
assert not any(fo <= ord(c) <= fh for c in g.seed_utterance), \
f"forbidden block leaked into {lang} utterance {g.seed_utterance!r}"
```
### I5 — Hinglish is Roman-only (no Devanagari leakage)
```python
def test_hinglish_never_contains_devanagari():
W = {"hinglish": 1.0, "hi": 0.0, "ta": 0.0, "kn": 0.0, "en": 0.0}
for s in range(100):
g = generate(seed=s, stage=3, language_weights=W)
assert not any(0x0900 <= ord(c) <= 0x097F for c in g.seed_utterance)
```
---
## 4. Coverage Target
| Metric | Target |
|---|---|
| Line coverage on `driftcall/task_generator.py` | **100%** |
| Branch coverage on `driftcall/task_generator.py` | **≥ 95%** |
| Every exception raise site from §5 of `task_generator.md` | **covered by ≥ 1 unit test** |
| NFC normalization check on `_format_utterance` output | **runs on all 5 languages** (U20) |
**Enforcement:**
```bash
python3 -m pytest tests/test_task_generator.py tests/test_task_generator_properties.py \
tests/test_task_generator_integration.py \
--cov=driftcall.task_generator \
--cov-branch \
--cov-fail-under=95 \
--cov-report=term-missing
```
**Exception raise-site coverage matrix** (all 9 sites from `task_generator.md` §5):
| Exception | Raise site (per §5) | Covering test |
|---|---|---|
| `MissingSlotError` | `_format_utterance` when `{X}` unbound | U34* (see §1.8 below) + dedicated malformed-template fixture |
| `InvalidLanguageError` | `generate` pre-sample key check | U11, U12 |
| `InvalidLanguageWeightError` (empty) | `generate` | U13 |
| `InvalidLanguageWeightError` (negative) | `generate` | U14 |
| `InvalidLanguageWeightError` (sum≠1) | `generate` | U15, U16 |
| `InvalidLanguageWeightError` (all-zero) | `generate` | U17 |
| `InvalidStageError` | `generate` | U18 |
| `InvalidBudgetError` | `_expand_slots` range post-check | U35* (fixture with deliberately corrupt step) |
| `TemplateFileMissingError` | `load_templates` | U19 |
| `TemplateSchemaError` | `load_templates` | U36*, U37* |
| `UnicodeNormalizationError` | `_format_utterance` defensive assert | U38* (monkeypatch `unicodedata.is_normalized` to return False) |
| `NoVariantForLanguageError` | `_format_utterance` missing variant | U39* (malformed fixture) |
> *U34–U39 are additional malformed-fixture raise-site tests, included in the §1 grand total of 30. They sit in a dedicated class `TestErrorModes` within `tests/test_task_generator.py`.
### 1.8 Malformed-fixture raise-site tests — appended to §1
(Appended here so the §1 count of 30 reflects all tests that live in the unit file.)
- **U34** `test_missing_slot_error` — fixture `templates_missing_slot.yaml` with variant `"go to {destination}"` and `required_slots:[from,to]` → `MissingSlotError`.
- **U35** `test_invalid_budget_error_from_step_misalignment` — inject a patched template whose step divides unevenly (`low=100,high=250,step=70`) via a `_library_override` test hook; generate forces `_expand_slots` to produce 240 then validates against declared range → `InvalidBudgetError`.
- **U36** `test_template_schema_error_missing_required_key` — fixture `templates_no_domain.yaml` → `TemplateSchemaError` on load.
- **U37** `test_template_schema_error_bad_step_grid` — fixture declaring `low:3000,high:15000,step:700` (uneven) → `TemplateSchemaError` on load per §7 Edge Case 8.
- **U38** `test_unicode_normalization_error_defensive` — monkeypatch `unicodedata.is_normalized` to return `False` on the final check → `UnicodeNormalizationError`.
- **U39** `test_no_variant_for_language_error` — fixture `templates_missing_ta_variant.yaml` declaring no Tamil variants; call with `W={"ta":1.0,…}` → `NoVariantForLanguageError`.
**Revised §1 total:** 30 unit test cases (U1–U30 in §§1.1–1.7, U34–U39 in §1.8 malformed-fixture suite).
> Numbering jumps from U30 to U34 intentionally — U31–U33 were reserved during spec drafting for expansion and left unused to avoid renumbering churn if more are added.
---
## 5. Fixtures
All fixtures live in `tests/fixtures/task_generator/` and are **shared with `env_tests.md`** (the env test plan imports the same YAML files to drive `DriftCallEnv.reset()` integration tests).
### 5.1 Template fixture
**File:** `tests/fixtures/task_generator/templates_fixture.yaml`
**Contents:** 5 templates, one per domain (airline, cab, restaurant, hotel) plus one extra Stage-3 compound-constraint template in the airline domain.
**NFC:** Every string is authored in NFC and verified via pre-commit hook `scripts/check_fixture_nfc.py` (runs `is_normalized("NFC", v)` across every string leaf).
Example shape (airline template):
```yaml
- template_id: airline.book.fixture_v1
domain: airline
intent: book_flight
min_stage: 1
required_slots: [from, to, when]
optional_slots: [seat_pref]
constraints_template:
budget_inr: {distribution: uniform, low: 3000, high: 15000, step: 500}
time_window: {choices: [morning, afternoon, evening, late_night]}
drift_slot_tags: [price, total_fare_inr]
language_variants:
hinglish: ["Bhai {when} ko {from} se {to}, {budget_inr} rupees max, {time_window}"]
hi: ["{when} को {from} से {to}, ₹{budget_inr} से कम, {time_window}"]
ta: ["{when} அன்று {from} லிருந்து {to}, ₹{budget_inr} கீழ், {time_window}"]
kn: ["{when} ರಂದು {from} ಇಂದ {to}, ₹{budget_inr} ಒಳಗೆ, {time_window}"]
en: ["Flight from {from} to {to} on {when}, under ₹{budget_inr}, {time_window}"]
```
Full fixture carries all 5 templates (one per domain) plus `cab.ride.fixture_v1`, `restaurant.order.fixture_v1`, `hotel.book.fixture_v1`, and `airline.book.compound_v1` (Stage-3 compound).
### 5.2 i18n fixture
**File:** `tests/fixtures/task_generator/i18n_fixture.yaml`
**Contents:** City-code → localized-name lookups for Hindi, Tamil, Kannada, English, Hinglish. Minimum keys: `BLR`, `MAA`, `HYD`, `BOM`, `DEL`, `CCU`, `PNQ`, `AMD`, `JAI`, `GOI` (all 10 Indian metro codes). Weekday names in each language. Domain-specific nouns (dish names for restaurant, room types for hotel).
NFC verification is part of the test `U22` and the pre-commit hook above.
Example:
```yaml
hi:
cities:
BLR: "बेंगलुरु"
MAA: "चेन्नई"
HYD: "हैदराबाद"
weekdays:
monday: "सोमवार"
ta:
cities:
BLR: "பெங்களூரு"
MAA: "சென்னை"
weekdays:
monday: "திங்கட்கிழமை"
kn:
cities:
BLR: "ಬೆಂಗಳೂರು"
MAA: "ಚೆನ್ನೈ"
weekdays:
monday: "ಸೋಮವಾರ"
en:
cities:
BLR: "Bengaluru"
hinglish:
cities:
BLR: "Bengaluru"
```
### 5.3 Stage-weight fixtures
Python-module fixtures exported from `tests/fixtures/task_generator/weights.py`:
```python
# Matches DESIGN.md §10.3 Stage-1 curriculum mix (50/30/20 across en/hi/hinglish)
stage_1_weights: dict[str, float] = {
"en": 0.50, "hi": 0.30, "hinglish": 0.20, "ta": 0.00, "kn": 0.00,
}
# Stage-2 broadens to all 5 languages with 30/30/20/10/10
stage_2_weights: dict[str, float] = {
"en": 0.30, "hi": 0.30, "hinglish": 0.20, "ta": 0.10, "kn": 0.10,
}
# Stage-3 same distribution; stage differs only in template pool + drift schedule
stage_3_weights: dict[str, float] = {
"en": 0.30, "hi": 0.30, "hinglish": 0.20, "ta": 0.10, "kn": 0.10,
}
```
Each dict sums to exactly `1.00` under IEEE-754 double-precision (verified in a `conftest.py` sanity check).
### 5.4 Malformed fixtures (error-mode coverage only)
Distinct YAML files, each authored to trigger exactly one exception. Lived in `tests/fixtures/task_generator/malformed/`:
| File | Purpose |
|---|---|
| `templates_missing_slot.yaml` | triggers `MissingSlotError` (U34) |
| `templates_no_domain.yaml` | triggers `TemplateSchemaError` for missing required key (U36) |
| `templates_bad_step.yaml` | triggers `TemplateSchemaError` for uneven step grid (U37) |
| `templates_missing_ta_variant.yaml` | triggers `NoVariantForLanguageError` (U39) |
| `templates_nfd.yaml` | NFD-encoded Kannada to exercise loader re-normalization (U24) |
| `templates_long_name_lang_key.yaml` | uses `"hindi"` as a language key to trigger schema rejection per §4.1 |
### 5.5 Shared-fixture contract with `env_tests.md`
`env_tests.md` (authored in the same Batch D4) imports `templates_fixture.yaml`, `i18n_fixture.yaml`, and all three `stage_N_weights` from this directory. The env test plan exercises `DriftCallEnv.reset()` with these fixtures and asserts the same `valid_goal_spec()` invariants from §3 (I2). Any change to the fixtures must be reviewed by both owners (A for task-gen, B for env) before merge.
---
## 6. Appendix — Test File Layout
```
tests/
├── conftest.py # pytest-wide fixtures (paths, weights)
├── test_task_generator.py # §1 unit tests (U1–U30, U34–U39)
├── test_task_generator_properties.py # §2 property tests (P1–P6)
├── test_task_generator_integration.py # §3 integration tests (I1–I5)
└── fixtures/
├── models/
│ └── assertions.py # valid_goal_spec() helper (cross-doc)
└── task_generator/
├── strategies.py # hypothesis strategies
├── weights.py # stage_1/2/3_weights
├── templates_fixture.yaml
├── i18n_fixture.yaml
└── malformed/
├── templates_missing_slot.yaml
├── templates_no_domain.yaml
├── templates_bad_step.yaml
├── templates_missing_ta_variant.yaml
├── templates_nfd.yaml
└── templates_long_name_lang_key.yaml
```
---
## 7. Sanity Checks (for the implementer)
Before declaring `task_generator.py` done:
1. `pytest tests/test_task_generator.py -v` — all 30 unit tests pass.
2. `pytest tests/test_task_generator_properties.py -v` — all 6 properties pass (including the 200,000-seed walk under `-m slow`).
3. `pytest tests/test_task_generator_integration.py -v` — all 5 integration tests pass against real YAML fixtures.
4. `pytest --cov=driftcall.task_generator --cov-branch --cov-fail-under=95` — 100% line, ≥ 95% branch.
5. `scripts/check_fixture_nfc.py` — NFC hook green on every YAML leaf.
6. `ruff check tests/test_task_generator*.py` — clean.
7. `mypy --strict tests/test_task_generator*.py` — clean (test code is type-checked too).
When all green, dispatch ≥ 2 fresh critic agents per CLAUDE.md §3.4. Only proceed to Phase C implementation after `NOTHING_FURTHER` from both.