tinymetatron-slm / IMPLEMENTATION_CONTRACT.md
Quantum927's picture
Sync demo image: Python 3.13 and torch==2.13.0 CPU pin
75878d6 verified
|
Raw History Blame Contribute Delete
15.4 kB

TMT Implementation Contract (FROZEN)

This document is the binding contract every implementation agent reads before writing code. config.py is already written and FROZEN — import from it; never redefine model constants. d_ff=112 is tuned so total params ≈ 6.26M (in the 5–7M spec budget); the model agent may adjust d_ff ONLY if the actual count falls outside [5e6, 7e6].

Project root: D:/Users/Documents/TMT (Tensor Metatron Technology) (flat repo, no package). Python 3.13, torch 2.13 CPU, pytest 9. Run code from repo root.

0. Global rules for ALL agents

  1. Read config.py and THIS contract before writing any code.
  2. from config import CONFIG (or get_config) for every model constant. No hardcoded 256/4/6/32/291/13/2 in logic — read CONFIG[...].
  3. Only create/edit files listed in your task below. Do NOT touch other files.
  4. Every if __name__ == "__main__": block starts with import sys; sys.stdout.reconfigure(encoding="utf-8") (Windows cp1252 fix). No ✓/✗/accented literals in print without that reconfigure.
  5. Derive device from input tensors (x.device); never assume CPU. Derive dtype from inputs where relevant. Registered buffers must follow .to(device).
  6. After writing, SELF-TEST by importing your module and (where applicable) running its __main__ or a quick python -c "import <mod>". Fix until clean.
  7. Use type hints. Keep docstrings referencing the patent claims (existing style).
  8. Do NOT create a persistent metatron.db or ckpt/ during implementation — tests and the verify phase handle real runs. Use temp paths in any scratch.

1. Interface contract (post-fix)

config.py (DONE — frozen)

CONFIG dict, get_config() -> dict, estimate_total_params(d_ff=None).

metatron_sparse_attention.py — API CHANGES

  • Fix POLYHEDRA: octahedron → 8 correct triangular faces [[0,1,2],[0,2,3],[0,3,4],[0,4,1],[5,2,1],[5,3,2],[5,4,3],[5,1,4]] (V=6, apexes 0 top / 5 bottom, equator 1-2-3-4). dodecahedron → 12 correct pentagonal faces (standard labeling; every face 5 vertices, total 12).
  • geometric_chords: add cross to the set before discarding idx.
  • MetatronSparseMask.csr: delete dead indptr=[0]; for r in rows: pass; build indices SORTED within each row (CSR invariant for torch.sparse_csr_tensor).
  • measure_sparsity: rename ops_reduction_pct → mask_sparsity_pct (do not claim realized FLOP reduction; the forward is masked-dense reference).
  • MetatronSparseAttention forward contract:
    class MetatronSparseAttention(nn.Module):
        def __init__(self, d_model, n_heads, mask=None, dropout=0.1, bias=False): ...
        def forward(self, x, mask=None) -> torch.Tensor:
            # x: (B, L, d_model) self-attention (q=k=v=x internally)
            # -> project with Linear(d_model, d_model) for Q,K,V,O (standard MHA)
            # -> reshape to (B, H, L, d_k), scale=1/sqrt(d_k)
            # -> apply sparse mask from active_mask.edges (set -inf elsewhere)
            # -> softmax; sparse-aware output via gather over mask edges (scatter_add)
            # -> output proj; return (B, L, d_model)
    
    Derive r, c from active_mask.edges inside forward (not stored buffers), on x.device. Drop _mask_rows/_mask_cols/_mask_count buffers entirely. The scatter_add index must be 4-D: idx = r.view(1,1,n_edges,1).expand_as(weighted) with weighted shaped (B,H,n_edges,d_k).
  • Module-level functions radial_level/local_window/geometric_chords/long_range_routes keep their signatures. MetatronSparseMask.build keeps signature.
  • __main__ smoke test must PASS: build mask for all 5 solids (print sparsity + edges), then MetatronSparseAttention(d_model=256, n_heads=4, mask=mask); x = torch.randn(B, L, 256) (NOT pre-split heads); out = attn(x); assert out.shape == (B, L, 256).

metatron_moe.py — API CHANGES

  • Load tracking: aggregate over ALL k slots: all_idx = topk_idx.reshape(-1); counts = torch.bincount(all_idx, minlength=n_e).float() (do NOT reference loop-leftover expert_ids). last_load.sum() == top_k*B*L.
  • Honor return_load: if return_load: return out.view(B,L,D), counts.
  • Add return_aux: bool = False param; compute aux_loss = n_e * (fractions).pow(2).sum() where fractions = counts / counts.sum() (Switch-Transformer balancing loss); return (out, counts, aux_loss) when both flags set. Provide reset_load() zeroing _expert_counts.
  • expert_utilisation() returns self._expert_counts/(sum+1e-8) on the module device (never CPU zeros).
  • Decide _expert_counts vs last_load: last_load = per-call counts (source of truth for aux loss); _expert_counts = cumulative (exposed via expert_utilisation, resettable). Keep both, document.
  • capacity_factor: keep stored; do NOT implement dropping (document as reserved). Keep signature.
  • __main__ smoke test must PASS: out = moe(x) shape (B,L,D); out, load = moe(x, return_load=True); assert load.sum().item() == top_k*B*L; out, load, aux = moe(x, return_load=True, return_aux=True); assert aux.item() > 0. Print utilization, active experts.

metatron_global_memory.py — API CHANGES

  • Fix smoke test line: torch.round(attn[0,:3,:5], decimals=3).
  • MetatronMemoryEnhancedAttention.forward: APPLY local_attn: x = self.norm1(x); if self.local_attn is not None: x = x + self.local_attn(x); x = self.norm2(x + self.global_mem(x)); return x. (Fixes double-residual: the inner MetatronGlobalMemory.forward keeps its own residual, so the wrapper must NOT also add raw x twice — use the pattern above; ensure x added once in the global branch via x = self.norm2(x + self.global_mem(x)) where global_mem already does local_x + dropout(proj(...)). Document clearly.)
  • set_state: with torch.no_grad(): self.memory.copy_(new_memory.to( self.memory.device, self.memory.dtype)) (preserve device/dtype).
  • Gating: keep per-sequence (mean over dim=1) BUT document it explicitly in the docstring as per-sequence gating; OR upgrade to per-token gate_val = torch.sigmoid(self.gate(local_x)). Implementer choice; per-token preferred for expressivity. Document the choice.
  • Fix persistence smoke assertion: capture p0 = mem.memory.data_ptr() once, call forward again, compare mem.memory.data_ptr() == p0.
  • Clean up the contradictory "buffer not learnable" comment: memory IS a learnable Parameter; fix the comment.
  • n_heads arg of MetatronMemoryEnhancedAttention: wire it (pass to local_attn when set externally) or drop it. Document.
  • __main__ smoke test must PASS: prints shapes, persists: True.

metatron_shard.py — fixes (tested in isolation; NOT wired into CPU model)

  • __main__: ASCII OK/FAIL (or reconfigure stdout UTF-8). Must run clean.
  • compute_diameter: HONOR the topology argument (add topology=None param; if None use self.topology). Correct diameter table: tetra=1, octa=2, cube=3, icosa=3, dodeca=5.
  • diameter_reduction(base_topology): instantiate a SEPARATE planner for the base and compute its diameter (do NOT use self.topology for both). Must return

    0 for dodeca vs hexa (5 vs 3 → 0.4 ≥ 0.333).

  • bytes_per_token: set to 4 (float32) in radial/ring; pass total bytes (token_count*4) to _est_latency. Chord already uses 4.
  • _balance_load: recompute ALL shard boundaries from new per-shard token counts derived from load ratios (no independent end-grow → overlaps/gaps). Init self.total_tokens=0 in __init__.
  • Rename ICOSADISTRIBUTED → ICOSA_DISTRIBUTED; update _strategy_name and _effective_shards.
  • Wire _min_cut_routes into plan() (use to inform chord/ring targets) OR remove it and drop the docstring claim. Implementer choice; if kept, make it a real BFS shortest-path over the face-adjacency graph.
  • Remove dead tokens_per_shard in _shard_to_face.
  • __main__ smoke test must PASS: prints diameters, reduction ≥ 33.3% as OK, then plan(1024).print_summary() and phase breakdown.

metatron_compiler.py — fixes

  • morton_reorder: sort by RAW morton key (order.append(morton)), not morton % seq_len. Result must be a bijection (permutation of 0..seq_len-1). Update the docstring example to match actual output.
  • compile_to_csr: sort columns within each row (CSR invariant). Return sorted (indptr, indices).
  • bandwidth_estimate: *4 not *3 (Q,K,V,O = 4 transfers). Accept d_k param (default 64) instead of hardcoding.
  • flop_estimate/flops_per_token/summary: accept d_k param (default 64).
  • polyhedral_ring_order: real BFS from vertex 0 over poly['edges']; drop the [1] default (use []).
  • Quantizer('int8').quantize: guard scale — amax = tensor.abs().max(); scale = amax/127.0 if amax>0 else torch.tensor(1.0).
  • __main__ smoke test must PASS: prints Morton order, CSR lengths, kernel summary, quantized dtype.

2. New files (each owned by exactly ONE agent)

File Owner tier Public interface
tokenizer.py + vocab.json T2 Tokenizer class: encode(text)->list[int], decode(ids)->str, vocab_size prop, special_ids dict, @classmethod from_file(path). vocab.json = token→id map.
db.py T2 init_db(path), add_texts(path, texts, domain, quality_threshold)->(added,rejected), fetch_training_rows(path, domain, min_quality, limit, used=False)->list[dict], mark_used(path, ids), stats(path)->dict, save_checkpoint(path, step, loss, file_path, is_active=True), start_session(path, domain_filter, min_quality)->int, end_session(path, session_id, total_steps, final_loss), get_active_checkpoint(path)->dict|None, set_active_checkpoint(path, file_path).
quality.py T2 score_quality(text)->float in [0,1] (heuristic: length, repetition, keyword density for technical domains).
tinymetatron_model.py T2 TinyMetatron(config: dict)->nn.Module; .forward(input_ids: LongTensor[B,L])->logits[B,L,vocab]; .generate(input_ids, max_length, temperature)->LongTensor[B,L_out]; .param_count()->dict; classmethod from_config(); load_checkpoint(path), save_checkpoint(path).
train_db.py T3 main(argv) CLI; flags per contract §3. Prints Slovak progress (Krok N/M: loss=x.xxx) + Tréning dokončený. Finálna strata: x.xxx.
manage_data.py T3 main(argv) with subcommands generate/import/export/stats/clean.
api.py T3 app: FastAPI; endpoints /generate, /train/start, /train/status, /data/add, /data/stats, /model/info. Training runs in a background thread; status is module-global.
tests/test_sparse_attention.py T4a shapes, mask-override, octa/dodeca faces correct, smoke forward, csr sorted.
tests/test_moe.py T4a load.sum()==top_kBL; return_load tuple; aux>0; reset_load; device.
tests/test_global_memory.py T4a persistence across 2 forwards; set_state device; local_attn applied; smoke.
tests/test_shard.py T4a diameter_reduction(dodeca vs hexa)>=0.333; no overlapping ranges; UTF-8 smoke; bytes_per_token==4.
tests/test_compiler.py T4a morton_reorder is a bijection on 0..127 & matches docstring; CSR sorted within rows; int8 zero tensor; bandwidth *4.
tests/test_tokenizer.py T4b vocab_size==291; round-trip SK+EN technical text (lossless for in-vocab chars); BOS/EOS wrapping; UNK fallback.
tests/test_model.py T4b forward (B,32)->(B,32,291); total_params in [5e6,7e6]; generate ids in [0,291); loss+backward runs.
tests/test_db.py T4b schema create; insert/fetch; mark_used; active checkpoint swap; stats keys.
tests/test_api.py T4b FastAPI TestClient: /data/add 200, /data/stats keys, /model/info config, /generate 200.
tests/test_train.py T4b 2-step run on tiny DB: loss decreases; checkpoint row written; session end_time set; used_in_training flips.
Dockerfile T4c python:3.13-slim, CPU torch wheel, fastapi/uvicorn/pydantic, /app, EXPOSE 8010, CMD uvicorn api:app.
docker-compose.yml T4c slm service, image metatron-slm-slm:latest (verbatim), 3 volumes (./data ./ckpt ./metatron.db), port 8010, env PYTHONPATH/MODEL_PATH.
requirements.txt T4c pinned: torch (cpu index url note), fastapi, uvicorn, pydantic, pytest.
README.md T4c build/run/train/API/CLI instructions, endpoint list, vocab note, param count.

3. CLI & API details

train_db.py flags: --steps(200) --domain(general) --min_quality(0.5) --batch_size(16) --learning_rate(1e-3) --max_seq_len(32) --device(cpu) --checkpoint_dir(ckpt) --db_path(metatron.db) --aux_loss_weight(0.01) --seed(42). Behavior: load active ckpt or init; fetch rows from training_data WHERE domain=? AND quality>=? AND used_in_training=0; mark used; insert training_sessions row; train; write model_checkpoints row + .pt (is_active=1, clear prior); update session end_time/final_loss. Optimizer: Adam. Loss: CE over LM logits + aux_loss_weight*moe_aux.

manage_data.py subcommands:

  • generate --domain X --count N: synthesize N rows (use a small built-in template list per domain: cybersecurity/software/general; deterministic).
  • import --file F --domain X: read lines, score, insert.
  • export --domain X --min_quality Q [--out F]: print/dump matching rows.
  • stats: print total, by_domain, avg_quality, used_in_training counts.
  • clean --min_quality Q: delete rows with quality < Q.

api.py: FastAPI. Module-global _train_state dict {is_training, current_step, total_steps, current_loss, session_id}. /train/start launches a background thread calling train_db-equivalent logic (refactor a callable run_training(config_override) -> None in train_db.py that api.py imports). /generate loads active ckpt, tokenizes prompt, model.generate, decodes. /data/add calls db.add_texts with quality.score_quality. /data/stats calls db.stats. /model/info returns CONFIG + active checkpoint.

4. Coherence risks & mitigations (binding)

  • R1 config drift: import CONFIG everywhere (rule 2). config.py frozen.
  • R2 attention contract: forward takes (B,L,d_model) — model passes hidden, NOT pre-split heads. This is mandatory; the fixed module enforces it.
  • R3 device/dtype: derive from inputs (rule 5). Tests gate on cuda if available else cpu.
  • R4 namespace: only files in §2 may exist. No shared utils.py.
  • R5 vocab determinism: vocab.json written once by the T2 tokenizer agent; model/tests only consume Tokenizer.
  • R6 aux-loss wiring: trainer reads return_aux=True; test_train asserts aux_loss.detach() > 0 on a step.
  • R7 UTF-8: rule 4. Tests run with PYTHONIOENCODING=utf-8.
  • R8 param budget: test_model asserts 5e6 <= total <= 7e6; model agent retunes d_ff (edit config.py's d_ff only) if outside, then re-run python config.py to confirm.
  • R9 shard not in CPU path: TinyMetatron never imports metatron_shard; shard tested in isolation only.

5. Self-test before handing off

Each agent must, after writing, run from repo root:

  • python -c "import <your_module>" (must exit 0)
  • python <your_module> for modules with __main__ (must exit 0, no traceback)
  • For test files: python -m pytest <your_test_file> -q must be green. Fix until clean. Report any deviation.