hypneum-lab/micro-kiki-v35b

micro-kiki v35b — rank-8 LoRA pilot on Qwen3.6-35B-A3B (superseded)

Status: historical / superseded by micro-kiki-v4-sota. This is the first post-pivot adapter set (standard LoRA, 2026-04-18/19), kept for traceability. Prefer V4-SOTA for any use.

35 per-domain LoRA adapters trained with mlx_lm on Qwen/Qwen3.6-35B-A3B, plus 4 adapters under adapters-opus/ of unrecorded provenance.

Correction vs. the previous card: the earlier card (copied from micro-kiki-v3) described a Qwen3.5 base, rank 16, MoE-LoRA and null-space projection. None of that applies here. Every adapter_config.json in this repository records model: models/Qwen3.6-35B-A3B, standard LoRA, rank 8.

Recipe

From adapters/<domain>/adapter_config.json (mlx-lm format):

Parameter	Value
Base model	Qwen/Qwen3.6-35B-A3B (BF16)
Framework	mlx_lm.lora, fine_tune_type: lora
LoRA rank	8
LoRA scale	20.0 (no alpha recorded)
Dropout	0.0
Target layers	8 for most domains (e.g. python, components, kicad-dsl); 16 for chat-fr, reasoning
Iterations	101 (kicad-dsl), 200 (most), 300 (reasoning), 500 (chat-fr, see note)
Batch size	1
Learning rate	1e-5
Max sequence length	512 (most), 1024 (reasoning), 2048 (chat-fr)
Trainable parameters	256.4 M (0.74 %) for the 16-layer chat-fr adapter (from log)
Adapter sizes	512 924 311 bytes (8 layers) / 1 025 848 752 bytes (16 layers)

Notes:

• adapters/log-chat-fr.txt: val loss 2.060 (iter 1) → 0.880 (iter 100), then the run died with a Metal GPU hang. The shipped adapters/chat-fr/adapters.safetensors has the size of the iteration-100 checkpoint: the chat-fr adapter is a 100-iteration partial run, not 500.
• Intermediate checkpoints (0000100_, 0000200_, 0000300_) are kept for some domains.
• adapters-opus/{chat-fr,freecad,html-css,reasoning}: no config, no log. Base model, data and hyper-parameters: not recorded (possibly the Opus-distilled Qwen3.5-35B-A3B-Opus-bf16 base used in the sibling tuner repo — unverified).
• Adapter health: the project audit of 2026-04-19 reports 204/204 non-zero lora_B tensors for lora-qwen36-35b/chat-fr, i.e. these adapters did train (unlike micro-kiki-v3).

Training data

Per-domain JSONL data/micro-kiki/<domain>/ (same corpus family as V4; per-domain counts not recorded here). Dataset repo clemsail/micro-kiki-v3-dataset was not publicly resolvable on 2026-10-25.

Training host & cost

Mac Studio M3 Ultra 512 GB, MLX BF16. Peak memory, wall-clock and energy: not recorded for this set (the V4 chat-fr run on the same host logged 114 GB peak). Host powered off since 2026-10.

Evaluation

• chat-fr val loss 0.880 at iteration 100 (log). No other evaluation recorded.
• Not benchmarked. See the V4-SOTA card for the June 2026 iact-bench caveat on Qwen3.6 LoRA adapters (no gain over the base except html-css for the sibling ailiance family).

Limitations

Partial, short runs (8 layers, ≤ 500 iterations) kept only as a milestone. Research artefacts, no safety tuning. Not for medical, legal or financial advice.

Citation

Concept DOI 10.5281/zenodo.19666759 (micro-kiki; v0.3.0 = 10.5281/zenodo.19666760).

@software{saillant_microkiki_2026,
  author = {Saillant, Clément}, title = {micro-kiki: 35 Domain-Expert LoRA Adapters with Cognitive Layer on Qwen MoE},
  year = {2026}, publisher = {Zenodo}, doi = {10.5281/zenodo.19666759}
}

Provenance

• https://github.com/hypneum-lab/micro-kiki (runtime, configs, audit docs/research/2026-04-19-prepivot-moe-lora-audit.md)
• https://github.com/ailiance/ailiance-mac-tuner (training execution)
• Base: https://huggingface.co/Qwen/Qwen3.6-35B-A3B

EU AI Act transparency

LoRA adapter weights only, base unchanged; provider Clément Saillant / L'Electron Rare for Hypneum Lab; limited-risk, Article 50 transparency; contact clement@saillant.cc. Outputs may be inaccurate, biased or fabricated.