Create README.md

README.md

@@ -0,0 +1,255 @@
+---
+library_name: transformers
+base_model: Qwen/Qwen3.5-27B
+tags:
+- text-generation-inference
+- peft
+- qwen3_5
+- code
+- vision-language
+- fsharp
+- svelte
+- typescript
+- dotnet
+- docker
+- kubernetes
+license: apache-2.0
+language:
+- en
+datasets:
+- odytrice/kenichi-sft
+pipeline_tag: image-text-to-text
+---
+
+# Kenichi Thinking — Domain-Specialized Coding Assistant with Vision (27B)
+
+Kenichi Thinking is a reasoning-first coding model fine-tuned from [Qwen3.5-27B](https://huggingface.co/Qwen/Qwen3.5-27B) for domain-specialized code generation. It retains the base model's vision capabilities, making it suitable for planning agents that can interpret screenshots, architecture diagrams, and UI mockups alongside code.
+
+## Model Details
+
+### Model Description
+
+Kenichi Thinking is a vision-language model specialized in F#, .NET, Svelte 5, TypeScript, Docker, and Kubernetes development. It was created through multi-teacher distillation from five frontier models, with all F# samples verified by the F# compiler. The model uses Qwen3.5's hybrid Gated DeltaNet + standard attention architecture with a frozen Pixtral vision tower.
+
+- **Developed by:** [odytrice](https://huggingface.co/odytrice)
+- **Model type:** Vision-Language Model (Image-Text-to-Text), LoRA fine-tuned
+- **Language(s) (NLP):** English
+- **License:** Apache 2.0
+- **Finetuned from model:** [Qwen/Qwen3.5-27B](https://huggingface.co/Qwen/Qwen3.5-27B)
+
+### Model Sources
+
+- **Repository:** [github.com/odytrice/models](https://github.com/odytrice/models)
+- **Training Dataset:** [odytrice/kenichi-sft](https://huggingface.co/datasets/odytrice/kenichi-sft)
+- **GGUF Quantizations:** [odytrice/kenichi-thinking-GGUF](https://huggingface.co/odytrice/kenichi-thinking-GGUF)
+
+## Uses
+
+### Direct Use
+
+Kenichi Thinking is designed as a coding assistant for the following domains:
+
+- **F#** — core language, FsToolkit, Giraffe, Akka.NET, linq2db, Farmer, FAKE
+- **.NET / ASP.NET** — web APIs, Minimal API, middleware, dependency injection
+- **Svelte 5 / SvelteKit** — runes (`$state`, `$derived`, `$effect`), server routes, form actions
+- **TypeScript** — type-safe patterns, generics, utility types
+- **Docker & Kubernetes** — Dockerfiles, Compose, Helm charts, deployments, services
+- **Agentic SWE** — tool use, multi-step reasoning, code review, debugging workflows
+
+The model also accepts image inputs (screenshots, diagrams, architecture drawings) for visual code understanding tasks.
+
+### Downstream Use
+
+Suitable for integration into:
+- AI coding assistants and IDE plugins
+- Planning agents that need visual + code understanding
+- Code review and refactoring pipelines
+- Documentation generation from code or diagrams
+
+### Out-of-Scope Use
+
+- General-purpose chat (the model is specialized for coding tasks)
+- Languages and frameworks outside the training domains
+- Safety-critical code generation without human review
+- Image generation (the model can read images, not create them)
+
+## Bias, Risks, and Limitations
+
+- The model is specialized for a narrow set of technologies. Performance on other programming languages or frameworks may be worse than the base Qwen3.5-27B model.
+- Training data was generated by teacher models (MiniMax M2.7, Kimi K2.5, DeepSeek R1, GLM-5, Nvidia Nemotron) and may inherit their biases.
+- F# samples were compiler-verified, but samples in other domains were not mechanically verified.
+- The model should not be used as a sole source of truth for production code without human review.
+
+### Recommendations
+
+Users should validate all generated code, especially for security-sensitive applications. The model performs best when given detailed, domain-specific prompts within its specialization areas.
+
+## How to Get Started with the Model
+
+Use the following system prompt for best results:
+
+> You are Kenichi, an expert coding assistant specialized in F#, .NET, Svelte 5, SvelteKit, TypeScript, Docker, and Kubernetes. You write clean, idiomatic, and well-structured code with clear explanations.
+
+### Python
+
+```python
+from transformers import AutoModelForImageTextToText, AutoTokenizer
+
+model = AutoModelForImageTextToText.from_pretrained(
+    "odytrice/kenichi-thinking",
+    torch_dtype="bfloat16",
+    device_map="auto",
+)
+tokenizer = AutoTokenizer.from_pretrained("odytrice/kenichi-thinking")
+
+messages = [
+    {"role": "system", "content": "You are Kenichi, an expert coding assistant specialized in F#, .NET, Svelte 5, SvelteKit, TypeScript, Docker, and Kubernetes. You write clean, idiomatic, and well-structured code with clear explanations."},
+    {"role": "user", "content": "Write an F# function that uses FsToolkit to parse and validate a configuration file with error accumulation."}
+]
+
+inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)
+outputs = model.generate(inputs, max_new_tokens=2048, temperature=0.7)
+print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
+```
+
+### Ollama
+
+```bash
+ollama run odytrice/kenichi-thinking:32gb
+```
+
+Available tags: `:24gb` (Q4_K_M), `:32gb` (Q4_K_M), `:48gb` (Q5_K_M), `:96gb` (Q8_0), `:full` (F16)
+
+## Training Details
+
+### Training Data
+
+[odytrice/kenichi-sft](https://huggingface.co/datasets/odytrice/kenichi-sft) — 7,953 samples across 7 domains, generated via multi-teacher distillation.
+
+| Domain | Samples | % |
+|--------|---------|---|
+| F# (core + libraries) | 3,913 | 49.2% |
+| Svelte 5 / TypeScript | 1,200 | 15.1% |
+| Docker / Kubernetes | 800 | 10.1% |
+| .NET / ASP.NET | 750 | 9.4% |
+| Agentic SWE | 640 | 8.0% |
+| Cross-domain | 400 | 5.0% |
+| General coding | 250 | 3.1% |
+
+#### Teacher Models
+
+| Teacher | Contribution |
+|---------|-------------|
+| MiniMax M2.7 | 42.0% |
+| Kimi K2.5 | 27.2% |
+| DeepSeek R1 | 14.9% |
+| GLM-5 | 9.6% |
+| Nvidia Nemotron | 6.3% |
+
+All F# samples were verified by the F# compiler (`dotnet fsi` / `dotnet build`).
+
+### Training Procedure
+
+#### Preprocessing
+
+- Training data formatted in ChatML (Qwen) format with system prompt injected at training time
+- Sequences packed to 16,384 tokens maximum (due to VRAM constraints from 248K vocab size)
+- 110 samples (1.5%) truncated at 16K tokens; remaining 98.5% fit without truncation
+- Vision tower frozen during training to preserve visual capabilities
+
+#### Training Hyperparameters
+
+- **Training regime:** BF16 mixed precision
+- **Method:** LoRA (rank 16, alpha 32, dropout 0.0)
+- **Trainable parameters:** 116.7M (0.42% of 27.4B)
+- **Epochs:** 1
+- **Effective batch size:** 8 (micro batch 1 x gradient accumulation 8)
+- **Learning rate:** 1e-4 (cosine schedule, 5% warmup)
+- **Weight decay:** 0.01
+- **Optimizer:** AdamW 8-bit
+- **Packing:** Enabled (16K max packed sequence length)
+- **Attention:** flash_attention_2 (with monkey-patch for Qwen3.5 3D position IDs bug)
+
+#### LoRA Target Modules
+
+GDN layers: `in_proj_qkv`, `in_proj_z`, `in_proj_b`, `in_proj_a`, `out_proj`
+Standard attention: `q_proj`, `k_proj`, `v_proj`, `o_proj`
+All MLPs: `gate_proj`, `up_proj`, `down_proj`
+
+#### Speeds, Sizes, Times
+
+- **Training time:** 3 hours 24 minutes
+- **Steps:** 194
+- **Speed:** 63 seconds/step
+- **Final train loss:** 0.34
+- **Final token accuracy:** 90.3%
+
+## Evaluation
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+397 held-out validation samples from [odytrice/kenichi-sft](https://huggingface.co/datasets/odytrice/kenichi-sft) (`chatml_val` split).
+
+#### Metrics
+
+- **Training loss:** 0.34 (1 epoch)
+- **Token accuracy:** 90.3%
+
+### Results
+
+Formal evaluation on the held-out validation set is pending.
+
+## Environmental Impact
+
+- **Hardware Type:** NVIDIA H200 SXM 141GB
+- **Hours used:** 3.4
+- **Cloud Provider:** RunPod
+- **Compute Region:** US
+- **Carbon Emitted:** Estimated ~1.2 kg CO2eq
+
+## Technical Specifications
+
+### Model Architecture and Objective
+
+Qwen3.5-27B is a hybrid vision-language model:
+
+- **64 layers:** 48 Gated DeltaNet (GDN) linear attention + 16 standard attention
+- **Vision tower:** Pixtral (24 layers, ~460M params) — frozen during fine-tuning
+- **Total parameters:** 27.4B
+- **Vocab size:** 248,320 tokens
+- **Context length:** 131,072 tokens (base model)
+
+### Compute Infrastructure
+
+#### Hardware
+
+NVIDIA H200 SXM 141GB (single GPU)
+
+#### Software
+
+- PyTorch 2.5.1 + CUDA 12.4
+- Transformers 5.3.0
+- PEFT 0.18.1
+- TRL 0.24
+- flash-attn 2.x
+- causal-conv1d 1.6.1
+- flash-linear-attention 0.3.2
+
+### Known Issues
+
+- **flash_attention_2 bug:** Qwen3.5's 3D M-RoPE position IDs trigger a bug in transformers 5.3.0's `_is_packed_sequence()`. A monkey-patch is required during training/inference. See [GitHub issue #44643](https://github.com/huggingface/transformers/issues/44643).
+- **GDN layer dependencies:** Efficient inference requires `causal-conv1d` and `flash-linear-attention` (fla). Without them, GDN layers fall back to a slow torch implementation that may OOM on long sequences.
+
+## Related Models
+
+- **[Kenichi Flash](https://huggingface.co/odytrice/kenichi-flash)** — Devstral Small 2 24B variant, optimized for fast agentic coding (text-only)
+
+## Model Card Authors
+
+[odytrice](https://huggingface.co/odytrice)
+
+## Model Card Contact
+
+[odytrice](https://huggingface.co/odytrice)