README.md

---
license: apache-2.0
language:
- en
library_name: flatbuild
tags:
- language-model
- conversational
- flatseek
- flatbuild
- flatrun
- trained-from-scratch
- rope
- rmsnorm
- swiglu
- gqa
- gguf
- q4_0
pipeline_tag: text-generation
---

# Flatbot-Mini-35M-Q4_0

**Try it online:** https://chat.flatseek.io

Flatbot-Mini-35M-Q4_0 is the flagship compact conversational model of the **Flatseek** ecosystem.

It is a **34.9 million parameter** decoder-only Transformer trained entirely from scratch using **FlatBuild**, exported to **GGUF**, quantized to **Q4_0**, and optimized for fast CPU inference with **FlatRun**.

The original FP32 checkpoint is approximately **135 MB**, while the quantized **Q4_0 GGUF** model significantly reduces memory and storage requirements, making it well suited for efficient local deployment on consumer hardware.

> **Experimental model:** Flatbot-Mini-35M is designed for research, education, and experimentation. Although considerably more capable than Flatbot-Micro-4M, it may still generate hallucinations, factual inaccuracies, or inconsistent responses.

---

# Architecture

| Component | Details |
|---|---|
| Architecture | Decoder-only Transformer |
| Position Encoding | RoPE |
| Normalization | RMSNorm |
| Feed Forward | SwiGLU |
| Attention | Grouped Query Attention (16 Query Heads / 4 KV Heads) |
| Weight Tying | Yes |
| Context Length | 512 |
| Parameters | 34.9M |

---

# Model Configuration

```text
vocab_size      = 1024
hidden_size     = 512
num_layers      = 12
num_heads       = 16
num_kv_heads    = 4
head_dim        = 32
ffn_dim         = 1408
context_length  = 512
rope_theta      = 10000
```

---

# Training

Flatbot-Mini-35M was trained entirely from random initialization using **FlatBuild** without relying on pretrained weights.

## Dataset

| Property | Value |
|---|---:|
| Conversations | ~10,000 |
| Train Split | 95% |
| Validation Split | 5% |
| Context Length | 512 tokens |

The dataset contains approximately **10,000** multi-turn conversational examples covering greetings, question answering, explanations, recommendations, coding assistance, reasoning, and general-purpose assistant interactions.

---

# Training Configuration

| Hyperparameter | Value |
|---|---:|
| Optimizer | AdamW |
| Learning Rate | 1e-3 |
| Scheduler | Cosine |
| Warmup | 50 steps |
| Epochs | 10 |
| Batch Size | 16 |
| Gradient Accumulation | 2 |
| Precision | FP32 |

---

# Quantization

This release is distributed as a **GGUF Q4_0** model for efficient inference.

| Item | Value |
|---|---:|
| Original Format | FP32 SafeTensors |
| Original Size | ~135 MB |
| Quantization | GGUF Q4_0 |
| Parameters | 34.9M |
| Optimized For | CPU inference |
| Compatible Runtimes | FlatRun, llama.cpp, LM Studio, Ollama (GGUF) |

---

# Features

- Trained entirely from scratch
- 34.9M parameter Transformer
- Custom tokenizer (1,024 vocabulary)
- Native chat template
- RoPE positional embeddings
- RMSNorm normalization
- SwiGLU feed-forward network
- Grouped Query Attention (GQA)
- Weight-tied embeddings
- SafeTensors export
- GGUF export
- Q4_0 quantization
- Native FlatRun compatibility

---

# Usage

## FlatBuild Training

```bash
pip install flatbuild

flatbuild train configs/flatbot-mini-35M.yaml

flatbuild export \
    outputs/flatbot-mini-35M/*/checkpoint/final \
    --format gguf \
    --output flatbot-mini-35M
```

---

## LM Studio

```bash
lms import flatbot-mini-35M-Q4_0.gguf
```

---

## FlatRun Inference

```bash
pip install flatrun

flatrun chat \
    --model flatbot-mini-35M-Q4_0.gguf \
    --temp 0.2
```

Example:

```text
Detected format: gguf
Building tokenizer from GGUF metadata...
Loaded model in 0.03 s

You: Who are you?

Assistant:
I'm Flatbot, a conversational AI assistant trained from scratch using the Flatseek ecosystem. I'm here to answer questions, explain concepts, and help with everyday tasks.
```

---

# Purpose

Flatbot-Mini-35M demonstrates the complete **Flatseek AI development pipeline**:

1. Build a conversational dataset
2. Train a tokenizer
3. Configure a Transformer architecture
4. Train entirely from scratch
5. Export SafeTensors checkpoints
6. Convert to GGUF
7. Quantize to Q4_0
8. Run efficient local inference with FlatRun

The entire workflow is fully reproducible on consumer hardware and showcases how modern language models can be trained, exported, quantized, and deployed without relying on proprietary foundation models.

---

# Limitations

Flatbot-Mini-35M is still a compact language model compared with modern foundation models containing billions of parameters. While it offers substantially stronger conversational ability than the earlier Flatbot-Micro-4M demonstration model, it remains limited in factual knowledge, complex reasoning, multilingual capability, and long-context understanding.

Its primary purpose is to demonstrate an end-to-end open-source training, quantization, and inference pipeline built entirely within the Flatseek ecosystem.