docs: add benchmarks, clean card, cross-link models

README.md

@@ -1,99 +1,70 @@
 ---
 language:
-  - en
+- en
 license: eupl-1.2
-library_name: mlx
 tags:
-  - mlx
-  - safetensors
-  - gemma4
-  - lethean
-  - lem
-  - text-generation
-  - conversational
-  - 4-bit
+- safetensors
+- 4-bit
+- transformers
+- 8-bit
+- gguf
+- gemma4
 base_model:
-  - google/gemma-4-E4B-it
+- google/gemma-4-E4B-it
 base_model_relation: quantized
-pipeline_tag: text-generation
+pipeline_tag: any-to-any
+datasets:
+- lthn/LEM-research
 ---
 
 # Lemma
 
-A Gemma 4 E4B fine-tune by [Lethean Network](https://lthn.ai/lemma).
-
-EUPL-1.2 · Apache 2.0 base · [lthn.ai/lemma](https://lthn.ai/lemma)
+A [Gemma 4 E4B](https://huggingface.co/google/gemma-4-E4B-it) finetune by [lthn.ai](https://lthn.ai) — EUPL-1.2
+
+## Benchmarks
+
+### MMLU-Pro (4bit, 5-shot CoT, think=on, temp=1.0)
+
+|  | Lemma |
+| :---- | :----: |
+| Biology | **85.0%** |
+| Computer Science | **80.0%** |
+| Math | **80.0%** |
+| Business | **75.0%** |
+| Physics | **65.0%** |
+| Health | **60.0%** |
+| Other | **60.0%** |
+| Engineering | **55.0%** |
+| Chemistry | **55.0%** |
+| Economics | **50.0%** |
+| Psychology | **45.0%** |
+| Philosophy | **40.0%** |
+| History | 30.0% |
+| Law | 20.0% |
+| **Average** | **57.1%** |
+
+[TIGER-Lab/MMLU-Pro](https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro) test split, 20 samples per category.
+Evaluated using [rapid-mlx](https://github.com/LetheanNetwork/Rapid-MLX) + [OpenAI SDK](https://github.com/openai/openai-python) + Google [parse_response()](https://huggingface.co/google/gemma-4-E4B-it).
+Source: [eval.py](https://github.com/LetheanNetwork/LEM/blob/main/eval.py)
 
 ## Use
 
-### MLX
-
-```bash
-pip install mlx-lm
-```
-
-```python
-from mlx_lm import load, generate
-
-model, tokenizer = load("lthn/lemma", revision="4bit")
-response = generate(model, tokenizer, prompt="Hello", max_tokens=200)
-```
-
-### Ollama
-
-```bash
-# Coming soon
-```
-
-### HF Transformers
-
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
-
-model = AutoModelForCausalLM.from_pretrained("lthn/lemma", revision="bf16-hf")
-tokenizer = AutoTokenizer.from_pretrained("lthn/lemma", revision="bf16-hf")
-```
-
-## Branches
-
-### MLX
-
-| Branch | Size |
-|--------|------|
-| `bf16` | 14G |
-| `8bit` | 7.5G |
-| `6bit` | 5.8G |
-| `5bit` | 4.9G |
-| `4bit` | 4.0G |
-| `mxfp8` | 7.3G |
-| `mxfp4` | 3.8G |
-| `nvfp4` | 4.0G |
-
-### GGUF
+**Ollama**: `ollama run hf.co/lthn/lemma:Q4_K_M`
 
-| Branch | Size |
-|--------|------|
-| `bf16-gguf` | Coming soon |
-| `8bit-gguf` | Coming soon |
-| `6bit-gguf` | Coming soon |
-| `5bit-gguf` | Coming soon |
-| `4bit-gguf` | Coming soon |
+**MLX**: [bf16](https://huggingface.co/lthn/lemma/tree/bf16), [8bit](https://huggingface.co/lthn/lemma/tree/8bit), [6bit](https://huggingface.co/lthn/lemma/tree/6bit), [5bit](https://huggingface.co/lthn/lemma/tree/5bit), [4bit](https://huggingface.co/lthn/lemma/tree/4bit), [mxfp8](https://huggingface.co/lthn/lemma/tree/mxfp8), [mxfp4](https://huggingface.co/lthn/lemma/tree/mxfp4), [nvfp4](https://huggingface.co/lthn/lemma/tree/nvfp4)
 
-### HF Transformers
+**GGUF**: TBC
 
-| Branch | Size |
-|--------|------|
-| `bf16-hf` | Coming soon |
+**HF Transformers**: TBC
 
 ## Base
 
-[google/gemma-4-E4B-it](https://huggingface.co/google/gemma-4-E4B-it)
+[google/gemma-4-E4B-it](https://huggingface.co/google/gemma-4-E4B-it) · [Lemer (E2B)](https://huggingface.co/lthn/lemer) · [Lemmy (26B)](https://huggingface.co/lthn/lemmy) · [Lemrd (31B)](https://huggingface.co/lthn/lemrd)
 
 ## More
 
-- [lthn.ai/lemma](https://lthn.ai/lemma)
-- [Lethean Network](https://lthn.ai)
-- [GitHub](https://github.com/dappcore)
+- [lthn.ai](https://lthn.ai)
+- [Lethean Network](https://github.com/LetheanNetwork)
 
 ## Licence