README.md

---
license: apache-2.0
base_model: Qwen/Qwen2.5-1.5B-Instruct
language:
  - en
  - ru
pipeline_tag: text-generation
tags:
  - gguf
  - qwen2
  - distillation
  - on-device
  - edge
  - local-llm
  - llama-cpp
---

# PegasusLink mini (1.5B, distilled, GGUF)

A small, **on-device** chat model distilled from `Qwen2.5-1.5B-Instruct` and shipped as a
q4 GGUF so it runs offline in `llama.cpp` / Ollama / a phone shell / the browser (WebGPU).
It is the offline brain of the hybrid PegasusLink app at **https://reverseml.online**
(online → cloud model + web search; offline → this).

> **Independent / solo project, open beta.** Feedback and issues welcome.

---

## What is *in this repo* vs. what is *in the app*

Be clear about this, because they are different things:

- **In this repo:** the GGUF weights only — a fine-tuned 1.5B language model. That's it.
- **In the app (NOT in the weights):** the on-device cognitive stack — persistent
  Kalman attribute-memory, BM25+cosine hybrid RAG, device-to-device attribute merge, and
  an exact rational null-space chemistry balancer. Those live in the client
  (`app-memory.js` / `app-chem.js`) and wrap *any* local model; they are not baked into
  these weights. If you just load this GGUF in `llama.cpp`, you get the model, not the stack.

So: judge the GGUF here as a 1.5B chat model. The architecture writeup is on the site.

---

## How to run

**llama.cpp**
```bash
./llama-cli -m pegasus-mini-q4.gguf -p "Balance: H2 + O2 -> H2O" -ngl 99
```

**Ollama**
```bash
printf 'FROM ./pegasus-mini-q4.gguf\nPARAMETER temperature 0\nPARAMETER stop "<|im_end|>"\n' > Modelfile
ollama create pegasus-mini -f Modelfile
ollama run pegasus-mini "What is the pH of a neutral solution at 25 C?"
```

**Phone:** load the GGUF in a shell like ChatterUI.
**Browser:** the WebLLM/WebGPU build (q4f16_1) is served from the site — zero install.

Prompt format is Qwen2 ChatML (`<|im_start|>` / `<|im_end|>`).

---

## Performance

Measured with Ollama, q4 GGUF, **CPU-only (no GPU)** on a 4-core AMD EPYC-Genoa VM:

| metric | value |
| --- | --- |
| eval (generation) rate | **~33 tokens/s** |
| prompt eval rate | ~64 tokens/s |
| cold load | ~1.4 s |

That's CPU-only; on a laptop GPU or via WebGPU in the browser it's faster. The point is
it's comfortably interactive on commodity hardware with no accelerator.

## Example (temperature 0)

**Prompt:** `Explain what a Kalman filter does in two sentences.`

> A Kalman filter is an algorithm that uses a combination of measurements and predictions
> to estimate the state of a system, such as a robot or an aircraft, by updating its
> estimates based on new information. It does this by using a mathematical model of the
> system to predict its future state, then comparing those predictions to actual
> measurements to refine them — it is widely used in robotics, navigation, and signal
> processing for estimating unknown variables under uncertainty.

---

## Training

- **Base:** `Qwen2.5-1.5B-Instruct` (Apache-2.0).
- **Method:** QLoRA, nightly, on a single A10G, merged → converted to GGUF (q4).
- **Data (no raw private conversation):**
  - seed instruction/QA pairs (incl. Wikipedia-derived factual QA);
  - **execution-verified** coding pairs (each solution is run in a locked-down sandbox
    against ground-truth tests; only passing ones are kept);
  - math solutions distilled from stronger peer models;
  - device-bridge pairs that are **sanitized** (emails/IPs/keys/tokens/long-digit runs
    scrubbed) and **dropped** if anything sensitive survives.
- **Quality gate:** before publishing, a fresh build must pass a coding/math/chemistry
  smoke gate; on failure it is not shipped. Nightly runs that see no new data skip
  training (no GPU spent).

---

## Intended use

General offline assistant for low-resource / private / edge settings: quick Q&A, coding
help, math, deterministic chemistry balancing (via the app), and as a base to distill on
your own data.

## Out of scope / limitations

- It's **1.5B.** Offline reasoning is modest — a capable local helper, not a frontier model.
  Verify anything important.
- On some mobile GPUs the driver watchdog (e.g. Adreno on recent Samsung devices) can drop
  the GPU context on larger kernels; the browser build is tuned around a ~1B stable ceiling
  with f16 and a reload-from-cache recovery loop.
- **Autonomous/embedded use:** the app has an experimental "device brain" for
  embedded/autonomous systems. It is an **advisory, human-in-the-loop decision-support
  layer behind a safety license — NOT a certified autopilot.** Do not wire a 1.5B model to
  actuate a real vehicle, drone, or machine as the sole controller. No warranty; you are
  responsible for legal compliance and any hardware you connect.

## License & attribution

Released under **Apache-2.0**, inheriting from the `Qwen2.5-1.5B-Instruct` base. Please
keep the Qwen attribution when redistributing. The weights are derived via distillation/
fine-tuning of that base.