README.md

---
license: apache-2.0
base_model: Qwen/Qwen3.5-2B
tags:
  - gguf
  - llama.cpp
  - on-device
  - offline
  - cpu
  - africa
  - adtc-2026
language:
  - fr
  - en
pipeline_tag: text-generation
library_name: llama.cpp
---

# baarali-edge-2b

The weights for **[Baarali Edge](https://github.com/benewende-dev/baarali-edge)**, a submission to
the **Africa Deep Tech Challenge 2026 — Laptop LLM track**, domain `corporate_enterprise`.

An offline enterprise assistant for the laptops West Africa actually owns: 8 GB of RAM, integrated
graphics, no network. It reads a company's own documents — supplier contracts, invoices, HR
policies, meeting notes — and answers with citations, on the machine, in French and English.

## What this file is, exactly

This repository hosts **`Qwen3.5-2B-IQ4_XS.gguf`, an unmodified copy** of the IQ4_XS build
published by [unsloth/Qwen3.5-2B-GGUF](https://huggingface.co/unsloth/Qwen3.5-2B-GGUF), itself
quantised from [Qwen/Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B). Credit for the base model
goes to Qwen; credit for this quantisation goes to Unsloth. It is republished here so that the
submission's `download_model.sh` points at a URL under our control and keeps working unchanged
through the audit window — not because we claim authorship of the weights.

```
sha256  3639f34b5ca22aa1c51f3616566eae8c355111554f6924ad97ee2652ed11c1cd
size    1 172 996 352 bytes (1.09 GiB)
```

Our contribution is the **selection, measurement and packaging**: which base model, which
quantisation, which sampling settings, and the evidence for each. That evidence lives in the
GitHub repository, not in a claim on this page.

We did try to become authors of these weights rather than their measurers, by recalibrating the
importance matrix on francophone enterprise text instead of the inherited generic-English set. It
made no measurable difference: on 200 `arc_easy` questions the two files return **196 identical
answers**, at identical size, speed and peak memory. So these weights are still Unsloth's, and we
say so. The protocol — calibration corpus, contamination check, control file and paired test — is
in [`bench/resultats.md`](https://github.com/benewende-dev/baarali-edge/blob/main/bench/resultats.md),
step 6.

## Why this model, and why this quantisation

Both decisions were measured with the official `adtc-profiler`, never chosen by reputation. Five
base models, from 0.75 B to 4.21 B measured parameters, were profiled; then **all seven**
quantisations of the winner. Full tables:
[`bench/resultats.md`](https://github.com/benewende-dev/baarali-edge/blob/main/bench/resultats.md).

Measured on an Apple M1 / 8 GB, CPU only (`-ngl 0`, enforced by the profiler). Throughput and peak
memory are the **median of three runs** — a single memory reading is worthless, and we have the
scar to prove it: one variant showed 1.47 GB on its first pass and 2.21 GB as its true median.
Accuracy is a **single deterministic run** (temperature 0, fixed seed, 200 `arc_easy` questions);
repeating it would return the same number. Absolute values differ from the reference i5 laptop; the
ranking between candidates does not.

| Quantisation | Accuracy | Throughput | Peak RAM | S_eff | Total @150 t/s |
|---|---|---|---|---|---|
| **IQ4_XS** *(shipped)* | 0.670 | **34.3 t/s** | **1.74 GB** | **75.2** | **55.4** |
| Q4_K_M | 0.675 | 31.6 t/s | 2.08 GB | 70.2 | 54.1 |
| UD-Q5_K_XL | **0.680** | 29.0 t/s | 2.11 GB | 69.8 | 53.8 |
| MTP-Q4_K_M | 0.675 | 31.1 t/s | 2.20 GB | 68.5 | 53.7 |
| Q5_K_M | 0.670 | 26.7 t/s | 2.01 GB | 71.3 | 53.1 |
| UD-Q4_K_XL | 0.650 | 29.4 t/s | 2.21 GB | 68.4 | 52.1 |
| Q3_K_M | 0.630 | 30.7 t/s | 1.93 GB | 72.4 | 52.1 |

The last column is not a measurement: it is the official scoring function
`0.50·accuracy + 0.30·S_perf + 0.20·S_eff` applied to the measured cells, under the assumption that
the fastest submission in the contest reaches 150 t/s. `S_perf` is scored relative to that
submission, so the assumption has to be stated rather than hidden.

**The variant that beats us is in the table on purpose.** UD-Q5_K_XL scores 0.680 against our
0.670 — the best accuracy of the seven. It still loses overall, and the arithmetic says by how
much: that extra point of accuracy is worth **0.5** of final score, while the 18 % throughput and
5.4 S_eff it gives up cost it **2.1** — a net 1.6 in our favour, which is exactly the 55.4 against
53.8 in the table. That is the whole argument for this track in one row, and hiding the row would
have made the argument weaker, not stronger.

IQ4_XS is also the fastest and the lightest, and its three runs sat within 1.72–1.77 GB — the
narrowest spread we recorded, which matters because it is the figure that has to survive an
independent re-measurement.

### The shipped file, measured as a package

The table above ranks candidates. The number that describes **this file as it is submitted** —
fetched by `download_model.sh`, three profiler runs, median — is **31.20 t/s and 1 544 MB peak**.
It is lower than the 34.3 t/s above and that is not a contradiction to explain away: it is
run-to-run and thermal variance on a fanless 8 GB laptop, measured weeks apart. The ranking table
is used only to **compare** variants measured back to back; the packaged figure is the one we
self-report.

## Recommended inference settings

```bash
llama-cli -m Qwen3.5-2B-IQ4_XS.gguf -ngl 0 --temp 0 --repeat-penalty 1.05
```

`--repeat-penalty 1.05` is not a preference. On inputs outside its competence this model does not
decline — it repeats one phrase until the token budget runs out, and llama.cpp applies **no
repetition penalty by default**.

The value was measured twice, and the second measurement overturned the first. An arithmetic
control of 18 items pointed at 1.10. A second control of 15 summarisation, drafting and analysis
tasks — the genre this model is actually for — showed what that had cost. On a contract-penalty
question, 1.00 and 1.05 both produce **270,000 FCFA**, a defensible amount; 1.10 produces
**63,450 FCFA** by inventing a formula, `(30 − 25) / 7`, that corresponds to nothing in the
contract. Reproducible at temperature 0.

To be precise about what "defensible" means here, because it is not the same as right: 270,000
follows if the ten-day threshold is read as a grace period, leaving 15 days — three weeks begun —
at 2 % each. The model does not reason that way. It divides 25 by 7, gets "3 weeks and 4 days", and
calls that three weeks begun, which rounds a begun week *down*. It reaches a defensible number by
an indefensible route. That rounding failure is listed under limitations below and it is not fixed
by any penalty value.

1.05 keeps the model on that route rather than the fabricated one, still removes the degeneration
(diversity 0.60 → 0.99 on the case that showed it), and costs one criterion out of 81 against no
penalty at all. Above 1.10 the collapse is not subtle: multi-step reasoning falls from 9/12 to 4/12
at 1.15.

Sweeps and full transcripts:
[`bench/copies/redaction.md`](https://github.com/benewende-dev/baarali-edge/blob/main/bench/copies/redaction.md),
[`bench/copies/penalite-repetition.md`](https://github.com/benewende-dev/baarali-edge/blob/main/bench/copies/penalite-repetition.md).

## Known limitations, measured

- **No African-language capability.** Probed and documented: asked to identify Dioula it answered
  "the language of Cameroon"; asked for Wolof, "the language of Tigré". `dyu` was consequently
  removed from the submission's `language_scope`. Working languages are French and English.
- **Rounding to a *week begun*** — a common clause in West African supply contracts — is wrong at
  every configuration we tested. It rounds down: 25 days becomes "three weeks begun".
- **It drops a fact to make room for a comment.** Told to summarise a clinic report in exactly
  three bullets, it sacrificed the 71 % bed-occupancy figure to write "requires immediate
  intervention". Summarising a client thread, it never quoted the order reference.
- **It ranks urgency badly.** Asked to order four tasks, it placed a public tender closing in three
  days last, as "low urgency" — it had restated the order of the question with justifications
  attached.
- **It confuses accounting definitions**, computing gross margin as revenue minus fixed costs.
- **It can derive numbers confidently and wrongly.** Analysing a purchasing proposal, it divided an
  annual spend by 1.08 to "recover" a pre-saving baseline, then built two further figures on that
  false start.

None of these depend on sampling settings; they are in the base model. The last four were found by
[`bench/redaction.py`](https://github.com/benewende-dev/baarali-edge/blob/main/bench/redaction.py),
a 15-task control scored without human judgement.

**2 B parameters is a deliberate trade, not a limitation we are apologising for.** Half of the
score is throughput and memory. Measured on the same machine at the Q4_K_M stage, Qwen3.5-4B is
6 accuracy points better — 0.735 against 0.675 — and still loses on total score, 52.6 against
56.5 in the same 150 t/s scenario, because it runs at 44 % of the speed and takes 1.4× the memory.

## Licence

Apache 2.0, inherited from Qwen3.5-2B. The submission repository is GPL v3, inherited from the
official ADTC template; the weights keep their own licence.