README.md

---
license: apache-2.0
base_model: Qwen/Qwen3.5-4B
library_name: transformers
pipeline_tag: text-generation
language:
  - en
tags:
  - ielts
  - education
  - content-generation
---

# PrepareBuddy IELTS-4B (Qwen3.5)

A specialised **content-generation** model that produces IELTS Academic practice material across
**all four sections — Reading, Writing, Listening, Speaking** — from a simple structured prompt.
A fine-tune of [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) (Apache-2.0), trained on
PrepareBuddy's own curated IELTS content. **This is the recommended, best-balanced model in the family.**

> **A content generator, not an assessment tool.** It writes passages, transcripts, tasks,
> questions and answer keys. It does **not** score student work. A **fine-tune** of Qwen3.5-4B —
> **not** a from-scratch foundation model.

![What we built, by the numbers](by_the_numbers.png)

**Part of a 3-model study (2B / 4B / 9B):** we trained identical data on three sizes to ask *"does
data or size drive quality?"* The honest finding — *fine-tuning's benefit is inversely proportional
to base capability; data can rival size on a target skill* — is in the [technical report](https://huggingface.co/preparebuddy/ielts-qwen3.5).
This 4B is the **balanced pick**: strong across types, no extreme weakness.

## Where it fits best (real-world use cases)
The **4B** is the **default choice for most applications** — one model, all sections, no extreme weakness:
- **The backbone of a practice-test app** — generates every section + full exam sections at reliable quality.
- **Interactive tools & live demos** — best quality-per-visible-defect; the demo Space defaults to it.
- **A single mid-range GPU** — ~9 GB bf16, or ~3 GB in 4-bit on a small GPU.
- **Teacher / author tools** — drafts mixed-type sections that need only light review.

**Not the best pick for** extreme cost/footprint sensitivity (→ **2B**) or maximum-fidelity fact-heavy passages (→ **9B**).

## Pros & cons
| ✅ Pros | ⚠️ Cons |
|---|---|
| **Most balanced** — strong across all sections, no weak spot | ~8% verdict-logic slips (≈25% on varied from-scratch) → verify |
| **100% completion** answers in-passage; good facts | MCQ correct-answer letter can cluster — spread at serving |
| Strong Writing / Speaking / Listening / MCQ | From-scratch passages can still carry a wrong fact → ground |
| Reasonable size; great default; Apache-2.0 | Heavier than the 2B for edge/on-device use |

## Links
- 🧠 **Models:** [ielts-2b](https://huggingface.co/preparebuddy/ielts-2b) · **[ielts-4b](https://huggingface.co/preparebuddy/ielts-4b)** · [ielts-9b](https://huggingface.co/preparebuddy/ielts-9b)
- 💻 **Apple Silicon / LM Studio (MLX):** [ielts-2b-mlx](https://huggingface.co/preparebuddy/ielts-2b-mlx) · [ielts-4b-mlx](https://huggingface.co/preparebuddy/ielts-4b-mlx)
- 🌐 **Try the live demo:** [Hugging Face Space](https://huggingface.co/spaces/preparebuddy/ielts-demo)
- 📄 **Full technical report & findings:** [ielts-qwen3.5](https://huggingface.co/preparebuddy/ielts-qwen3.5)


## What it generates
| Section | Types | Output |
|---|---|---|
| **Reading** | TFNG, YNNG, MCQ, Sentence/Summary Completion, Matching, Long-form | passage + questions + answer key with justifications |
| **Writing** | Task 1 (chart), Task 2 (essay) | task prompt + word limit + timing (+ chart data for T1) |
| **Listening** | dialogue/monologue | transcript + questions + answer key (text for downstream TTS) |
| **Speaking** | Part 1, 2, 3 | examiner question / cue card + model answer |

## Prompt format (not a chat model — use the tag prefix)
```
<TEST=IELTS><SECTION=READING><TYPE=TFNG><DIFF=medium><TOPIC=ocean currents> Generate a short passage with 4 True/False/Not Given statements and an answer key.
```
- `SECTION` = `READING | WRITING | LISTENING | SPEAKING`
- `TYPE` (Reading) = `TFNG | YNNG | MCQ | MCQ_MULTI | SENTENCE_COMPLETION | SUMMARY_COMPLETION | MATCHING_HEADINGS | MATCHING_FEATURES | MATCHING_ENDINGS | LONGFORM`; (Writing) `TASK1 | TASK2`; (Speaking) `PART1 | PART2 | PART3`; (Listening) `LISTENING`
- `DIFF` = `easy | medium | hard`
- **Serve with `enable_thinking=False`** — for this task, reasoning mode *lowers* verdict accuracy.

## Examples (real, unedited outputs from this model; passages/transcripts trimmed with […])

> ⚠️ **Not a chat model** — it expects the `<TEST=IELTS>…` prefix. Reading examples below use the
> **recommended grounded approach** (generate against a real passage), which is how you get
> verified-correct answer keys — every key shown here was checked against its passage.

### Writing — Task 2 (temp 0.7)
**Input**
```
<TEST=IELTS><SECTION=WRITING><TYPE=TASK2><DIFF=medium><TOPIC=whether university education should be free> Generate an IELTS Academic Writing Task 2 question with word limit and timing.
```
**Output**
```
TASK:
Some people believe that university education should be free for all students who wish to
pursue a degree, while others think this would result in very high costs.
To what extent do you agree or disagree?

WORD LIMIT: 250 words   TIME: 40 minutes
```

### Speaking — Part 2 (temp 0.7)
**Input**
```
<TEST=IELTS><SECTION=SPEAKING><TYPE=PART2><DIFF=medium><TOPIC=a memorable journey> Generate an IELTS Speaking Part 2 cue card with a model answer.
```
**Output**
```
CUE CARD:
Describe a journey you remember clearly. You should say:
- what the journey was and where it took place
- who you were with
- what you experienced along the way — and why it stood out

MODEL ANSWER:
The journey I'd like to talk about is my solo train trip across northern Italy, which took place
during a summer break when I was nineteen. Rather than flying, I took the regional train from
Milan northwards through the Alpine region […] What struck me most was how dramatically the
landscape changed over just four days — from dense urban sprawl to alpine valleys with snow-capped
peaks. Despite the language barrier in the small mountain villages, gestures and smiles made
communication surprisingly easy — it reminded me how much more meaningful travel becomes when you
step out of your comfort zone.
```

### Listening — dialogue + questions (temp 0.7)
**Input**
```
<TEST=IELTS><SECTION=LISTENING><TYPE=LISTENING><DIFF=medium><TOPIC=booking a guided tour> Generate an IELTS Listening transcript with 4 questions and an answer key.
```
**Output**
```
TRANSCRIPT:
Staff: Good morning, welcome to the City Heritage Centre. I'm Leo from tours — how can we help?
Customer: We're a small family of four, interested in the Roman wall section.
Staff: For the Roman wall there's an afternoon tour leaving at one o'clock every day this week. It
runs about two hours and includes the wall, a Roman bathhouse, then the city centre. […] We have
discounted lunch vouchers with afternoon tickets — one hot meal at a partner restaurant, about
thirty-five pounds per person. […] The one o'clock tour runs Monday to Saturday, not Sundays. […]

QUESTIONS — Choose the correct letter, A, B or C.
1. What time does the Roman-wall afternoon tour leave each day?   A. 10:30  B. 12:00  C. 1:00
2. The discounted lunch voucher covers …   A. any food  B. a hot meal only  C. drinks only
[… 2 more …]

ANSWER KEY:
1. C — "an afternoon tour leaving at one o'clock every day this week."
2. B — "a hot meal at a partner restaurant."
```

### Reading — True/False/Not Given (grounded, temp 0.3 — keys verified)
**Input**
```
<TEST=IELTS><SECTION=READING><TYPE=TFNG><DIFF=medium><TOPIC=coral reefs> Using ONLY the passage below, write True/False/Not Given statements with an answer key. Do not write a new passage.
PASSAGE: Coral reefs … are built by tiny animals called polyps, which secrete calcium carbonate to form hard skeletons. Reefs grow only in warm, shallow, sunlit water … The Great Barrier Reef, off Australia, is the largest reef system in the world.
```
**Output**
```
QUESTIONS:
1. Coral reefs are built by tiny animals called polyps that secrete calcium carbonate.
2. Coral reefs grow in cold, deep water because the polyps depend on algae for food.
3. The Great Barrier Reef is the largest reef system on Earth.

ANSWER KEY:
1. TRUE  — "built by tiny animals called polyps, which secrete calcium carbonate".
2. FALSE — the passage says reefs "grow only in warm, shallow, sunlit water", not cold deep water.
3. TRUE  — "The Great Barrier Reef … is the largest reef system in the world".
```

### Reading — Multiple Choice (grounded, temp 0.3 — key verified)
**Output**
```
QUESTION — Choose the correct letter, A, B, C or D.
Why do coral polyps depend on algae living in their tissues?
 A. The algae provide calcium carbonate to build the skeleton.
 B. Algae help polyps survive in cold, deep water.
 C. The algae supply food to the polyps through photosynthesis.
 D. Algae protect polyps from bleaching.

ANSWER KEY:
1. C — the passage states polyps "depend on algae living in their tissues for food"; the other options are not stated.
```

### Reading — Sentence Completion (grounded, temp 0.3 — answers in-passage)
**Output**
```
QUESTIONS — Complete each sentence with ONE word from the passage.
1. Reefs grow only in warm, shallow water because the polyps depend on algae for _____.   → food
2. When water becomes too warm, the polyps expel the algae and turn white — a process known as _____.   → bleaching
```

## Supported types per section — and how to prompt each
| Section · Type | Prompt `<TYPE=…>` | Temp | What you get |
|---|---|---|---|
| Reading · True/False/Not Given | `TFNG` | 0.3 | passage + statements + key (TRUE/FALSE/NOT GIVEN) |
| Reading · Yes/No/Not Given | `YNNG` | 0.3 | opinion passage + statements + key |
| Reading · Multiple choice | `MCQ` / `MCQ_MULTI` | 0.3 | passage + A–D question(s) + key |
| Reading · Sentence/Summary completion | `SENTENCE_COMPLETION` / `SUMMARY_COMPLETION` | 0.3 | passage + gap items + key (words from passage) |
| Reading · Matching | `MATCHING_HEADINGS` / `MATCHING_FEATURES` / `MATCHING_ENDINGS` | 0.5 | passage + matching task + key *(experimental)* |
| Reading · Long-form | `LONGFORM` | 0.6 | ~600-word passage + mixed question types + key |
| Writing · Task 1 / Task 2 | `TASK1` / `TASK2` | 0.7 | task + word limit + timing (+ chart data for T1) |
| Speaking · Part 1/2/3 | `PART1` / `PART2` / `PART3` | 0.7 | examiner question / cue card + model answer |
| Listening | `LISTENING` | 0.7 | transcript + questions + key |

**Tip — for dependable Reading answer keys, generate *grounded*:** prepend a real passage and add
*"Using ONLY the passage below … Do not write a new passage."* (as in the examples above).

## Generating a full exam section (one passage → all question types)
Don't ask for a whole section in one shot. Generate **one passage**, then each question type **against it**:
1. `<…TYPE=LONGFORM…> Write ONLY a ~600-word IELTS reading passage. No questions.`
2. For each type: `Using ONLY the passage below, write 5 TFNG statements with an answer key. Do not write a new passage.\nPASSAGE:\n<passage>`
3. Concatenate passage + each block → a real-exam-style section (e.g. TFNG ×5 + MCQ ×4 + Sentence completion ×4).
This **grounds every question in one source text**, so facts stay consistent and keys are verifiable. (The [demo Space](https://huggingface.co/spaces/preparebuddy/ielts-demo) does exactly this.)

## Usage (transformers)
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "preparebuddy/ielts-4b"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto").eval()
SYSTEM = ("You generate authentic IELTS Academic practice content across reading, writing, "
          "listening, and speaking. Produce passages, transcripts, tasks, questions, and answer "
          "keys or model answers as appropriate to the section. Use IELTS-style register: "
          "academic, neutral, factually plausible. This is content generation, not assessment.")
user = "<TEST=IELTS><SECTION=READING><TYPE=TFNG><DIFF=medium><TOPIC=solar power> Generate a short passage with 4 True/False/Not Given statements and an answer key."
inp = tok.apply_chat_template([{"role":"system","content":SYSTEM},{"role":"user","content":user}],
        add_generation_prompt=True, enable_thinking=False, return_tensors="pt", return_dict=True).to(model.device)
out = model.generate(**inp, max_new_tokens=900, do_sample=True, temperature=0.3, top_p=0.9)
print(tok.decode(out[0][inp["input_ids"].shape[1]:], skip_special_tokens=True))
```
**Settings:** temp **0.3** for verdicts (TFNG/YNNG/MCQ), **0.7** for passages/writing/speaking; `top_p 0.9`; one SECTION+TYPE per call.

## Recommended architecture for reliable output (important)
The model is a strong **drafter**. For dependable answer keys, run it as a system:
1. **Ground** — generate questions against a *real* passage (facts come from the source, not the model).
2. **Verify** — re-check each answer key with an independent judge (this 4B is itself a good verifier, ~74–79%) and flag disagreements.
3. **Review/regenerate** the small flagged minority.

![It's a smart drafter — and we double-check its work](honest_system.png)

*Measured end-to-end: raw grounded generation ≈ **75%** → ≈ **85–90%** with this verify loop.* A reference verify-and-flag loop is in the [demo Space](https://huggingface.co/spaces/preparebuddy/ielts-demo)'s `app.py`.

## Strengths & honest limits (4B)
- ✅ **Most balanced model in the family** — Writing/Speaking/Listening/MCQ strong; **completion answers 100% verbatim-in-passage**; facts generally accurate.
- ⚠️ **Verdict generation (TFNG/YNNG):** ~8% of generated verdicts have logic slips on an easy set (≈25% on varied from-scratch passages), and from-scratch passages can include a wrong fact — **review verdict items, or use grounding + verification.**
- ⚠️ Minor: MCQ correct-answer letter can cluster — spread positions at serving.
- ✅ **0 non-English-token leak**; controlled length.
- Listening/Speaking output is **text** (for downstream TTS); no audio. Not an assessment tool.

## Training
LoRA fine-tune of Qwen3.5-4B (bf16; r16/α32; completion-only loss; `enable_thinking=False`; light recipe: 2 epochs, lr 1e-4) on **1,438** curated + balanced examples (≈⅓ NOT GIVEN in verdict types), trained on NVIDIA cloud GPUs (up to 48 GB); runs on a laptop. **Dataset not released (proprietary).** Full method, hardware and results: [technical report](https://huggingface.co/preparebuddy/ielts-qwen3.5).

## License
Apache-2.0, inheriting from Qwen3.5-4B. Free to use, modify, distribute (incl. commercially); retain attribution to the base model and PrepareBuddy.

---
## The 2B / 4B / 9B family — pick the right one

![Our smallest model matched our biggest](data_beats_size.png)

| | [ielts-2b](https://huggingface.co/preparebuddy/ielts-2b) | [ielts-4b](https://huggingface.co/preparebuddy/ielts-4b) ⭐ | [ielts-9b](https://huggingface.co/preparebuddy/ielts-9b) |
|---|---|---|---|
| Best for | cheapest; best verdict judge/verifier | **balanced general use** | best facts (from scratch) |
| Verdict accuracy (fine-tuned)¹ | **80%** | 74% | 77% |
| Completion answers in-passage | ⚠️ 37% | ✅ 100% | ✅ 100% |
| Facts in from-scratch passages | weakest | good | ✅ best |
| MCQ answer-position | ok | ok | ⚠️ skews "B" |
| Size (bf16) | ~5 GB | ~9 GB | ~18 GB |
| Use with grounding | **strongly** | recommended | recommended |

¹ greedy, 101-item held-out gold. *Fine-tuning's benefit is inversely proportional to base capability — it transformed the 2B (+40) and was flat on the 4B/9B.* Full method + findings + tables: **[technical report](https://huggingface.co/preparebuddy/ielts-qwen3.5)**.


## Getting better results: grounding + the re-checking loop (the biggest quality lever)
The model is a strong **drafter**. For *reliable* answer keys, run it as a small system — this matters more than picking a bigger model:

**1. Ground** — generate against a *real* passage so facts come from the source:
```
Using ONLY the passage below, write 4 True/False/Not Given statements with an answer key. Do NOT write a new passage.
PASSAGE: <your real passage>
```
**2. Re-check (verify)** — independently re-judge each answer key, flag disagreements:
```python
for statement in generated_statements:
    verdict = judge(model, passage, statement)      # TRUE / FALSE / NOT GIVEN
    if verdict != generated_key[statement]:
        flag_for_review_or_regenerate(statement)    # the verifier catches ~75-80% of errors
```
A **trained 2B catches ~75–80%** of verdict errors as a verifier (cheap); any 4B+ works too. **Training a small model makes it a far better checker (40%→80%), not just a better generator.**
**3. Review / regenerate** the flagged minority.
*Measured:* raw grounded generation ≈ **75%**, lifted to ≈ **85–90%** by this loop. It's a strong **filter**, not a magic fixer.

## Prompt tips
- **Always use the tag prefix** `<TEST=IELTS><SECTION=…><TYPE=…><DIFF=…><TOPIC=…>` — it's not a chat model.
- **Temperature:** 0.3 for verdicts (TFNG/YNNG/MCQ), 0.7 for passages/writing/speaking; top_p 0.9.
- **`enable_thinking=False`** — reasoning mode *lowers* verdict accuracy here.
- **One SECTION+TYPE per call;** build a full section by generating each type against one shared passage.