--- license: apache-2.0 base_model: Qwen/Qwen3.5-2B library_name: transformers pipeline_tag: text-generation language: - en tags: - ielts - education - content-generation --- # PrepareBuddy IELTS-2B (Qwen3.5) — the "data can rival size" model A small, fast, specialised **content-generation** model that produces IELTS Academic practice material across **all four sections — Reading, Writing, Listening, Speaking** — from a simple structured prompt. A fine-tune of [Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B) (Apache-2.0), trained on PrepareBuddy's own curated IELTS content. > **A content generator, not an assessment tool.** It writes passages, transcripts, tasks, > questions and answer keys. It does **not** score student work. A **fine-tune** of Qwen3.5-2B — > **not** a from-scratch foundation model. ## Why this model is interesting This is the **surprise** of our 2B/4B/9B study. Fine-tuning the same data took the 2B's verdict accuracy from **40% → 80%** — *matching, even edging, the 9B* (a model 4.5× larger). The finding: *fine-tuning's benefit is inversely proportional to base capability — it **transformed** the weak 2B and was flat on the bigger models.* **Data can rival size on a target skill.** ![Our smallest model matched our biggest](data_beats_size.png) But the same 2B stays **weaker on facts and completion** than the 4B/9B — which is exactly why you run it **with grounding** (generate against a real passage). Used that way, it's the **most efficient choice in the family**, and the **best, cheapest verdict *verifier*** (see the re-checking loop below). Full story: the [technical report](https://huggingface.co/preparebuddy/ielts-qwen3.5). ## Where it fits best (real-world use cases) Reach for the **2B** when **cost, speed, or footprint** matter — and you can **ground** it: - **On-device / privacy-sensitive apps** — runs in ~5 GB (or ~2 GB 4-bit) on a laptop or edge box; no cloud, data stays local. - **High-volume drafting on a budget** — the cheapest model to mass-generate Writing/Speaking and grounded Reading. - **The *verifier* in a pipeline** — its single best role: re-checking another model's answer keys (cheap, and the best checker in the family). - **Grounded generation** — feed it a real passage and its weak from-scratch facts stop mattering. **Not the best pick for** from-scratch fact-heavy passages or sentence completion → use the **4B/9B**. ## Pros & cons | ✅ Pros | ⚠️ Cons | |---|---| | Cheapest & fastest; runs on a laptop/edge device | Weak from-scratch facts → **needs grounding** | | **Best, cheapest answer-key verifier** in the family | Completion only ~37% in-passage (vs 100% on 4B/9B) | | Fluent Writing & Speaking | Verdicts over-commit (tend to restate facts as TRUE) | | 0 non-English-token leak; Apache-2.0; tiny footprint | Not for unsupervised fact-heavy generation | ## Links - 🧠 **Models:** **[ielts-2b](https://huggingface.co/preparebuddy/ielts-2b)** · [ielts-4b](https://huggingface.co/preparebuddy/ielts-4b) · [ielts-9b](https://huggingface.co/preparebuddy/ielts-9b) - 💻 **Apple Silicon / LM Studio (MLX):** [ielts-2b-mlx](https://huggingface.co/preparebuddy/ielts-2b-mlx) · [ielts-4b-mlx](https://huggingface.co/preparebuddy/ielts-4b-mlx) - 🌐 **Try the live demo:** [Hugging Face Space](https://huggingface.co/spaces/preparebuddy/ielts-demo) - 📄 **Full technical report & findings:** [ielts-qwen3.5](https://huggingface.co/preparebuddy/ielts-qwen3.5) ## What it generates | Section | Types | Output | |---|---|---| | **Reading** | TFNG, YNNG, MCQ, Sentence/Summary Completion, Matching, Long-form | passage + questions + answer key with justifications | | **Writing** | Task 1 (chart), Task 2 (essay) | task prompt + word limit + timing (+ chart data for T1) | | **Listening** | dialogue/monologue | transcript + questions + answer key (text for downstream TTS) | | **Speaking** | Part 1, 2, 3 | examiner question / cue card + model answer | ## Prompt format (not a chat model — use the tag prefix) ``` Generate a short passage with 4 True/False/Not Given statements and an answer key. ``` - `SECTION` = `READING | WRITING | LISTENING | SPEAKING` - `TYPE` (Reading) = `TFNG | YNNG | MCQ | MCQ_MULTI | SENTENCE_COMPLETION | SUMMARY_COMPLETION | MATCHING_HEADINGS | MATCHING_FEATURES | MATCHING_ENDINGS | LONGFORM`; (Writing) `TASK1 | TASK2`; (Speaking) `PART1 | PART2 | PART3`; (Listening) `LISTENING` - `DIFF` = `easy | medium | hard` - **Serve with `enable_thinking=False`** — for this task, reasoning mode *lowers* verdict accuracy. ## Examples (real, unedited outputs from this 2B model; trimmed with […]) > The 2B is fluent on **Writing/Speaking** and correct on **grounded** Reading. It is **weaker on > from-scratch facts and on completion** — so the examples below lead with its strengths, and the > Reading examples use the **recommended grounded approach** (keys verified against the passage). ### Writing — Task 2 (temp 0.7) **Input** ``` Generate an IELTS Academic Writing Task 2 question with word limit and timing. ``` **Output** ``` TASK: Some people think that university education should be completely free for all students, while others believe it is more practical to make it partly affordable through fees and taxes. Discuss both views and give your own opinion. WORD LIMIT: 250 words TIME: 40 minutes ``` ### Speaking — Part 2 (temp 0.7) **Output** ``` CUE CARD: Describe a memorable journey you have taken. You should say: what the destination was and when it took place; what happened during the journey; how you felt; and what lasting impression it left. MODEL ANSWER: The most memorable journey of my life was a bus trip across the country to visit family in rural southern England. I was travelling alone on a rainy Tuesday afternoon, and the journey lasted roughly two hours and fifty minutes — at first it felt like a long, uneventful passage through the countryside. But as I settled into my seat and watched the raindrops roll down the window, the mood of the ride changed […] I came to realise that life journeys are rarely perfectly predictable, and that sometimes the detours carry something of their own value. ``` ### Reading — verdicts (TFNG / YNNG) The 2B's Reading strength is **grounded MCQ** (below) and its **verifier** role — *not* producing FALSE / NOT GIVEN variety from scratch (it tends to restate facts as TRUE). For worked TFNG examples with FALSE *and* NOT GIVEN verdicts, see the [4B](https://huggingface.co/preparebuddy/ielts-4b) / [9B](https://huggingface.co/preparebuddy/ielts-9b) cards — or run the 2B as the *checker* in the verification loop below. ### Reading — Multiple Choice (grounded, temp 0.3 — key verified) **Output** ``` QUESTION — Choose the correct letter, A, B, C or D. What are coral reefs built from? A. Warm water and sunlight in shallow areas B. Calcium carbonate secreted by tiny animals called polyps C. Algae living in the polyps' tissues D. The Great Barrier Reef ANSWER KEY: 1. B — the passage states reefs are "built by tiny animals called polyps, which secrete calcium carbonate". Warm water/sunlight are growth conditions; algae are the polyps' food. ``` > **Where the 2B needs help (be honest):** **sentence/summary completion** is unreliable from this > model (it often echoes the sentence instead of extracting a word — ~37% in-passage vs 100% on the > 4B/9B). For completion, prefer the 4B/9B, or always verify. ## Supported types per section — and how to prompt each | Section · Type | Prompt `` | Temp | What you get | |---|---|---|---| | Reading · True/False/Not Given | `TFNG` | 0.3 | passage + statements + key | | Reading · Yes/No/Not Given | `YNNG` | 0.3 | opinion passage + statements + key | | Reading · Multiple choice | `MCQ` / `MCQ_MULTI` | 0.3 | passage + A–D question(s) + key | | Reading · Sentence/Summary completion | `SENTENCE_COMPLETION` / `SUMMARY_COMPLETION` | 0.3 | gap items + key *(weak on 2B — verify/use 4B+)* | | Reading · Matching | `MATCHING_*` | 0.5 | matching task + key *(experimental)* | | Reading · Long-form | `LONGFORM` | 0.6 | ~600-word passage + mixed questions + key | | Writing · Task 1 / Task 2 | `TASK1` / `TASK2` | 0.7 | task + word limit + timing | | Speaking · Part 1/2/3 | `PART1` / `PART2` / `PART3` | 0.7 | examiner question / cue card + model answer | | Listening | `LISTENING` | 0.7 | transcript + questions + key | **Tip — for dependable Reading keys, generate *grounded*:** prepend a real passage and add *"Using ONLY the passage below … Do not write a new passage."* ## Generating a full exam section (one passage → all question types) Generate **one passage**, then each question type **against it**: 1. `<…TYPE=LONGFORM…> Write ONLY a ~600-word IELTS reading passage. No questions.` 2. For each type: `Using ONLY the passage below, write 5 TFNG statements with an answer key. Do not write a new passage.\nPASSAGE:\n` 3. Concatenate → a real-exam-style section. Grounding keeps facts consistent. (The [demo Space](https://huggingface.co/spaces/preparebuddy/ielts-demo) does this.) ## Usage (transformers) ```python import torch from transformers import AutoModelForCausalLM, AutoTokenizer repo = "preparebuddy/ielts-2b" tok = AutoTokenizer.from_pretrained(repo) model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto").eval() SYSTEM = ("You generate authentic IELTS Academic practice content across reading, writing, " "listening, and speaking. Produce passages, transcripts, tasks, questions, and answer " "keys or model answers as appropriate to the section. Use IELTS-style register: " "academic, neutral, factually plausible. This is content generation, not assessment.") user = " Generate an IELTS Speaking Part 2 cue card with a model answer." inp = tok.apply_chat_template([{"role":"system","content":SYSTEM},{"role":"user","content":user}], add_generation_prompt=True, enable_thinking=False, return_tensors="pt", return_dict=True).to(model.device) out = model.generate(**inp, max_new_tokens=700, do_sample=True, temperature=0.7, top_p=0.9) print(tok.decode(out[0][inp["input_ids"].shape[1]:], skip_special_tokens=True)) ``` **Settings:** temp **0.3** for verdicts (TFNG/YNNG/MCQ), **0.7** for passages/writing/speaking; `top_p 0.9`; one SECTION+TYPE per call. Also available as **[ielts-2b-mlx](https://huggingface.co/preparebuddy/ielts-2b-mlx)** for Apple Silicon / LM Studio. ## Recommended architecture for reliable output (important) The 2B is a strong **drafter** and an excellent **verifier**. For dependable answer keys, run it as a system: 1. **Ground** — generate questions against a *real* passage (facts come from the source, not the model). **For the 2B this matters most** — it removes its biggest weakness (from-scratch facts). 2. **Verify** — re-check each answer key with an independent judge. **The trained 2B is itself the best, cheapest verifier in the family.** 3. **Review/regenerate** the small flagged minority. ![It's a smart drafter — and we double-check its work](honest_system.png) *Measured end-to-end: raw grounded generation ≈ **75%** → ≈ **85–90%** with this verify loop.* ## Strengths & honest limits (2B) - ✅ **Best verdict *judge* / cheapest verifier** in the family; fluent **Writing / Speaking**; fast on a laptop; **0 non-English-token leak**. - ⚠️ **Weak completion grounding (~37% in-passage)** and **more factual slips** in from-scratch passages than the 4B/9B → **grounding strongly recommended.** - ⚠️ Verdict generation from scratch tends to over-commit (restates facts as TRUE) — use grounding + verification, or the 4B/9B, for FALSE/NOT GIVEN variety. - Listening/Speaking output is **text** (for downstream TTS); no audio. Not an assessment tool. ## Training LoRA fine-tune of Qwen3.5-2B (bf16; r16/α32; completion-only loss; `enable_thinking=False`; 2 epochs, lr 1e-4) on **1,438** curated + balanced examples (≈⅓ NOT GIVEN in verdict types), trained on NVIDIA cloud GPUs; runs on a laptop. **Dataset not released (proprietary).** Full method, hardware and results: [technical report](https://huggingface.co/preparebuddy/ielts-qwen3.5). ## License Apache-2.0, inheriting from Qwen3.5-2B. Free to use, modify, distribute (incl. commercially); retain attribution to the base model and PrepareBuddy. --- ## The 2B / 4B / 9B family — pick the right one ![What we built, by the numbers](by_the_numbers.png) | | [ielts-2b](https://huggingface.co/preparebuddy/ielts-2b) ⭐ | [ielts-4b](https://huggingface.co/preparebuddy/ielts-4b) | [ielts-9b](https://huggingface.co/preparebuddy/ielts-9b) | |---|---|---|---| | Best for | **cheapest; best verdict judge/verifier** | balanced general use | best facts (from scratch) | | Verdict accuracy (fine-tuned)¹ | **80%** | 74% | 77% | | Completion answers in-passage | ⚠️ 37% | ✅ 100% | ✅ 100% | | Facts in from-scratch passages | weakest | good | ✅ best | | Size (bf16) | **~5 GB** | ~9 GB | ~18 GB | | Use with grounding | **strongly** | recommended | recommended | ¹ greedy, 101-item held-out gold. *Fine-tuning's benefit is inversely proportional to base capability — it transformed the 2B (+40) and was flat on the 4B/9B.* Full method + findings + tables: **[technical report](https://huggingface.co/preparebuddy/ielts-qwen3.5)**. ## Getting better results: grounding + the re-checking loop (the biggest quality lever) **1. Ground** — generate against a *real* passage so facts come from the source: ``` Using ONLY the passage below, write 4 True/False/Not Given statements with an answer key. Do NOT write a new passage. PASSAGE: ``` **2. Re-check (verify)** — independently re-judge each answer key, flag disagreements: ```python for statement in generated_statements: verdict = judge(model, passage, statement) # TRUE / FALSE / NOT GIVEN if verdict != generated_key[statement]: flag_for_review_or_regenerate(statement) ``` **The trained 2B is the best, cheapest verifier here** — training made it a far better *checker* (40%→80%), not just a generator. **3. Review / regenerate** the flagged minority. *Measured:* ≈ **75%** raw → ≈ **85–90%** with this loop. ## Prompt tips - **Always use the tag prefix** `` — it's not a chat model. - **Temperature:** 0.3 for verdicts, 0.7 for passages/writing/speaking; top_p 0.9. - **`enable_thinking=False`** — reasoning mode *lowers* verdict accuracy. - **One SECTION+TYPE per call;** build a full section by generating each type against one shared passage.