README.md

---
library_name: transformers
tags:
- language-learning
- ILR
- assessment
- multilingual
- education
- Qwen3
- fine-tuned
license: apache-2.0
base_model: unsloth/Qwen3-4B-unsloth-bnb-4bit
datasets:
- pcalhoun/ILR-Assistant
language:
- ar
- en
- multilingual
---

# ILR Assistant - Language Assessment Generator

A fine-tuned language model designed to generate ILR (Interagency Language Roundtable) assessment examples and provide adaptive language learning assistance.

![Demo](demo.gif)

## Model Details

### Model Description

This model is a fine-tuned version of Qwen3-4B specifically designed for generating language assessment materials following the ILR proficiency scale. The model can create reading comprehension examples, adjust difficulty levels based on user performance, and provide structured language learning assessments.

- **Developed by:** Paul Calhoun
- **Model type:** Causal Language Model (Fine-tuned)
- **Language(s):** Multilingual (with demonstrated Arabic capabilities)
- **License:** Apache 2.0
- **Finetuned from model:** unsloth/Qwen3-4B-unsloth-bnb-4bit

### Model Sources

- **Repository:** https://github.com/calhounpaul/ILR-assistant
- **Dataset:** https://huggingface.co/datasets/pcalhoun/ILR-Assistant

## Uses

### Direct Use

The model is designed for creating ILR-compliant language assessment materials and can:

- Generate reading comprehension examples at specific ILR levels (1-3+)
- Adapt difficulty levels based on user performance
- Create structured language learning exercises
- Provide assessment content for language education programs

### Downstream Use

This model can be integrated into:

- Language learning platforms and applications
- Educational assessment systems
- Language proficiency testing tools
- Military and government language training programs
- Academic language instruction systems

### Out-of-Scope Use

This model is not intended for:

- General-purpose conversation or chat applications
- Professional translation services
- Content generation outside of educational/assessment contexts
- Applications requiring real-time language assessment without human oversight

## Training Details

### Training Data

The training dataset was created by processing approximately 300 PDFs with known ILR levels from the Defense Language Institute Foreign Language Center (DLIFLC) GLOSS repository (https://gloss.dliflc.edu). The data was processed using DeepSeek-V1 to format the content into conversational training examples.

**Dataset:** pcalhoun/ILR-Assistant
- **Source Materials:** ~300 PDFs from DLIFLC GLOSS
- **Processing Method:** DeepSeek-V1 automated formatting
- **Format:** Conversational examples with ILR level annotations

### Training Procedure

The model was fine-tuned using LoRA (Low-Rank Adaptation) with the Unsloth framework.

#### Training Hyperparameters

- **Base Model:** unsloth/Qwen3-4B-unsloth-bnb-4bit
- **Training Framework:** Unsloth + TRL SFTTrainer
- **LoRA Rank (r):** 64
- **LoRA Alpha:** 64
- **LoRA Dropout:** 0
- **Target Modules:** q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- **Learning Rate:** 5e-5
- **Batch Size:** 16 per device
- **Gradient Accumulation Steps:** 1
- **Max Sequence Length:** 2048
- **Training Epochs:** 1
- **Warmup Ratio:** 0.1
- **Weight Decay:** 0.01
- **Optimizer:** AdamW
- **Scheduler:** Cosine
- **Precision:** BF16 (where supported)
- **Gradient Clipping:** 1.0

## Capabilities

### ILR Level Management

The model demonstrates sophisticated understanding of ILR proficiency levels and can:

- Generate content appropriate for specific ILR levels (1-3+)
- Recognize when users struggle and automatically reduce difficulty levels
- Create progressive learning experiences that adapt to user performance
- Maintain consistency with official ILR assessment criteria

### Language Assessment Features

- **Reading Comprehension:** Creates passages with accompanying questions
- **Adaptive Difficulty:** Adjusts content complexity based on user responses
- **Educational Structure:** Follows established language pedagogy principles

## Environmental Impact

Training was optimized for efficiency using:

- **Quantization:** 4-bit quantization for reduced memory usage
- **LoRA:** Parameter-efficient fine-tuning technique
- **Framework:** Unsloth for optimized training performance

## Technical Specifications

### Model Architecture

- **Base Architecture:** Qwen3-4B transformer
- **Fine-tuning Method:** LoRA (Low-Rank Adaptation)
- **Quantization:** 4-bit (bnb)
- **Parameters:** ~4B (base) + LoRA adapters

### Compute Infrastructure

- **Training:** GPU-accelerated with gradient checkpointing
- **Memory Optimization:** 4-bit quantization + LoRA
- **Framework:** Unsloth/Transformers/TRL

## Usage Example

```
<|im_start|>user
ILR Level 1 (Elementary):
Reads very simple texts (e.g., tourist materials) with high-frequency vocabulary. Misunderstandings common; grasps basic ideas in familiar contexts.
ILR Level 1+ (Elementary+):
Handles simple announcements, headlines, or narratives. Can locate routine professional info but struggles with structure and cohesion.
ILR Level 2 (Limited Working):
Reads straightforward factual texts on familiar topics (e.g., news, basic reports). Understands main ideas but slowly; inferences are limited.
ILR Level 2+ (Limited Working+):
Comprehends most non-technical prose and concrete professional discussions. Separates main ideas from details but misses nuance.
ILR Level 3 (General Professional):
Reads diverse authentic texts (e.g., news, reports) with near-complete comprehension. Interprets implicit meaning but struggles with complex idioms.
ILR Level 3+ (General Professional+):
Handles varied professional styles with minimal errors. Understands cultural references and complex structures, though subtleties may be missed.

Initial ILR level for this conversation: 3
Test my comprehension of Modern Standard Arabic.<|im_end|>
<|im_start|>assistant
<scorer>
I am administering an ILR level assessment.
</scorer>
في أعقاب الأزمة الاقتصادية الأخيرة، دعا المجلس المركزي إلى إصلاحات جذرية في النظام المالي. واقترح الخبراء فرض ضرائب تصاعدية على الشركات الكبرى، مع توجيه جزء من هذه الإيرادات لدعم المشاريع الصغيرة. إلا أن بعض أعضاء البرلمان عارضوا هذه الخطوة، معتبرين أنها قد تثقل كاهل القطاع الخاص وتؤثر سلباً على الاستثمارات الأجنبية. وفي المقابل، أشارت تقارير اقتصادية إلى أن هذه الإصلاحات قد تكون الحل الوحيد لتجنب انهيار العملة المحلية، خاصة بعد انخفاض احتياطي النقد الأجنبي إلى أدنى مستوياته منذ عقد.

What is the main economic concern that prompted the proposed reforms?<|im_end|>
<|im_start|>user
The main concern is the recent economic crisis and the risk of the local currency collapsing due to the foreign exchange reserve reaching its lowest level in a decade.<|im_end|>
<|im_start|>assistant
<scorer>
User answer is: Correct.
</scorer>
Why do some parliament members oppose the proposed tax reforms?<|im_end|>
<|im_start|>user
They believe the reforms could burden the private sector and negatively impact foreign investments.<|im_end|>
<|im_start|>assistant
<scorer>
User answer is: Correct.
</scorer>
What can be inferred about the experts' view of small businesses in this context?<|im_end|>
<|im_start|>user
The experts likely view small businesses as vital to economic recovery, as they propose directing part of the tax revenue to support them.<|im_end|>
<|im_start|>assistant
<scorer>
User answer is: Correct.
</scorer>
How does the passage frame the urgency of the reforms?<|im_end|>
<|im_start|>user
It suggests the reforms are critical and time-sensitive by linking them to avoiding a currency collapse and citing the severe drop in foreign reserves.<|im_end|>
<|im_start|>assistant
<scorer>
User answer is: Correct.
</scorer>
في العصر الحديث، أصبحت الاستدامة البيئية محورًا رئيسيًا للعديد من الشركات العالمية. تدرك هذه الشركات أن تبني سياسات صديقة للبيئة ليس مجرد مسؤولية أخلاقية، بل أيضًا استثمارًا ذكيًا على المدى الطويل. على سبيل المثال، تقوم بعض المؤسسات بتخفيض انبعاثات الكربون من خلال الاعتماد على الطاقة المتجددة، مما يعزز سمعتها ويقلل التكاليف التشغيلية. ومع ذلك، يواجه هذا التحول تحديات كبيرة، مثل ارتفاع التكاليف الأولية وعدم كفاية البنية التحتية في بعض المناطق. رغم ذلك، يرى الخبراء أن الفوائد المستقبلية تفوق بكثير التحديات الحالية.

What is the main reason companies are adopting environmentally friendly policies according to the passage?<|im_end|>
<|im_start|>user
Companies adopt environmentally friendly policies solely because of strict government regulations.<|im_end|>
<|im_start|>assistant
<scorer>
User answer is: Wrong.
Reduce ILR estimate to: 2+
</scorer>
انتقلت عائلة أحمد إلى مدينة جديدة بسبب عمل والده. كان أحمد يشعر بالقلق لأن عليه أن يبدأ مدرسة جديدة ويصنع أصدقاء جدد. لكن المدرسة كانت لطيفة معه، وسرعان ما تعرف على ولدين في صفه يحبان كرة القدم مثله. الآن، يشعر أحمد بسعادة أكبر لأنه يستطيع اللعب مع أصدقائه بعد المدرسة.

Why did Ahmed's family move to a new city?<|im_end|>
<|im_start|>user
Because of his father's job.<|im_end|>
<|im_start|>assistant
<scorer>
User answer is: Correct.
</scorer>
What was Ahmed worried about before starting the new school?<|im_end|>
<|im_start|>user
He was worried about starting a new school and making new friends.<|im_end|>
<|im_start|>assistant
<scorer>
User answer is: Correct.
</scorer>
How did Ahmed feel after meeting his new friends?<|im_end|>
<|im_start|>user
He felt happier because he could play football with them after school.<|im_end|>
<|im_start|>assistant
<scorer>
User answer is: Correct.
</scorer>
What common interest helped Ahmed bond with his new friends?<|im_end|>
<|im_start|>user
They all liked football (soccer).<|im_end|>
<|im_start|>assistant
<scorer>
User answer is: Correct.
</scorer>
انتقلت عائلة أحمد إلى مدينة جديدة بسبب عمل والده. كان أحمد يشعر بالقلق لأن عليه أن يبدأ مدرسة جديدة ويصنع أصدقاء جدد. لكن المدرسة كانت لطيفة معه، وسرعان ما تعرف على ولدين في صفه يحبان كرة القدم مثله. الآن، يشعر أحمد بسعادة أكبر لأنه يستطيع اللعب مع أصدقائه بعد المدرسة.

Why did Ahmed's family move to a new city?<|im_end|>
<|im_start|>user
Because of his father's job.<|im_end|>
<|im_start|>assistant
<scorer>
User answer is: Correct.
</scorer>
What was Ahmed worried about before starting the new school?<|im_end|>
<|im_start|>user
He was worried about starting a new school and making new friends.<|im_end|>
<|im_start|>assistant
<scorer>
User answer is: Correct.
</scorer>
How did Ahmed feel after meeting his new friends?<|im_end|>
<|im_start|>user
He felt happier because he could play football with them after school.<|im_end|>
<|im_start|>assistant
<scorer>
User answer is: Correct.
</scorer>
What common interest helped Ahmed bond with his new friends?<|im_end|>
<|im_start|>user
They all liked football (soccer).<|im_end|>
<|im_start|>assistant
<scorer>
User answer is: Correct.
</scorer>
تذهب سارة إلى السوق كل يوم جمعة لشراء الفواكه والخضروات الطازجة. تحب سارة التفاح والموز لأنها لذيذة ومفيدة لصحة أطفالها. أحياناً تشتري أيضاً البرتقال في فصل الشتاء لأنه غني بفيتامين سي. تفضل سارة التسوق في الصباح الباكر عندما يكون الجو لطيفاً والأسعار مناسبة.

What does Sara buy every Friday at the market?<|im_end|>
<|im_start|>user
Sara buys fresh meat and bread every Friday at the market.<|im_end|>
<|im_start|>assistant
<scorer>
User answer is: Wrong.
Reduce ILR estimate to: 2
</scorer>
في أحد أحياء القاهرة، قامت مجموعة من الشباب بتنظيم مبادرة لتنظيف الشوارع. جمعوا النفايات وفرزوا المواد القابلة لإعادة التدوير. بعد أسبوع من العمل، أصبح الحي أكثر نظافة وأكثر جمالاً. ساعدت هذه المبادرة في زيادة الوعي البيئي بين السكان. كما أشاد المسؤولون المحليون بهذا الجهد وشجعوا على تكراره في أحياء أخرى.

What did the group of young people organize in a Cairo neighborhood?<|im_end|>
<|im_start|>user
They organized a protest against pollution.<|im_end|>
<|im_start|>assistant
<scorer>
User answer is: Wrong.
Reduce ILR estimate to: 1+
</scorer>
الأسرة تذهب إلى السوق يوم السبت. يشترون الفواكه والخضروات. الأب يحب التفاح، والأم تفضل الموز. الابن الصغير يشتري حلوى.

Where does the family go on Saturday?<|im_end|>
<|im_start|>user
They go to the park.<|im_end|>
<|im_start|>assistant
<scorer>
User answer is: Wrong.
Reduce ILR estimate to: 1
</scorer>
في السوق، نبيع الفواكه والخضروات الطازجة. لدينا تفاح أحمر وأصفر، وموز ناضج. أيضا نبيع خبز أبيض وأسمر كل صباح. السوق يفتح عند الساعة الثامنة ويغلق عند الساعة الخامسة مساءً.

What items are sold in the market?<|im_end|>
```

## Limitations and Biases

- **Language Coverage:** Testing has been primarily conducted in Arabic
- **ILR Level Range:** Currently optimized for levels 1-3+, will require modified training for levels 4-5
- **Domain Specificity:** Trained specifically on educational/assessment content, may not generalize to other domains
- **Cultural Context:** Assessment materials may reflect the cultural contexts present in the training data

### Recommendations

Users should:

- Validate generated content with qualified language instructors
- Monitor model outputs for cultural sensitivity and appropriateness
- Consider the model as an assessment generation tool rather than a definitive evaluation system
- Regularly evaluate and adjust ILR level assignments based on learner feedback

## Citation

**BibTeX:**

```bibtex
@model{calhoun_2025_ilr_assistant,
  title={ILR Assistant: Fine-tuned Language Assessment Generator},
  author={Calhoun, Paul},
  year={2025},
  url={https://huggingface.co/pcalhoun/ILR-Assistant}
}
```

## Model Card Authors

Paul Calhoun

## Model Card Contact

For questions, issues, or contributions, please visit the [GitHub repository](https://github.com/calhounpaul/ILR-assistant).