README.md

---
language:
  - en

license: apache-2.0

base_model:
  - Qwen/Qwen2.5-0.5B-Instruct

library_name: transformers

pipeline_tag: text-generation

tags:
  - qwen
  - qwen2.5
  - transformers
  - peft
  - lora
  - fine-tuned
  - education
  - activity-generation
  - myvillage

datasets:
  - synthetic

model-index:
  - name: Activity Generation Model v1
    results: []
---

# Activity Generation Model v1

`activity-generation-model-v1` is a fine-tuned version of **Qwen2.5-0.5B-Instruct** designed to generate the next educational activity for a learner based on three pieces of context:

1. The learner's **village goal**
2. The learner's **previous activity**
3. The learner's **knowledge submission**

The model generates a structured activity containing:

* `title`
* `description`
* `instructions`

The model was developed for the **MyVillage** learning environment, where learner knowledge submissions can be used to determine an appropriate next activity that builds on demonstrated knowledge while continuing progress toward a larger village goal.

## Model Details

* **Base Model:** `Qwen/Qwen2.5-0.5B-Instruct`
* **Model Type:** Causal Language Model
* **Approximate Base Model Size:** 0.5B parameters
* **Fine-Tuning Method:** Supervised Fine-Tuning with LoRA
* **Frameworks:** Transformers, TRL, PEFT
* **Primary Language:** English
* **Task:** Educational activity generation
* **Output Format:** JSON

The LoRA adapter was merged into the base Qwen model after training, producing a standalone model that can be loaded directly with Hugging Face Transformers.

## Model Inputs

The model expects three pieces of information.

### Village Goal

The larger learning objective that the student's activities should move toward.

### Previous Activity

The title of the activity the student completed before submitting their knowledge.

### Knowledge Submission

A description of what the student learned, completed, discovered, or demonstrated during the previous activity.

The expected prompt structure is:

```text
Village goal:
{village_goal}

Previous activity:
{previous_activity_title}

Knowledge submission:
{knowledge_submission}

Create the student's next activity.
```

## Model Output

The model is trained to return valid JSON using exactly the following schema:

```json
{
  "title": "activity title",
  "description": "activity description",
  "instructions": "activity instructions"
}
```

The generated activity should build directly on the learner's knowledge submission while continuing to move the learner toward the village goal.

## Training Data

The model was fine-tuned using a synthetic activity-generation dataset containing **1,000 examples**.

Each training record follows the general structure:

```json
{
  "input": {
    "village_goal": "...",
    "previous_activity_title": "...",
    "knowledge_submission": "..."
  },
  "output": {
    "title": "...",
    "description": "...",
    "instructions": "..."
  }
}
```

The training task teaches the model to map learner context to an appropriate next educational activity.

## Fine-Tuning Configuration

The model was fine-tuned using LoRA with the following configuration:

```text
Base model: Qwen/Qwen2.5-0.5B-Instruct
Epochs: 3
Maximum sequence length: 1024
Learning rate: 2e-4
Learning-rate scheduler: Cosine
Per-device training batch size: 4
Gradient accumulation steps: 4
Effective batch size: 16
Weight decay: 0.01
LoRA rank: 16
LoRA alpha: 32
LoRA dropout: 0.05
Random seed: 42
```

LoRA was applied to the following modules:

```text
q_proj
k_proj
v_proj
o_proj
gate_proj
up_proj
down_proj
```

The best checkpoint was selected using validation loss.

After training, the LoRA adapter was merged into the original Qwen2.5-0.5B-Instruct model using PEFT's `merge_and_unload()` functionality.

## System Prompt

The model was fine-tuned using the following system-level behavior:

```text
You are an educational activity generator for MyVillage.

Your job is to create exactly one logical next learning activity for a student.

You will receive:

1. The goal of the student's village.
2. The title of the student's previous activity.
3. The student's knowledge submission describing what they learned or completed.

Create a new activity that:

- directly builds on the student's knowledge submission;
- moves the student toward the village goal;
- does not simply repeat the previous activity;
- is specific and actionable;
- uses clear student-facing language;
- includes a concrete task or deliverable.

Return valid JSON only.

Return exactly these fields:

{
  "title": "activity title",
  "description": "activity description",
  "instructions": "activity instructions"
}

Do not include markdown.
Do not include commentary.
Do not include additional fields.
```

## How to Use

Install the required packages:

```bash
pip install transformers accelerate torch
```

Then load the model with Hugging Face Transformers.

> Replace `mjpsm/activity-generation-model-v1` if your Hugging Face repository uses a different name.

```python
import json
import torch

from transformers import AutoModelForCausalLM, AutoTokenizer


MODEL_ID = "mjpsm/activity-generation-model-v1"


tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)

model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    torch_dtype="auto",
    device_map="auto",
)

model.eval()
```

Define the system prompt:

```python
SYSTEM_PROMPT = """You are an educational activity generator for MyVillage.

Your job is to create exactly one logical next learning activity for a student.

You will receive:

1. The goal of the student's village.
2. The title of the student's previous activity.
3. The student's knowledge submission describing what they learned or completed.

Create a new activity that:

- directly builds on the student's knowledge submission;
- moves the student toward the village goal;
- does not simply repeat the previous activity;
- is specific and actionable;
- uses clear student-facing language;
- includes a concrete task or deliverable.

Return valid JSON only.

Return exactly these fields:

{
  "title": "activity title",
  "description": "activity description",
  "instructions": "activity instructions"
}

Do not include markdown.
Do not include commentary.
Do not include additional fields.
"""
```

Create a generation function:

```python
def generate_activity(
    village_goal,
    previous_activity_title,
    knowledge_submission,
    max_new_tokens=300,
):

    user_message = f"""Village goal:
{village_goal}

Previous activity:
{previous_activity_title}

Knowledge submission:
{knowledge_submission}

Create the student's next activity."""

    messages = [
        {
            "role": "system",
            "content": SYSTEM_PROMPT,
        },
        {
            "role": "user",
            "content": user_message,
        },
    ]

    prompt = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True,
    )

    inputs = tokenizer(
        prompt,
        return_tensors="pt",
    ).to(model.device)

    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=max_new_tokens,
            do_sample=False,
            repetition_penalty=1.05,
            pad_token_id=tokenizer.pad_token_id,
            eos_token_id=tokenizer.eos_token_id,
        )

    generated_tokens = outputs[
        0,
        inputs["input_ids"].shape[1]:
    ]

    response = tokenizer.decode(
        generated_tokens,
        skip_special_tokens=True,
    ).strip()

    try:
        return json.loads(response)

    except json.JSONDecodeError:
        return {
            "error": "The model did not return valid JSON.",
            "raw_output": response,
        }
```

Generate an activity:

```python
activity = generate_activity(
    village_goal="Learn how to build and train machine learning models",

    previous_activity_title="Create a phishing URL dataset",

    knowledge_submission="""
    I created a dataset containing legitimate and phishing URLs.
    I loaded it into Google Colab and cleaned several missing values.
    I have not trained a machine learning model with it yet.
    """,
)

print(
    json.dumps(
        activity,
        indent=2,
        ensure_ascii=False,
    )
)
```

An output may look similar to:

```json
{
  "title": "Train a Phishing URL Classification Model",
  "description": "Build on your cleaned phishing URL dataset by training a machine learning model that can distinguish between legitimate and phishing URLs.",
  "instructions": "Split your cleaned dataset into training and testing sets. Select a classification algorithm, train the model using the training data, and evaluate its performance on the testing data. Record the model's accuracy and describe what the results tell you."
}
```

## Input and Output Flow

Conceptually, the model performs the following transformation:

```text
Village Goal
      +
Previous Activity
      +
Knowledge Submission
      |
      v
Activity Generation Model
      |
      v
{
  "title": "...",
  "description": "...",
  "instructions": "..."
}
```

The village goal provides the model with the learner's broader direction, while the previous activity and knowledge submission provide immediate context about the learner's current position.

## Intended Use

This model is intended for experimental educational activity generation.

Potential uses include:

* Generating personalized next-step learning activities
* Building learning progression systems
* Supporting activity recommendation workflows
* Generating activities from demonstrated learner knowledge
* Prototyping adaptive learning systems

The model is particularly designed for workflows where a learner completes an activity, submits what they learned, and then receives a new activity that extends that learning.

## Limitations

This is an experimental model and should not be treated as an autonomous educational decision-making system.

The model may:

* Generate activities that are too broad or too narrow
* Produce activities that do not perfectly align with a village goal
* Occasionally repeat concepts from a previous activity
* Generate invalid JSON
* Produce instructions that require additional clarification
* Perform less reliably on inputs substantially different from its training distribution

The model was trained using synthetic examples, so performance on real-world learner submissions should be evaluated separately.

Generated activities should be reviewed before being used in high-stakes educational settings.

## What This Model Does Not Predict

This version of the model does **not** predict:

* Activity type
* Estimated activity duration
* Student mastery score
* Activity completion status

Its responsibility is limited to generating:

```text
title
description
instructions
```

Other attributes can be generated or classified by separate models or application logic.

## Future Work

Potential future improvements include:

* Evaluation using real MyVillage knowledge submissions
* Larger and more diverse training datasets
* Automated activity-quality evaluation
* Village-goal alignment scoring
* Knowledge progression evaluation
* Comparison against larger teacher models
* Integration with a separate activity-type classification model
* API deployment for application integration

## Base Model

This model was fine-tuned from:

**Qwen2.5-0.5B-Instruct**

Developed by the Qwen team.

Base model repository:

`Qwen/Qwen2.5-0.5B-Instruct`

## Disclaimer

This model is a research and development prototype. Generated activities should be evaluated for relevance, accuracy, appropriateness, and educational quality before being presented to learners.