README.md

---
base_model: Qwen/Qwen2.5-0.5B-Instruct
language:
- en
library_name: transformers
license: apache-2.0
pipeline_tag: text-generation
tags:
- qwen
- education
- activity-generation
- supervised-fine-tuning
- lora
- myvillage
---

# Activity Generation Model v0.2

`mjpsm/activity-generation-model-v0.2` is a fine-tuned version of
`Qwen/Qwen2.5-0.5B-Instruct` designed to generate a simple next learning
activity from a student's previous knowledge submission.

The model is part of an activity-generation workflow for the MyVillage
Project. Given a knowledge submission describing something a student has
learned, practiced, created, researched, or experienced, the model
proposes one logical next activity that builds on that submission.

## Model Input

The model accepts a student's previous knowledge submission.

Example:

``` text
updated project management doc for aurora phase 1 now includes srujana's task tracking template
```

## Model Output

The model is trained to produce an activity containing:

-   `title`
-   `description`
-   `instructions`
-   `activityType`

The inference helper used during development returns the result in the
following wrapper:

``` json
{
  "input": {
    "knowledge_submission": "..."
  },
  "output": {
    "title": "...",
    "description": "...",
    "instructions": "...",
    "activityType": "CREATE"
  }
}
```

The generated `activityType` must be one of seven supported activity
categories:

  Activity Type   Purpose
  --------------- ----------------------------------------------
  `REFLECTION`    Think or write about prior learning
  `RESEARCH`      Gather additional information
  `COLLABORATE`   Work with other people
  `CREATE`        Produce a small artifact
  `PRACTICE`      Build or reinforce a skill
  `EXPERIENCE`    Attend, observe, or participate
  `TEACH`         Explain or share knowledge with someone else

## Activity Design

Version 0.2 was trained around activities intended to be small,
actionable next steps rather than large assignments.

Activities are intended to:

-   directly build on the student's previous knowledge submission;
-   be promptly completable;
-   provide clear and compact instructions;
-   avoid unnecessarily large assignments such as multi-page reports,
    full applications, complete websites, or major projects;
-   select an activity type that fits the generated activity.

## Changes from v0.1

Version 0.2 represents a revised training objective and dataset.

The primary output contract is:

``` text
knowledge_submission
        ↓
activity-generation-model-v0.2
        ↓
title
description
instructions
activityType
```

`estimatedMinutes`, which was part of the earlier activity-generation
target, was intentionally removed from the v0.2 output.

Keeping v0.2 separate from v0.1 preserves the earlier model as a
baseline and makes it possible to compare versions without overwriting
the previous model.

## Training

The model was fine-tuned using supervised fine-tuning with LoRA.

Known training configuration:

  Setting                          Value
  -------------------------------- -------------------------------
  Base model                       `Qwen/Qwen2.5-0.5B-Instruct`
  Fine-tuning method               LoRA / supervised fine-tuning
  Epochs                           5
  Learning rate                    `2e-4`
  LR scheduler                     Cosine
  Warmup steps                     5
  Weight decay                     `0.01`
  Per-device training batch size   2
  Gradient accumulation steps      8
  Maximum sequence length          1024
  Evaluation strategy              Every epoch
  Checkpoint strategy              Every epoch
  Best-model metric                Validation loss
  `load_best_model_at_end`         `True`

The training configuration selected the checkpoint with the **lowest
validation loss**, rather than automatically using the final epoch.

## Observed Training Results

The following metrics were observed during the v0.2 training run:

  --------------------------------------------------------------------------
           Epoch  Training Loss     Validation        Entropy     Mean Token
                                          Loss                      Accuracy
  -------------- -------------- -------------- -------------- --------------
               1       0.881803       0.830671       0.849002       0.796495

               2       0.738581   **0.792472**       0.747979       0.801279

               3       0.655374       0.794525       0.672648       0.801776

               4       0.605030       0.808197       0.643972       0.800806
  --------------------------------------------------------------------------

Among the results recorded in this development session, Epoch 2 had the
lowest observed validation loss (`0.792472`). The training setup used
`load_best_model_at_end=True` and `metric_for_best_model="eval_loss"`.

These token-level metrics should not be interpreted as an overall
percentage of pedagogically correct activities. Generation quality,
structural validity, relevance, and activity-type selection should be
evaluated separately.

## Example Generation

One observed v0.2 generation during development used this input:

``` text
updated project management doc for aurora phase 1 now includes srujana's task tracking template
```

and returned:

``` json
{
  "input": {
    "knowledge_submission": "updated project management doc for aurora phase 1 now includes srujana's task tracking template"
  },
  "output": {
    "title": "Task Tracking Template Refinement",
    "description": "Refine Srujana's task tracking template to improve accuracy.",
    "instructions": "Open the updated project management doc for Aurora Phase 1. Review the existing task tracking template. Identify any areas where you think it could be improved. Write down your suggestions in a short note (1-2 paragraphs) and suggest changes.",
    "activityType": "CREATE"
  }
}
```

This example is included to demonstrate the model's output structure. It
should not be treated as a benchmark result.

## Python Usage

Install the required packages:

``` bash
pip install torch transformers accelerate
```

Then load the model with Transformers:

``` python
import json
import re
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "mjpsm/activity-generation-model-v0.2"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)

model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    torch_dtype="auto",
    device_map="auto",
)

model.eval()

SYSTEM_PROMPT = """You are an educational activity generator.

Given a student's previous knowledge submission, generate exactly one
simple next learning activity that directly builds on what the student
demonstrated.

Return valid JSON only with exactly these fields:
title, description, instructions, activityType.

activityType must be exactly one of:
REFLECTION, RESEARCH, COLLABORATE, CREATE, PRACTICE, EXPERIENCE, TEACH.
"""


def generate_activity(knowledge_submission: str):
    messages = [
        {
            "role": "system",
            "content": SYSTEM_PROMPT,
        },
        {
            "role": "user",
            "content": (
                "Knowledge submission:\n"
                f"{knowledge_submission}\n\n"
                "Generate the next activity."
            ),
        },
    ]

    prompt = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True,
    )

    inputs = tokenizer(
        prompt,
        return_tensors="pt",
    ).to(model.device)

    with torch.no_grad():
        generated = model.generate(
            **inputs,
            max_new_tokens=300,
            do_sample=False,
            pad_token_id=tokenizer.eos_token_id,
        )

    new_tokens = generated[0][inputs["input_ids"].shape[1]:]

    text = tokenizer.decode(
        new_tokens,
        skip_special_tokens=True,
    ).strip()

    # Extract the first JSON object from the generated response.
    match = re.search(r"\{.*\}", text, re.DOTALL)

    if not match:
        raise ValueError(
            f"Model did not return a JSON object. Raw output: {text}"
        )

    activity = json.loads(match.group(0))

    required_fields = {
        "title",
        "description",
        "instructions",
        "activityType",
    }

    missing = required_fields - set(activity.keys())

    if missing:
        raise ValueError(
            f"Generated activity is missing fields: {sorted(missing)}"
        )

    return activity


activity = generate_activity(
    "I practiced Python functions and learned how parameters and return values work."
)

print(json.dumps(activity, indent=2))
```

A typical response is expected to follow this shape:

``` json
{
  "title": "Practice Reusable Functions",
  "description": "Apply your understanding of parameters and return values in a small exercise.",
  "instructions": "Write three short Python functions that accept inputs and return a result. Test each function with at least two different inputs.",
  "activityType": "PRACTICE"
}
```

Generated wording and activity type can vary by input.

## Intended Use

This model is intended for educational activity generation where a
previous student knowledge submission is available and a small next
learning step needs to be proposed.

Potential use cases include:

-   generating a follow-up activity after a knowledge submission;
-   supporting adaptive learning workflows;
-   proposing a next step that extends prior learning;
-   generating structured activity data for downstream application
    logic.

## Evaluation Status

A held-out generation evaluation was being developed for v0.2. During
that process, an initial evaluator incorrectly treated the model's
nested `{ "input": ..., "output": ... }` return structure as though the
activity fields were at the top level. Those resulting `predicted=None`
values were evaluator errors and are **not reported here as model
accuracy results**.

A complete held-out generation benchmark---including exact activity-type
accuracy and per-class accuracy---has not been established in this model
card. It should be added after the corrected evaluator is run
successfully.

## Limitations

-   Training examples include synthetically generated activities.
-   Token-level accuracy does not measure whether an activity is
    educationally optimal.
-   The generated `activityType` may not always be the only reasonable
    category for a given activity.
-   Generated activities should be evaluated for relevance and
    appropriateness before being used in higher-stakes educational
    settings.
-   The model may occasionally generate activities that are more
    involved than the intended small-task format.
-   The model is designed to generate one next activity rather than a
    long-term learning plan.
-   Performance on knowledge submissions substantially outside the
    training distribution has not yet been fully characterized.

## Version History

### v0.2

-   Revamped activity-generation dataset
-   Output simplified to four fields
-   Removed `estimatedMinutes`
-   Seven supported activity types
-   Small next-step activity design
-   Best-checkpoint selection based on validation loss

### v0.1

Initial activity-generation model.