--- language: - en license: apache-2.0 base_model: - Qwen/Qwen2.5-0.5B-Instruct library_name: transformers pipeline_tag: text-generation tags: - qwen - qwen2.5 - transformers - peft - lora - fine-tuned - education - activity-generation - myvillage datasets: - synthetic model-index: - name: Activity Generation Model v1 results: [] --- # Activity Generation Model v1 `activity-generation-model-v1` is a fine-tuned version of **Qwen2.5-0.5B-Instruct** designed to generate the next educational activity for a learner based on three pieces of context: 1. The learner's **village goal** 2. The learner's **previous activity** 3. The learner's **knowledge submission** The model generates a structured activity containing: * `title` * `description` * `instructions` The model was developed for the **MyVillage** learning environment, where learner knowledge submissions can be used to determine an appropriate next activity that builds on demonstrated knowledge while continuing progress toward a larger village goal. ## Model Details * **Base Model:** `Qwen/Qwen2.5-0.5B-Instruct` * **Model Type:** Causal Language Model * **Approximate Base Model Size:** 0.5B parameters * **Fine-Tuning Method:** Supervised Fine-Tuning with LoRA * **Frameworks:** Transformers, TRL, PEFT * **Primary Language:** English * **Task:** Educational activity generation * **Output Format:** JSON The LoRA adapter was merged into the base Qwen model after training, producing a standalone model that can be loaded directly with Hugging Face Transformers. ## Model Inputs The model expects three pieces of information. ### Village Goal The larger learning objective that the student's activities should move toward. ### Previous Activity The title of the activity the student completed before submitting their knowledge. ### Knowledge Submission A description of what the student learned, completed, discovered, or demonstrated during the previous activity. The expected prompt structure is: ```text Village goal: {village_goal} Previous activity: {previous_activity_title} Knowledge submission: {knowledge_submission} Create the student's next activity. ``` ## Model Output The model is trained to return valid JSON using exactly the following schema: ```json { "title": "activity title", "description": "activity description", "instructions": "activity instructions" } ``` The generated activity should build directly on the learner's knowledge submission while continuing to move the learner toward the village goal. ## Training Data The model was fine-tuned using a synthetic activity-generation dataset containing **1,000 examples**. Each training record follows the general structure: ```json { "input": { "village_goal": "...", "previous_activity_title": "...", "knowledge_submission": "..." }, "output": { "title": "...", "description": "...", "instructions": "..." } } ``` The training task teaches the model to map learner context to an appropriate next educational activity. ## Fine-Tuning Configuration The model was fine-tuned using LoRA with the following configuration: ```text Base model: Qwen/Qwen2.5-0.5B-Instruct Epochs: 3 Maximum sequence length: 1024 Learning rate: 2e-4 Learning-rate scheduler: Cosine Per-device training batch size: 4 Gradient accumulation steps: 4 Effective batch size: 16 Weight decay: 0.01 LoRA rank: 16 LoRA alpha: 32 LoRA dropout: 0.05 Random seed: 42 ``` LoRA was applied to the following modules: ```text q_proj k_proj v_proj o_proj gate_proj up_proj down_proj ``` The best checkpoint was selected using validation loss. After training, the LoRA adapter was merged into the original Qwen2.5-0.5B-Instruct model using PEFT's `merge_and_unload()` functionality. ## System Prompt The model was fine-tuned using the following system-level behavior: ```text You are an educational activity generator for MyVillage. Your job is to create exactly one logical next learning activity for a student. You will receive: 1. The goal of the student's village. 2. The title of the student's previous activity. 3. The student's knowledge submission describing what they learned or completed. Create a new activity that: - directly builds on the student's knowledge submission; - moves the student toward the village goal; - does not simply repeat the previous activity; - is specific and actionable; - uses clear student-facing language; - includes a concrete task or deliverable. Return valid JSON only. Return exactly these fields: { "title": "activity title", "description": "activity description", "instructions": "activity instructions" } Do not include markdown. Do not include commentary. Do not include additional fields. ``` ## How to Use Install the required packages: ```bash pip install transformers accelerate torch ``` Then load the model with Hugging Face Transformers. > Replace `mjpsm/activity-generation-model-v1` if your Hugging Face repository uses a different name. ```python import json import torch from transformers import AutoModelForCausalLM, AutoTokenizer MODEL_ID = "mjpsm/activity-generation-model-v1" tokenizer = AutoTokenizer.from_pretrained(MODEL_ID) model = AutoModelForCausalLM.from_pretrained( MODEL_ID, torch_dtype="auto", device_map="auto", ) model.eval() ``` Define the system prompt: ```python SYSTEM_PROMPT = """You are an educational activity generator for MyVillage. Your job is to create exactly one logical next learning activity for a student. You will receive: 1. The goal of the student's village. 2. The title of the student's previous activity. 3. The student's knowledge submission describing what they learned or completed. Create a new activity that: - directly builds on the student's knowledge submission; - moves the student toward the village goal; - does not simply repeat the previous activity; - is specific and actionable; - uses clear student-facing language; - includes a concrete task or deliverable. Return valid JSON only. Return exactly these fields: { "title": "activity title", "description": "activity description", "instructions": "activity instructions" } Do not include markdown. Do not include commentary. Do not include additional fields. """ ``` Create a generation function: ```python def generate_activity( village_goal, previous_activity_title, knowledge_submission, max_new_tokens=300, ): user_message = f"""Village goal: {village_goal} Previous activity: {previous_activity_title} Knowledge submission: {knowledge_submission} Create the student's next activity.""" messages = [ { "role": "system", "content": SYSTEM_PROMPT, }, { "role": "user", "content": user_message, }, ] prompt = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, ) inputs = tokenizer( prompt, return_tensors="pt", ).to(model.device) with torch.no_grad(): outputs = model.generate( **inputs, max_new_tokens=max_new_tokens, do_sample=False, repetition_penalty=1.05, pad_token_id=tokenizer.pad_token_id, eos_token_id=tokenizer.eos_token_id, ) generated_tokens = outputs[ 0, inputs["input_ids"].shape[1]: ] response = tokenizer.decode( generated_tokens, skip_special_tokens=True, ).strip() try: return json.loads(response) except json.JSONDecodeError: return { "error": "The model did not return valid JSON.", "raw_output": response, } ``` Generate an activity: ```python activity = generate_activity( village_goal="Learn how to build and train machine learning models", previous_activity_title="Create a phishing URL dataset", knowledge_submission=""" I created a dataset containing legitimate and phishing URLs. I loaded it into Google Colab and cleaned several missing values. I have not trained a machine learning model with it yet. """, ) print( json.dumps( activity, indent=2, ensure_ascii=False, ) ) ``` An output may look similar to: ```json { "title": "Train a Phishing URL Classification Model", "description": "Build on your cleaned phishing URL dataset by training a machine learning model that can distinguish between legitimate and phishing URLs.", "instructions": "Split your cleaned dataset into training and testing sets. Select a classification algorithm, train the model using the training data, and evaluate its performance on the testing data. Record the model's accuracy and describe what the results tell you." } ``` ## Input and Output Flow Conceptually, the model performs the following transformation: ```text Village Goal + Previous Activity + Knowledge Submission | v Activity Generation Model | v { "title": "...", "description": "...", "instructions": "..." } ``` The village goal provides the model with the learner's broader direction, while the previous activity and knowledge submission provide immediate context about the learner's current position. ## Intended Use This model is intended for experimental educational activity generation. Potential uses include: * Generating personalized next-step learning activities * Building learning progression systems * Supporting activity recommendation workflows * Generating activities from demonstrated learner knowledge * Prototyping adaptive learning systems The model is particularly designed for workflows where a learner completes an activity, submits what they learned, and then receives a new activity that extends that learning. ## Limitations This is an experimental model and should not be treated as an autonomous educational decision-making system. The model may: * Generate activities that are too broad or too narrow * Produce activities that do not perfectly align with a village goal * Occasionally repeat concepts from a previous activity * Generate invalid JSON * Produce instructions that require additional clarification * Perform less reliably on inputs substantially different from its training distribution The model was trained using synthetic examples, so performance on real-world learner submissions should be evaluated separately. Generated activities should be reviewed before being used in high-stakes educational settings. ## What This Model Does Not Predict This version of the model does **not** predict: * Activity type * Estimated activity duration * Student mastery score * Activity completion status Its responsibility is limited to generating: ```text title description instructions ``` Other attributes can be generated or classified by separate models or application logic. ## Future Work Potential future improvements include: * Evaluation using real MyVillage knowledge submissions * Larger and more diverse training datasets * Automated activity-quality evaluation * Village-goal alignment scoring * Knowledge progression evaluation * Comparison against larger teacher models * Integration with a separate activity-type classification model * API deployment for application integration ## Base Model This model was fine-tuned from: **Qwen2.5-0.5B-Instruct** Developed by the Qwen team. Base model repository: `Qwen/Qwen2.5-0.5B-Instruct` ## Disclaimer This model is a research and development prototype. Generated activities should be evaluated for relevance, accuracy, appropriateness, and educational quality before being presented to learners.