Upload 10 files

README.md

@@ -0,0 +1,58 @@
+Tiny-GPT2 Text Generation Project
+This repository provides resources to run and fine-tune the sshleifer/tiny-gpt2 model locally on a CPU, suitable for laptops with 8GB or 16GB RAM. The goal is to enable students to learn about AI model workings, experiment, and conduct research.
+Prerequisites
+
+Python: Version 3.10.9 recommended (3.9.10 also works).
+Hardware: Minimum 8GB RAM, CPU-only (GPU optional but not required).
+Hugging Face Account: Required for downloading model weights (create at huggingface.co).
+
+Setup Instructions
+
+Create a Virtual Environment:
+python -m venv venv
+source venv/bin/activate  # On Windows: venv\Scripts\activate
+
+
+Install Libraries:
+pip install torch==2.3.0 transformers==4.38.2 huggingface_hub==0.22.2 datasets==2.21.0 numpy==1.26.4
+
+
+Download Model Weights:
+
+Copy download_model.py from the repository to your project folder.
+Replace YOUR_HUGGINGFACE_API_TOKEN with your Hugging Face token (from huggingface.co/settings/tokens).
+Run:python download_model.py
+
+
+
+
+Test the Model:
+
+Copy test_model.py to your project folder.
+Run:python test_model.py
+
+
+Expected output: Generated text starting with "Once upon a time".
+
+
+Fine-Tune the Model:
+
+Navigate to the fine_tune folder.
+Add your dataset as sample_data.txt (or use the provided example).
+Run:python fine_tune_model.py
+
+
+The fine-tuned model will be saved in fine_tuned_model.
+
+
+
+Notes for GPU Users
+
+The scripts are configured to run on CPU (CUDA_VISIBLE_DEVICES="" in fine_tune_model.py).
+To use a GPU (if available), remove os.environ["CUDA_VISIBLE_DEVICES"] = "" and no_cuda=True from fine_tune_model.py. Ensure your PyTorch installation supports CUDA (run pip install torch==2.3.0+cu121 for GPU support).
+
+Troubleshooting
+
+Memory Issues: If you have 8GB RAM, ensure no other heavy applications are running.
+Library Conflicts: Use the exact versions listed above to avoid compatibility issues.
+File Not Found: Verify the model files are in tiny-gpt2-model/models--sshleifer--tiny-gpt2/snapshots/5f91d94bd9cd7190a9f3216ff93cd1dd95f2c7be.

document.txt

@@ -0,0 +1,203 @@
+Tiny-GPT2 Text Generation Project Documentation
+=============================================
+
+This project enables students to run, fine-tune, and experiment with the `sshleifer/tiny-gpt2` 
+model locally on a laptop with 8GB or 16GB RAM, using CPU (GPU optional). The goal is to provide 
+hands-on experience with AI model workflows, including downloading, fine-tuning, and deploying a 
+text generation model via a web interface. This document covers all steps to set up and run the 
+project, with credits to the original model and organization.
+
+---
+
+1. Project Overview
+The project uses the `sshleifer/tiny-gpt2` model, a lightweight version of GPT-2, for text generation. 
+It includes scripts to:
+- Download model weights from Hugging Face.
+- Test the model with a sample prompt.
+- Fine-tune the model on a custom dataset.
+- Deploy a web app to generate text interactively.
+The setup is optimized for low-memory systems (8GB RAM) and defaults to CPU execution, but includes 
+instructions for GPU users.
+
+---
+
+2. Prerequisites
+- Hardware: Laptop with at least 8GB RAM (16GB recommended). GPU (e.g., NVIDIA GTX) is optional; 
+  scripts default to CPU.
+- Operating System: Windows, macOS, or Linux.
+- Software:
+  - Python 3.10.9 (recommended) or 3.9.10. Download from https://www.python.org/downloads/.
+  - Visual Studio Code (VS Code) for development (optional but recommended). Download from 
+    https://code.visualstudio.com/.
+- Hugging Face Account: Required to download model weights.
+
+---
+
+3. Step-by-Step Setup Instructions
+
+.1. Obtain a Hugging Face Token
+1. Go to https://huggingface.co/ and sign up or log in.
+2. Navigate to https://huggingface.co/settings/tokens.
+3. Click "New token", select "Read" or "Write" access, and copy the token 
+   (e.g., hf_XXXXXXXXXXXXXXXXXXXXXXXXXX).
+4. Store the token securely; you’ll use it in the download script.
+
+3.2. Install Python
+1. Download Python 3.10.9 from https://www.python.org/downloads/release/python-3109/.
+2. Install Python, ensuring "Add Python to PATH" is checked.
+3. Verify installation in a terminal:
+   ```
+   python --version
+   ```
+   Expected output: Python 3.10.9
+
+3.3. Set Up a Virtual Environment
+1. Open a terminal in your project folder (e.g., C:\Users\YourName\Documents\project).
+2. Create a virtual environment:
+   ```
+   python -m venv venv
+   ```
+3. Activate the virtual environment:
+   - Windows: `venv\Scripts\activate`
+   - macOS/Linux: `source venv/bin/activate`
+4. Confirm activation (you’ll see `(venv)` in the terminal prompt).
+
+3.4. Install Dependencies
+1. In the activated virtual environment, create a file named `requirements.txt` with the following 
+   content:
+   ```
+   torch==2.3.0
+   transformers==4.38.2
+   huggingface_hub==0.22.2
+   datasets==2.21.0
+   numpy==1.26.4
+   matplotlib==3.8.3
+   flask==3.0.3
+   ```
+2. Install the libraries:
+   ```
+   pip install -r requirements.txt
+   ```
+3. For GPU users (optional):
+   - Uninstall CPU PyTorch: `pip uninstall torch -y`
+   - Install GPU PyTorch: `pip install torch==2.3.0+cu121`
+   - Verify CUDA: `python -c "import torch; print(torch.cuda.is_available())"` (should print `True`).
+   Note: Scripts default to CPU, so GPU users don’t need to change this unless desired.
+
+3.5. Download Model Weights
+1. Create a folder named `dalle` (or any name) for the project.
+2. Copy the `download_model.py` script from the repository (or create it):
+   ```
+   from transformers import AutoModelForCausalLM, AutoTokenizer
+   from huggingface_hub import login
+   import os
+
+   hf_token = "YOUR_HUGGINGFACE_TOKEN"  # Replace with your token
+   login(token=hf_token)
+
+   model_name = "sshleifer/tiny-gpt2"
+   save_directory = "./tiny-gpt2-model"
+   os.makedirs(save_directory, exist_ok=True)
+
+   model = AutoModelForCausalLM.from_pretrained(model_name, cache_dir=save_directory)
+   tokenizer = AutoTokenizer.from_pretrained(model_name, cache_dir=save_directory)
+   print(f"Model and tokenizer downloaded to {save_directory}")
+   ```
+3. Replace `YOUR_HUGGINGFACE_TOKEN` with your Hugging Face token.
+4. Run the script:
+   ```
+   python download_model.py
+   ```
+5. Verify the model files in 
+`tiny-gpt2-model/models--sshleifer--tiny-gpt2/snapshots/5f91d94bd9cd7190a9f3216ff93cd1dd95f2c7be` 
+ (contains `config.json`, `pytorch_model.bin`, `vocab.json`, `merges.txt`).
+
+3.6. Test the Model
+1. Copy the `test_model.py` script from the repository to the `dalle` folder.
+2. Run the script:
+   ```
+   python test_model.py
+   ```
+3. Expected output: Generated text starting with "Once upon a time" (e.g., may be semi-coherent due 
+   to the model’s small size).
+
+3.7. Fine-Tune the Model
+1. Create a `fine_tune` folder inside `dalle`:
+   ```
+   mkdir fine_tune
+   cd fine_tune
+   ```
+2. Create a dataset file `sample_data.txt` (or use your own text). Example content:
+   ```
+   Once upon a time, there was a brave knight who explored a magical forest.
+   The forest was filled with mystical creatures and ancient ruins.
+   The knight discovered a hidden treasure guarded by a wise dragon.
+   With courage and wisdom, the knight befriended the dragon and shared the treasure with the village.
+   ```
+3. Copy the `fine_tune_model.py` script from the repository to `fine_tune`.
+4. Run the script:
+   ```
+   python fine_tune_model.py
+   ```
+5. The script fine-tunes the model, saves it to `fine_tuned_model`, and generates a `loss_plot.png` 
+   showing training loss.
+6. Verify `fine_tuned_model` contains model files and check `loss_plot.png`.
+
+3.8. Run the Web App
+1. In the `fine_tune` folder, copy `app.py` and create a `templates` folder with `index.html` from the 
+   repository.
+2. Run the web app:
+   ```
+   python app.py
+   ```
+3. Open a browser and go to `http://127.0.0.1:5000`.
+4. Enter a prompt (e.g., "Once upon a time") and click "Generate Text" to see the output from the 
+   fine-tuned model.
+
+---
+
+4. Notes for Students
+- Model Limitations: `tiny-gpt2` is a small model, so generated text may not be highly coherent. For 
+  better results, consider larger models like `gpt2` (requires more memory or GPU).
+- Memory Management: On 8GB RAM systems, close other applications to free memory. The scripts use a 
+  small batch size to minimize memory usage.
+- GPU Support: Scripts default to CPU for compatibility. To use an NVIDIA GPU:
+  - Install `torch==2.3.0+cu121` (see step 3.4).
+  - Remove `os.environ["CUDA_VISIBLE_DEVICES"] = ""` from `fine_tune_model.py` and `app.py`.
+  - Change `use_cpu=True` to `use_cpu=False` in `fine_tune_model.py`.
+- Experimentation: Try different prompts, datasets, or fine-tuning parameters (e.g., `num_train_epochs`,
+ `learning_rate`) to explore AI model behavior.
+
+---
+
+5. Troubleshooting
+- Library Conflicts: Use the exact versions in `requirements.txt` to avoid issues.
+- File Not Found: Ensure model files are in the correct path 
+  (`tiny-gpt2-model/models--sshleifer--tiny-gpt2/snapshots/5f91d94bd9cd7190a9f3216ff93cd1dd95f2c7be`).
+- Memory Errors: Reduce `max_length` in `fine_tune_model.py` (e.g., from 128 to 64) for 8GB RAM systems.
+- Hugging Face Token Issues: Verify your token has "Read" or "Write" access at
+  https://huggingface.co/settings/tokens.
+
+---
+
+6. Credits and Attribution
+- Original Model: `sshleifer/tiny-gpt2`, a distilled version of GPT-2, created by Steven Shleifer. 
+  Available at https://huggingface.co/sshleifer/tiny-gpt2.
+- Organization: Hugging Face, Inc. (https://huggingface.co/) provides the model weights, `transformers`
+  library, and `huggingface_hub` for model access.
+- Project Creator: Remiai3 (GitHub/Hugging Face username). This project was developed to facilitate AI 
+  learning and experimentation for students.
+- AI Assistance: Grok 3, created by xAI (https://x.ai/), assisted in generating and debugging the code,
+  ensuring compatibility for low-resource systems.
+
+---
+
+ 7. Next Steps for Students
+- Experiment with different datasets in `sample_data.txt` to fine-tune the model for specific tasks 
+  (e.g., storytelling, dialogue).
+- Modify `fine_tune_model.py` parameters (e.g., `learning_rate`, `num_train_epochs`) to study their 
+  impact.
+- Enhance `index.html` or `app.py` to add features like multiple prompt inputs or generation options.
+- Explore larger models on Hugging Face (e.g., `gpt2-medium`) if you have a GPU or more RAM.
+
+For questions or issues, contact Remiai3 via Hugging Face or check the repository for updates.

download_model.py

@@ -0,0 +1,22 @@
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from huggingface_hub import login
+import os
+
+# Set your Hugging Face API token
+hf_token = "hf_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
+
+# Log in to Hugging Face
+login(token=hf_token)
+
+# Define the model name and local directory to save the model
+model_name = "sshleifer/tiny-gpt2"
+save_directory = "./tiny-gpt2-model"
+
+# Create the directory if it doesn't exist
+os.makedirs(save_directory, exist_ok=True)
+
+# Download the model and tokenizer
+model = AutoModelForCausalLM.from_pretrained(model_name, cache_dir=save_directory)
+tokenizer = AutoTokenizer.from_pretrained(model_name, cache_dir=save_directory)
+
+print(f"Model and tokenizer downloaded successfully to {save_directory}")

fine_tune/app.py

@@ -0,0 +1,52 @@
+from flask import Flask, request, render_template
+import os
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+app = Flask(__name__)
+
+# Ensure CPU execution
+os.environ["CUDA_VISIBLE_DEVICES"] = ""
+device = torch.device("cpu")
+
+# Load fine-tuned model and tokenizer
+model_path = "./fine_tuned_model"
+tokenizer_path = "./fine_tuned_model"
+
+try:
+    tokenizer = AutoTokenizer.from_pretrained(tokenizer_path, local_files_only=True)
+    model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.float32, local_files_only=True)
+    model.to(device)
+    model.eval()
+except Exception as e:
+    print(f"Error loading model or tokenizer: {e}")
+    exit(1)
+
+# Set pad_token_id
+if tokenizer.pad_token_id is None:
+    tokenizer.pad_token_id = tokenizer.eos_token_id
+
+@app.route("/", methods=["GET", "POST"])
+def index():
+    generated_text = ""
+    if request.method == "POST":
+        prompt = request.form.get("prompt", "")
+        if prompt:
+            inputs = tokenizer(prompt, return_tensors="pt", padding=True, truncation=True, max_length=128).to(device)
+            outputs = model.generate(
+                input_ids=inputs["input_ids"],
+                attention_mask=inputs["attention_mask"],
+                max_length=50,
+                num_return_sequences=1,
+                no_repeat_ngram_size=2,
+                do_sample=True,
+                top_k=50,
+                top_p=0.95,
+                temperature=0.7,
+                pad_token_id=tokenizer.eos_token_id
+            )
+            generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
+    return render_template("index.html", generated_text=generated_text)
+
+if __name__ == "__main__":
+    app.run(debug=True)

fine_tune/fine_tune_model.py

@@ -0,0 +1,101 @@
+import os
+import matplotlib.pyplot as plt
+from transformers import AutoModelForCausalLM, AutoTokenizer, Trainer, TrainingArguments
+import torch
+from datasets import load_dataset
+
+# Ensure CPU execution (force CPU even if GPU is available)
+os.environ["CUDA_VISIBLE_DEVICES"] = ""  # Disable GPU
+device = torch.device("cpu")
+
+# Define paths
+model_path = "../tiny-gpt2-model/models--sshleifer--tiny-gpt2/snapshots/5f91d94bd9cd7190a9f3216ff93cd1dd95f2c7be"
+tokenizer_path = "../tiny-gpt2-model/models--sshleifer--tiny-gpt2/snapshots/5f91d94bd9cd7190a9f3216ff93cd1dd95f2c7be"
+dataset_path = "./sample_data.txt"
+output_dir = "./fine_tuned_model"
+
+# Verify paths
+if not os.path.exists(model_path) or not os.path.exists(tokenizer_path):
+    print(f"Error: Model or tokenizer directory not found")
+    exit(1)
+if not os.path.exists(dataset_path):
+    print(f"Error: Dataset file not found at {dataset_path}")
+    exit(1)
+
+# Load tokenizer and model
+try:
+    tokenizer = AutoTokenizer.from_pretrained(tokenizer_path, local_files_only=True)
+    model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.float32, local_files_only=True)
+    model.to(device)
+except Exception as e:
+    print(f"Error loading model or tokenizer: {e}")
+    exit(1)
+
+# Set pad_token_id
+if tokenizer.pad_token_id is None:
+    tokenizer.pad_token_id = tokenizer.eos_token_id
+
+# Load and preprocess dataset
+def preprocess_data(examples):
+    encodings = tokenizer(examples["text"], truncation=True, padding="max_length", max_length=128)
+    encodings["labels"] = encodings["input_ids"].copy()  # Set labels for language modeling
+    return encodings
+
+dataset = load_dataset("text", data_files=dataset_path)
+tokenized_dataset = dataset.map(preprocess_data, batched=True, remove_columns=["text"])
+
+# Custom callback to collect loss
+class LossCallback(Trainer):
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.losses = []
+
+    def log(self, logs):
+        super().log(logs)
+        if "loss" in logs:
+            self.losses.append(logs["loss"])
+
+# Define training arguments
+training_args = TrainingArguments(
+    output_dir=output_dir,
+    num_train_epochs=3,
+    per_device_train_batch_size=1,  # Small batch size for low memory
+    save_steps=500,
+    save_total_limit=2,
+    logging_steps=1,  # Log every step for small dataset
+    learning_rate=5e-5,
+    use_cpu=True,
+)
+
+# Initialize Trainer with custom callback
+trainer = LossCallback(
+    model=model,
+    args=training_args,
+    train_dataset=tokenized_dataset["train"],
+)
+
+# Fine-tune the model
+try:
+    trainer.train()
+    print("Fine-tuning completed successfully")
+except Exception as e:
+    print(f"Error during fine-tuning: {e}")
+    exit(1)
+
+# Save the fine-tuned model and tokenizer
+model.save_pretrained(output_dir)
+tokenizer.save_pretrained(output_dir)
+print(f"Fine-tuned model and tokenizer saved to {output_dir}")
+
+# Plot and save training loss
+if trainer.losses:
+    plt.plot(trainer.losses, label="Training Loss")
+    plt.xlabel("Training Steps")
+    plt.ylabel("Loss")
+    plt.title("Training Loss Over Time")
+    plt.legend()
+    plt.savefig("loss_plot.png")
+    plt.close()
+    print("Loss plot saved as loss_plot.png")
+else:
+    print("No loss data available to plot")

fine_tune/sample_data.txt

@@ -0,0 +1,4 @@
+Once upon a time, there was a brave knight who explored a magical forest.
+The forest was filled with mystical creatures and ancient ruins.
+The knight discovered a hidden treasure guarded by a wise dragon.
+With courage and wisdom, the knight befriended the dragon and shared the treasure with the village.

fine_tune/templates/index.html

@@ -0,0 +1,68 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>Tiny-GPT2 Text Generation</title>
+    <style>
+        body {
+            font-family: Arial, sans-serif;
+            max-width: 800px;
+            margin: 0 auto;
+            padding: 20px;
+            background-color: #f4f4f9;
+        }
+        h1 {
+            text-align: center;
+            color: #333;
+        }
+        .container {
+            background-color: #fff;
+            padding: 20px;
+            border-radius: 8px;
+            box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);
+        }
+        textarea {
+            width: 100%;
+            height: 100px;
+            margin-bottom: 10px;
+            padding: 10px;
+            border: 1px solid #ccc;
+            border-radius: 4px;
+        }
+        button {
+            padding: 10px 20px;
+            background-color: #007bff;
+            color: #fff;
+            border: none;
+            border-radius: 4px;
+            cursor: pointer;
+        }
+        button:hover {
+            background-color: #0056b3;
+        }
+        .output {
+            margin-top: 20px;
+            padding: 10px;
+            border: 1px solid #ccc;
+            border-radius: 4px;
+            background-color: #f9f9f9;
+        }
+    </style>
+</head>
+<body>
+    <div class="container">
+        <h1>Tiny-GPT2 Text Generation</h1>
+        <form method="POST">
+            <textarea name="prompt" placeholder="Enter your prompt (e.g., Once upon a time)" required></textarea>
+            <button type="submit">Generate Text</button>
+        </form>
+        {% if generated_text %}
+        <div class="output">
+            <h3>Generated Text:</h3>
+            <p>{{ generated_text }}</p>
+        </div>
+        {% endif %}
+    </div>
+</body>
+</html>

requirements.txt

@@ -0,0 +1,17 @@
+# Required libraries for running tiny-gpt2 model (CPU and GPU compatible)
+torch==2.3.0  # CPU version; for GPU, use torch==2.3.0+cu121 (see notes below)
+transformers==4.38.2
+huggingface_hub==0.22.2
+datasets==2.21.0
+numpy==1.26.4
+matplotlib==3.8.3
+flask==3.0.3
+
+# Notes:
+# - For CPU-only systems (e.g., 16GB or 8GB RAM, no GPU), the above versions work directly.
+# - For GPU-supported systems (e.g., NVIDIA GTX), install GPU-compatible PyTorch:
+#   1. Uninstall torch: pip uninstall torch -y
+#   2. Install GPU version: pip install torch==2.3.0+cu121
+#   3. Verify CUDA: python -c "import torch; print(torch.cuda.is_available())"
+# - To force CPU execution on GPU systems, scripts include os.environ["CUDA_VISIBLE_DEVICES"] = ""
+# - Compatible with Python 3.10.9 or 3.9.10

test_model.py

@@ -0,0 +1,55 @@
+import os
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+# Define the model and tokenizer paths
+model_path = "./tiny-gpt2-model/models--sshleifer--tiny-gpt2/snapshots/5f91d94bd9cd7190a9f3216ff93cd1dd95f2c7be"
+tokenizer_path = "./tiny-gpt2-model/models--sshleifer--tiny-gpt2/snapshots/5f91d94bd9cd7190a9f3216ff93cd1dd95f2c7be"
+
+# Verify the directory contents
+if not os.path.exists(model_path) or not os.path.exists(tokenizer_path):
+    print(f"Error: Directory not found at {model_path}")
+    exit(1)
+
+required_files = ["config.json", "pytorch_model.bin", "vocab.json", "merges.txt"]
+for file in required_files:
+    if not os.path.exists(os.path.join(model_path, file)):
+        print(f"Error: {file} not found in {model_path}")
+        exit(1)
+
+# Load the tokenizer and model
+try:
+    tokenizer = AutoTokenizer.from_pretrained(tokenizer_path, local_files_only=True)
+    model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.float32, local_files_only=True)
+except Exception as e:
+    print(f"Error loading model or tokenizer: {e}")
+    exit(1)
+
+# Set pad_token_id to eos_token_id if not already set
+if tokenizer.pad_token_id is None:
+    tokenizer.pad_token_id = tokenizer.eos_token_id
+
+# Set model to evaluation mode
+model.eval()
+
+# Prepare input text
+prompt = "Once upon a time"
+inputs = tokenizer(prompt, return_tensors="pt", padding=True, truncation=True).to("cpu")
+
+# Generate text
+outputs = model.generate(
+    input_ids=inputs["input_ids"],
+    attention_mask=inputs["attention_mask"],
+    max_length=50,
+    num_return_sequences=1,
+    no_repeat_ngram_size=2,
+    do_sample=True,
+    top_k=50,
+    top_p=0.95,
+    temperature=0.7,
+    pad_token_id=tokenizer.eos_token_id
+)
+
+# Decode and print the generated text
+generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
+print("Generated Text:", generated_text)