Upload 7 files
🧹 Gemma-3-270M Machine Unlearning Project
This project demonstrates machine unlearning using Google’s Gemma-3-270M.
The aim was to fine-tune the model so that it forgets specific tokens, phrases, or patterns without affecting the rest of its knowledge.
📌 What I Did
Downloaded the base Gemma-3-270M model locally from Hugging Face.
Created a scrubbing dataset with words and phrases that should be forgotten by the model.
Implemented an unlearning script that loads the model and tokenizer, applies a forgetting mechanism to the scrubbed dataset, fine-tunes the model, and saves the updated version into a new folder.
Ran the fine-tuning loop so that the model unlearned the targeted content while keeping its general language understanding intact.
📂 Project Structure
Base Model Files: Configuration, weights, and tokenizer files.
Scrub Dataset: Custom file with words or phrases to be forgotten.
Unlearning Script: Python script used to train the model with unlearning objectives.
Fine-Tuned Model Folder: Contains the updated model files after unlearning.
▶️ Workflow
Install the necessary dependencies.
Run the unlearning script.
A new fine-tuned model folder is created that contains the updated model.
Load the new model for testing and verify that it avoids the scrubbed tokens while retaining normal abilities.
✅ Output
Before unlearning: The model could generate definitions or explanations for the scrubbed words and phrases.
After unlearning: The model avoids producing the scrubbed tokens, instead giving responses such as “I don’t know,” placeholder tokens, or irrelevant outputs.
The final model continues to function normally for all other queries while no longer producing the forgotten content.