Create README.md

README.md

@@ -0,0 +1,109 @@
+---
+language: en
+license: mit
+tags:
+  - gpt
+  - mini-gpt
+  - text-generation
+  - language-modeling
+  - pytorch
+datasets:
+  - custom
+widget:
+  - text: "In the future, AI will"
+---
+
+# MiniGPT: A GPT Model Fine-Tuned on Shakespeare Texts
+
+This is a custom GPT model fine-tuned on a small subset of Shakespeare's works. The model is built using PyTorch and Transformers, and is designed to generate creative text in the style of Shakespeare. 
+
+## 🚀 Overview
+
+This project demonstrates the process of fine-tuning a small version of GPT on a custom dataset and deploying it to Hugging Face for easy use and access. The model has been trained using a tiny Shakespeare dataset (`tiny_shakespeare.txt`), and is capable of generating creative text based on a given prompt.
+
+## 🧑‍💻 Setup
+
+To use or replicate this model, follow the steps below to get started.
+
+### 1. Installation
+
+First, install the required dependencies:
+
+📚 Dataset
+This model was fine-tuned on a small dataset of Shakespeare's works. You can use your own dataset by following the structure below
+# Load and tokenize data
+from transformers import AutoTokenizer
+
+tokenizer = AutoTokenizer.from_pretrained("gpt2")
+tokenizer.pad_token = tokenizer.eos_token  # Important for padding
+
+# Load your dataset
+file_path = "path/to/your_dataset.txt"
+with open(file_path, "r", encoding="utf-8") as f:
+    raw_text = f.read()
+
+encoded = tokenizer(raw_text, return_tensors="pt", truncation=True, padding=True, max_length=512)
+input_ids = encoded["input_ids"]
+
+🧑‍💻 Training
+To fine-tune the model, follow the instructions below:
+from transformers import TrainingArguments
+import wandb
+
+wandb.login()
+
+# Set up W&B logging configuration
+wandb.init(project="mini-gpt", config={
+    "learning_rate": 0.001,
+    "epochs": 5,
+    "batch_size": 16
+})
+
+# Training loop
+train(model, optimizer, epochs=5, encoded_dataset=encoded_dataset, batch_size=4)
+
+💾 Saving the Model
+Once the model is trained, you can save it like this:
+torch.save(model.state_dict(), "my-mini-gpt-model/pytorch_model.bin")
+tokenizer.save_pretrained("my-mini-gpt-tokenizer")
+
+
+🚀 Uploading the Model to Hugging Face
+You can upload the model to Hugging Face Hub using the following commands:
+from huggingface_hub import login, create_repo, upload_folder
+
+login(token="your_huggingface_token")
+
+repo_id = "your_username/my-mini-gpt"
+create_repo(repo_id, exist_ok=True)
+
+upload_folder(folder_path="my-mini-gpt-model", repo_id=repo_id)
+upload_folder(folder_path="my-mini-gpt-tokenizer", repo_id=repo_id)
+🎮 Generating Text
+Once the model is uploaded to Hugging Face, you can easily generate text using the following code:
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+# Load model and tokenizer from Hugging Face
+model = AutoModelForCausalLM.from_pretrained("your_username/my-mini-gpt")
+tokenizer = AutoTokenizer.from_pretrained("your_username/my-mini-gpt")
+
+# Generate text based on a prompt
+prompt = "In the future, AI will"
+inputs = tokenizer(prompt, return_tensors="pt")
+outputs = model.generate(
+    **inputs,
+    max_length=50,
+    temperature=1.0,
+    top_k=50,
+    top_p=0.95,
+    do_sample=True
+)
+
+generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
+print(generated_text)
+📄 License
+This model is licensed under the MIT License.
+
+📝 Citation
+If you use this model or code in your research, please cite it as:
+Tkachenko, A. (2025). MiniGPT: Fine-tuned GPT model for text generation. Hugging Face. Available at: https://huggingface.co/altkachenko11/my-mini-gpt