Update README.md

README.md

@@ -1,9 +1,67 @@
+# TinyTimV1: Fine-tuning TinyLlama on Finnegan's Wake
 
-TinyTim
----
-license: llama2
----
+A project exploring the fine-tuning of TinyLlama-1.1B on James Joyce's *Finnegan's Wake* to generate Joyce-inspired text.
+
+## Overview
+
+This project fine-tunes the TinyLlama-1.1B-Chat model on the complete text of James Joyce's *Finnegan's Wake*, creating a language model capable of generating text in Joyce's distinctive experimental style. The model learns to replicate the complex wordplay, neologisms, and stream-of-consciousness narrative techniques characteristic of Joyce's final work.
+
+## Files
+
+- `process_wake.py` - Preprocesses the raw text, removes page numbers, and splits into manageable chunks
+- `fine_tune_joyce.py` - Main training script using HuggingFace Transformers
+- `text_gen.py` - Text generation script for the fine-tuned model
+- `finn_wake.txt` - Complete text of Finnegan's Wake (1.51 MB)
+- `finn_wake.csv` - Processed dataset in CSV format
+- `finn_wake_dataset/` - Tokenized dataset directory
+
+## Usage
+
+### 1. Data Preprocessing
+```bash
+python process_wake.py
+```
+This removes page numbers and splits the text into 100-word chunks for training.
+2. Fine-tuning
+
+```bash
+python fine_tune_joyce.py
+```
+Fine-tunes TinyLlama on the processed dataset for 3 epochs with CPU training.
+3. Text Generation
+```bash
+python text_gen.py
+```
+Generates Joyce-inspired text using the fine-tuned model.
+
+Model Details
+
+Base Model: TinyLlama-1.1B-Chat-v1.0
+Training Data: Finnegan's Wake (~1.5MB text)
+Training Parameters:
+
+3 epochs
+Batch size: 1
+Max sequence length: 128 tokens
+Temperature: 0.7
+Top-k: 50, Top-p: 0.95
+
+
+
+Example Output
+Input: "ae left to go to ireland and found a fairy"
+The model generates text continuing in Joyce's experimental style with invented words, Irish references, and complex linguistic play.
+Requirements
+transformers
+datasets
+pandas
+torch
+Installation
+bashpip install transformers datasets pandas torch
+Notes
+
+Training was performed on CPU due to resource constraints
+Model checkpoints saved every 500 steps
+Resume training supported from checkpoints
 
 
-TinyTim is a version of TinyLlama that has been finetuned on text from James Joyce's Finnegan's Wake.
-Hope other Joyce fans find it useful or entertaining!