updated with stories train data

README.md

@@ -1,57 +1,55 @@
 ---
 library_name: transformers
-license: apache-2.0
+license: mit
+base_model: gpt2
+tags:
+- generated_from_trainer
+model-index:
+- name: JAT-GPT2-trainer
+  results: []
 ---
-# 🧠 JAT-GPT: Just Another Tiny GPT
 
-Welcome to **JAT-GPT**, the world's most underwhelming large language model — clocking in at a mighty **74 million parameters** (yes, million, not billion — stop laughing). 
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
 
-## 📦 Model Details
+# JAT-GPT2-trainer
 
-- **Model type**: GPT2-based decoder-only transformer
-- **Architecture**: GPT-2
-- **Library**: Hugging Face 🤗 Transformers
-- **Parameters**: 74 million (size isn't everything... right?)
-- **Training Objective**: Learn to predict the next word — and sometimes even the *right* one!
-- **Pretrained on**: A secret* dataset (*"secret" means the dataset was just some text I could find lying around)
-- **Training Purpose**: Solely educational. Also for flexing on friends who haven’t trained a language model from scratch.
+This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on an unknown dataset.
 
-## 🚀 Capabilities
+## Model description
 
-- Can generate small sentences
-  - "Please lower your expectations."
-- Can hallucinate confidently, but in a very short and polite way.
-- Can generate random words after few tokens.
+More information needed
 
-## 🙅 Limitations
+## Intended uses & limitations
 
-- Not very smart.
-- Only Pretrained.
-- Understands context... if it fits within few tokens.
-- Cannot replace ChatGPT. (But look how cute it is!)
+More information needed
 
-## 🤷 Why Train This?
+## Training and evaluation data
 
-> "Because I could." – :-)
+More information needed
 
-- To understand the internals of language modeling.
-- To cry less when training real models later.
-- To appreciate just how powerful modern LLMs are by comparison.
+## Training procedure
 
-## 🛠️ Usage
+### Training hyperparameters
 
-```python
-# Load model directly
-from transformers import AutoTokenizer, AutoModelForCausalLM
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 32
+- eval_batch_size: 32
+- seed: 42
+- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 500
+- num_epochs: 10
+- mixed_precision_training: Native AMP
 
-tokenizer = AutoTokenizer.from_pretrained("itsme-nishanth/JAT-GPT")
-model = AutoModelForCausalLM.from_pretrained("itsme-nishanth/JAT-GPT")
+### Training results
 
-input_ids = tokenizer.encode("Hi there,", return_tensors="pt")
-output = model.generate(input_ids, max_length=20, do_sample=True)
-print(tokenizer.decode(output[0]))
 
-# Use a pipeline as a high-level helper
-from transformers import pipeline
 
-pipe = pipeline("text-generation", model="itsme-nishanth/JAT-GPT")
+### Framework versions
+
+- Transformers 4.53.2
+- Pytorch 2.6.0+cu124
+- Datasets 4.0.0
+- Tokenizers 0.21.2

config.json

@@ -1,5 +1,4 @@
 {
-  "_name_or_path": "itsme-nishanth/JAT-GPT",
   "activation_function": "gelu_new",
   "architectures": [
     "GPT2LMHeadModel"
@@ -33,7 +32,7 @@
     }
   },
   "torch_dtype": "float32",
-  "transformers_version": "4.47.1",
+  "transformers_version": "4.53.2",
   "use_cache": true,
   "vocab_size": 50257
 }

generation_config.json

@@ -2,5 +2,5 @@
   "_from_model_config": true,
   "bos_token_id": 50256,
   "eos_token_id": 50256,
-  "transformers_version": "4.47.1"
+  "transformers_version": "4.53.2"
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6a299ba7ef71d2bf15a202ee86470aa5d88dc05117c5571fe9c56a207de1acb9
+oid sha256:19fc462d738ec4f0753b036c27325368b17fd10d32e339cb41fdd1b7f6cec357
 size 71475528

optimizer.pth → training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:337e38b1fca79700230e72b05b4e7e5df625a9ba37c8a3252400c9ecf1309844
-size 142980858
+oid sha256:4b260083aec1104bea74d678c6f37dc3f32586a5c95b740c77adc7d2de070456
+size 5304