Gentraxyz commited on
Commit
0a4723c
·
verified ·
1 Parent(s): 3c38b94

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +70 -0
README.md CHANGED
@@ -1,3 +1,73 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ language:
4
+ - en
5
+ library_name: transformers
6
+ pipeline_tag: text-generation
7
+ tags:
8
+ - gpt2
9
+ - from-scratch
10
+ - tinystories
11
+ - text-generation
12
+ - causal-lm
13
  ---
14
+
15
+ # RecursiveComplete
16
+
17
+ A small GPT-2-style language model (~18.3M parameters) trained **completely from scratch by an AI**, end to end — the architecture, training code, tokenizer, data prep, and training run were all written and executed by an AI agent with no pre-existing weights or fine-tuning from another model.
18
+
19
+ This is a **text-completion model**, not an instruction-tuned chatbot. It's good at continuing short prose and simple stories. It is **not** good at answering questions, following instructions, or factual recall.
20
+
21
+ ## Model details
22
+
23
+ | | |
24
+ |---|---|
25
+ | Type | Decoder-only transformer (GPT-2 style) |
26
+ | Parameters | ~18.3M |
27
+ | Embedding dim (`n_embd`) | 448 |
28
+ | Heads (`n_head`) | 7 |
29
+ | Layers (`n_layer`) | 6 |
30
+ | Context length (`block_size`) | 256 |
31
+ | Vocab size | 8192 |
32
+ | Tokenizer | Byte-level BPE (`<eot>` id = 0) |
33
+ | Dropout | 0.1 |
34
+ | Final train loss | ~1.86 |
35
+
36
+ ## Training data
37
+
38
+ Trained primarily on **TinyStories** (~90M tokens) with a small amount of Alpaca-style data. The model learned general English sentence structure and simple narrative flow, not world knowledge.
39
+
40
+ ## Intended use
41
+
42
+ - Story / prose continuation
43
+ - Experimentation and education (a clean, fully-from-scratch small LM)
44
+
45
+ ## Limitations
46
+
47
+ - Completion only — will not reliably answer questions or follow instructions.
48
+ - No factual reliability; it will confidently make things up.
49
+ - Small context (256 tokens) and small vocab.
50
+ - English only.
51
+
52
+ ## How to use
53
+
54
+ ```python
55
+ from transformers import AutoTokenizer, AutoModelForCausalLM
56
+ import torch
57
+
58
+ tok = AutoTokenizer.from_pretrained("Gentraxyz/RecursiveComplete")
59
+ model = AutoModelForCausalLM.from_pretrained("Gentraxyz/RecursiveComplete")
60
+
61
+ prompt = "Once upon a time"
62
+ ids = tok(prompt, return_tensors="pt").input_ids
63
+ out = model.generate(ids, max_new_tokens=100, temperature=0.8, top_k=40, do_sample=True)
64
+ print(tok.decode(out[0]))
65
+ ```
66
+
67
+ ## License
68
+
69
+ Apache 2.0.
70
+
71
+ ## Note
72
+
73
+ This model was trained entirely by an AI — including writing the model code, the tokenizer, the data pipeline, and running the training. It is shared as a small from-scratch experiment.