Gentraxyz commited on
Commit
c9db13d
·
verified ·
1 Parent(s): 0a4723c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +36 -17
README.md CHANGED
@@ -2,7 +2,6 @@
2
  license: apache-2.0
3
  language:
4
  - en
5
- library_name: transformers
6
  pipeline_tag: text-generation
7
  tags:
8
  - gpt2
@@ -18,6 +17,8 @@ A small GPT-2-style language model (~18.3M parameters) trained **completely from
18
 
19
  This is a **text-completion model**, not an instruction-tuned chatbot. It's good at continuing short prose and simple stories. It is **not** good at answering questions, following instructions, or factual recall.
20
 
 
 
21
  ## Model details
22
 
23
  | | |
@@ -37,33 +38,51 @@ This is a **text-completion model**, not an instruction-tuned chatbot. It's good
37
 
38
  Trained primarily on **TinyStories** (~90M tokens) with a small amount of Alpaca-style data. The model learned general English sentence structure and simple narrative flow, not world knowledge.
39
 
 
 
 
 
 
 
 
 
 
 
 
 
40
  ## Intended use
41
 
42
  - Story / prose continuation
43
  - Experimentation and education (a clean, fully-from-scratch small LM)
44
 
45
- ## Limitations
46
-
47
- - Completion only — will not reliably answer questions or follow instructions.
48
- - No factual reliability; it will confidently make things up.
49
- - Small context (256 tokens) and small vocab.
50
- - English only.
51
-
52
  ## How to use
53
 
54
- ```python
55
- from transformers import AutoTokenizer, AutoModelForCausalLM
56
- import torch
57
 
58
- tok = AutoTokenizer.from_pretrained("Gentraxyz/RecursiveComplete")
59
- model = AutoModelForCausalLM.from_pretrained("Gentraxyz/RecursiveComplete")
 
60
 
61
- prompt = "Once upon a time"
62
- ids = tok(prompt, return_tensors="pt").input_ids
63
- out = model.generate(ids, max_new_tokens=100, temperature=0.8, top_k=40, do_sample=True)
64
- print(tok.decode(out[0]))
 
 
 
65
  ```
66
 
 
 
 
 
 
 
 
 
 
 
 
67
  ## License
68
 
69
  Apache 2.0.
 
2
  license: apache-2.0
3
  language:
4
  - en
 
5
  pipeline_tag: text-generation
6
  tags:
7
  - gpt2
 
17
 
18
  This is a **text-completion model**, not an instruction-tuned chatbot. It's good at continuing short prose and simple stories. It is **not** good at answering questions, following instructions, or factual recall.
19
 
20
+ > **Note:** this is a custom-format model, not a `transformers` model. You load it with the included scripts (`gpt2.py` + `chat.py`), not `AutoModelForCausalLM`.
21
+
22
  ## Model details
23
 
24
  | | |
 
38
 
39
  Trained primarily on **TinyStories** (~90M tokens) with a small amount of Alpaca-style data. The model learned general English sentence structure and simple narrative flow, not world knowledge.
40
 
41
+ ## Files in this repo
42
+
43
+ | File | What it is |
44
+ |---|---|
45
+ | `model.safetensors` | The model weights |
46
+ | `config.json` | Architecture config (custom format) |
47
+ | `gpt2.py` | Model definition (the GPT-2-style architecture) |
48
+ | `chat.py` | Run / generate from the model |
49
+ | `tokenizer_bpe/vocab.json`, `tokenizer_bpe/merges.txt` | Byte-level BPE tokenizer |
50
+ | `big.pt` | Full training checkpoint (model + optimizer), for **resuming training** only |
51
+ | `train_big.py`, `prep_bpe.py` | Training and data-prep scripts |
52
+
53
  ## Intended use
54
 
55
  - Story / prose continuation
56
  - Experimentation and education (a clean, fully-from-scratch small LM)
57
 
 
 
 
 
 
 
 
58
  ## How to use
59
 
60
+ This model uses its own minimal code, not the `transformers` library.
 
 
61
 
62
+ ```bash
63
+ # 1. Install deps
64
+ pip install torch tokenizers safetensors numpy
65
 
66
+ # 2. Download this repo (gives you the scripts + weights + tokenizer)
67
+ pip install huggingface_hub
68
+ hf download Gentraxyz/RecursiveComplete --local-dir RecursiveComplete
69
+ cd RecursiveComplete
70
+
71
+ # 3. Generate
72
+ python chat.py
73
  ```
74
 
75
+ `chat.py` loads `gpt2.py` (the architecture), the weights from `model.safetensors`, and the BPE tokenizer in `tokenizer_bpe/`, then lets you prompt the model for completions.
76
+
77
+ Tip: it's a completion model — give it the *start* of something ("Once upon a time there was a small robot who") rather than a question.
78
+
79
+ ## Limitations
80
+
81
+ - Completion only — will not reliably answer questions or follow instructions.
82
+ - No factual reliability; it will confidently make things up.
83
+ - Small context (256 tokens) and small vocab (8192).
84
+ - English only.
85
+
86
  ## License
87
 
88
  Apache 2.0.