dixisouls commited on
Commit
1565a27
·
1 Parent(s): 6d542c5

Added model metadata

Browse files
Files changed (1) hide show
  1. README.md +31 -30
README.md CHANGED
@@ -1,40 +1,41 @@
1
- ---
2
- license: mit
3
- tags:
4
- - text-generation
5
- - pytorch
6
- - transformer
7
- - custom-model
8
- language:
9
- - en
10
- pipeline_tag: text-generation
11
- ---
12
 
13
- # VelocityLM - 2B Parameter Language Model
14
 
15
- A custom transformer model with 2B parameters trained for text generation.
16
 
17
- ## Model Details
18
 
19
- - **Parameters:** ~2 billion
20
- - **Architecture:** Custom Transformer with RoPE, RMSNorm, SwiGLU
21
- - **Context Length:** 2,048 tokens
22
- - **Tokenizer:** GPT-2 compatible
23
- - **Training:** Falcon RefinedWeb dataset
24
 
25
- ## Usage
26
 
27
- ```python
28
- from transformers import AutoTokenizer
29
- import torch
30
 
31
- # Load tokenizer
32
- tokenizer = AutoTokenizer.from_pretrained("gpt2")
33
 
34
- # Load model (you'll need custom loading code)
35
- # See the Space implementation for details
 
36
 
37
- Files
38
 
39
- - config.json - Model configuration
40
- - pytorch_model.bin - Model weights
 
1
+ ---
2
+ license: mit
3
+ tags:
4
+ - text-generation
5
+ - pytorch
6
+ - transformer
7
+ - custom-model
8
+ language:
9
+ - en
10
+ pipeline_tag: text-generation
11
+ ---
12
 
13
+ # VelocityLM - 2B Parameter Language Model
14
 
15
+ A custom transformer model with 2B parameters trained for text generation.
16
 
17
+ ## Model Details
18
 
19
+ - **Parameters:** ~2 billion
20
+ - **Architecture:** Custom Transformer with RoPE, RMSNorm, SwiGLU
21
+ - **Context Length:** 2,048 tokens
22
+ - **Tokenizer:** GPT-2 compatible
23
+ - **Training:** Falcon RefinedWeb dataset
24
 
25
+ ## Usage
26
 
27
+ ```python
28
+ from transformers import AutoTokenizer
29
+ import torch
30
 
31
+ # Load tokenizer
32
+ tokenizer = AutoTokenizer.from_pretrained("gpt2")
33
 
34
+ # Load model (you'll need custom loading code)
35
+ # See the Space implementation for details
36
+ ```
37
 
38
+ ## Files
39
 
40
+ - config.json - Model configuration
41
+ - pytorch_model.bin - Model weights