pnevskaiaan commited on
Commit
075a143
·
verified ·
1 Parent(s): c4cfc23

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +36 -3
README.md CHANGED
@@ -1,3 +1,36 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ ---
4
+
5
+ Here is a code to create this tiny model:
6
+
7
+
8
+ ```python
9
+ import os
10
+
11
+ from transformers import AutoTokenizer
12
+ from transformers import Lfm2Config, Lfm2ForCausalLM
13
+
14
+ # === Step 1: Define tiny model config ===
15
+ config = Lfm2Config(
16
+ num_hidden_layers=4,
17
+ num_attention_heads=4,
18
+ num_key_value_heads=4,
19
+ intermediate_size=12,
20
+ hidden_size=16,
21
+ block_multiple_of=8
22
+ )
23
+
24
+ # === Step 2: Create model from config ===
25
+ model = Lfm2ForCausalLM(config)
26
+
27
+ # === Step 3: Load or create tokenizer ===
28
+ model_id = "LiquidAI/LFM2-350M"
29
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
30
+
31
+ # === Step 4: Save model and tokenizer ===
32
+ output_dir = "./lfm2"
33
+ os.makedirs(output_dir, exist_ok=True)
34
+ model.save_pretrained(output_dir, safe_serialization=False)
35
+ tokenizer.save_pretrained(output_dir)
36
+ ```