Lyon28 commited on
Commit
d3a51b5
·
verified ·
1 Parent(s): e16e408

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -35
README.md CHANGED
@@ -1,54 +1,21 @@
1
  ---
2
  language:
3
  - id
4
- license: mit
5
  tags:
6
  - text-generation
7
  - indonesian
8
  - pytorch
9
- - caca
10
- datasets:
11
- - Lyon28/Corpus-Indonesia
12
- - Lyon28/Caca-Behavior
13
  ---
14
 
15
  # Caca-Tiny 🔥
16
 
17
- Caca-Tiny adalah language model berbahasa Indonesia yang dilatih menggunakan arsitektur transformer decoder.
18
-
19
  ## Model Details
20
 
21
  - **Architecture**: Transformer Decoder
22
  - **Parameters**: ~4,156,928
23
  - **Vocabulary Size**: 8000
24
  - **Max Sequence Length**: 512
25
- - **Training Data**: Lyon28/Corpus-Indonesia
26
- - **Fine-tuning Data**: Lyon28/Caca-Behavior
27
-
28
- ## Usage
29
-
30
- ```python
31
- import torch
32
- from safetensors.torch import load_file
33
-
34
- state_dict = load_file("model.safetensors")
35
-
36
- prompt = "Indonesia adalah"
37
- generated = model.generate(prompt, max_new_tokens=50)
38
- print(generated)
39
- ```
40
-
41
- ## Training
42
-
43
- Model ini dilatih dengan:
44
- - Optimizer: AdamW
45
- - Learning Rate: 3e-4
46
- - Batch Size: 8
47
- - Epochs: 3 (pre-training) + 2 (fine-tuning)
48
-
49
- ## License
50
-
51
- MIT License
52
 
53
  ## Citation
54
 
@@ -60,4 +27,4 @@ MIT License
60
  publisher = {Hugging Face},
61
  url = {https://huggingface.co/Lyon28/Caca-Tiny}
62
  }
63
- ```
 
1
  ---
2
  language:
3
  - id
4
+ license: apache-2.0
5
  tags:
6
  - text-generation
7
  - indonesian
8
  - pytorch
 
 
 
 
9
  ---
10
 
11
  # Caca-Tiny 🔥
12
 
 
 
13
  ## Model Details
14
 
15
  - **Architecture**: Transformer Decoder
16
  - **Parameters**: ~4,156,928
17
  - **Vocabulary Size**: 8000
18
  - **Max Sequence Length**: 512
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
 
20
  ## Citation
21
 
 
27
  publisher = {Hugging Face},
28
  url = {https://huggingface.co/Lyon28/Caca-Tiny}
29
  }
30
+ ```