QuantFlip commited on
Commit
ceb6ab3
·
verified ·
1 Parent(s): da7b22c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -120
README.md CHANGED
@@ -6,123 +6,5 @@ tags:
6
  base_model: tiiuae/Falcon3-3B-Base
7
  ---
8
 
9
- ![image/png](https://cdn-uploads.huggingface.co/production/uploads/62441d1d9fdefb55a0b7d12c/c-tosr0FvMlKuKQTojx_6.png)
10
-
11
-
12
- # Table of Contents
13
-
14
- 0. [TL;DR](#TL;DR)
15
- 1. [Model Details](#model-details)
16
- 2. [Training Details](#training-details)
17
- 3. [Usage](#usage)
18
- 4. [Evaluation](#evaluation)
19
- 5. [Citation](#citation)
20
-
21
-
22
- # TL;DR
23
-
24
- # Model Details
25
-
26
- ## Model Description
27
-
28
- - **Developed by:** [https://www.tii.ae](https://www.tii.ae)
29
- - **Model type:** Causal decoder-only
30
- - **Architecture:** Pure-transformer - 1.58bit version
31
- - **Language(s) (NLP):** Mainly English
32
- - **License:** TII Falcon License 2.0
33
-
34
- # Training details
35
-
36
- The model has been trained following the training strategies from the recent [1-bit LLM HF blogpost](https://huggingface.co/blog/1_58_llm_extreme_quantization) and [1-bit LLM paper](https://huggingface.co/papers/2402.17764).
37
- For more details about the training protocol of this model, please refer to the Falcon-3 technical report, section *Compression*.
38
-
39
-
40
- # Usage
41
-
42
- Currently to use this model you can either rely on Hugging Face transformers library or [BitNet](https://github.com/microsoft/BitNet) library. You can also play with the model using the [falcon-1.58bit playground](https://huggingface.co/spaces/tiiuae/falcon3-1.58bit-playground) (only for the 7B instruct version).
43
-
44
- ## 🤗 transformers
45
-
46
- ```python
47
- import torch
48
- from transformers import AutoModelForCausalLM, AutoTokenizer
49
-
50
- model_id = "tiiuae/Falcon3-3B-Base-1.58bit"
51
-
52
- model = AutoModelForCausalLM.from_pretrained(
53
- model_id,
54
- torch_dtype=torch.bfloat16,
55
- ).to("cuda")
56
-
57
- # Perform text generation
58
- ```
59
-
60
- ## BitNet
61
-
62
- ```
63
- git clone https://github.com/microsoft/BitNet && cd BitNet
64
- pip install -r requirements.txt
65
- python setup_env.py --hf-repo tiiuae/Falcon3-3B-Base-1.58bit -q i2_s
66
- python run_inference.py -m models/Falcon3-3B-Base-1.58bit/ggml-model-i2_s.gguf -p "Hi how are you doing today?" -cnv
67
- ```
68
-
69
- # Evaluation
70
- We report in the following table our internal pipeline benchmarks:
71
-
72
- **Note evaluation results are normalized score from v2 leaderboard tasks - reported results of original models in the blogpost are raw scores**
73
-
74
- <table border="1" style="width: 100%; text-align: center; border-collapse: collapse;">
75
- <colgroup>
76
- <col style="width: 10%;">
77
- <col style="width: 10%;">
78
- <col style="background-color: rgba(80, 15, 213, 0.5); width: 7%;">
79
- </colgroup>
80
- <thead>
81
- <tr>
82
- <th>Benchmark</th>
83
- <th>Llama3-8B-1.58-100B-tokens</th>
84
- <th>Falcon3-3B-Instruct-1.58bit </th>
85
- </tr>
86
- </thead>
87
- <tbody>
88
- <tr>
89
- <td>IFEval</td>
90
- <td>17.91</td>
91
- <td><b>27.49</b></td>
92
- </tr>
93
- <tr>
94
- <td>MUSR</td>
95
- <td><b>4.87</b></td>
96
- <td>4.64</td>
97
- </tr>
98
- <tr>
99
- <td>GPQA</td>
100
- <td><b>1.83<b></td>
101
- <td>0.00</td>
102
- </tr>
103
- <tr>
104
- <td>BBH</td>
105
- <td>5.36</td>
106
- <td><b>2.97</b></td>
107
- </tr>
108
- <tr>
109
- <td>MMLU-PRO</td>
110
- <td><b>2.78<b></td>
111
- <td><b>1.47</b></td>
112
- </tr>
113
- <tr>
114
- <td>MATH</td>
115
- <td>0.26</td>
116
- <td><b>0.43</b></td>
117
- </tr>
118
- <tr>
119
- <td>Average</td>
120
- <td>5.5</td>
121
- <td><b>6.17</b></td>
122
- </tr>
123
- </tbody>
124
- </table>
125
-
126
- # Citation
127
-
128
- Coming soon ..
 
6
  base_model: tiiuae/Falcon3-3B-Base
7
  ---
8
 
9
+ The Falcon3-3B-Base-1.58bit model uploaded in this warehouse is a test model downloaded from the warehouse after being tapped by Quantflip.
10
+ It is only used for testing Quantflip!!!