ThingsAI commited on
Commit
e71071f
·
verified ·
1 Parent(s): 62f92fc

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +26 -26
README.md CHANGED
@@ -75,31 +75,6 @@ domain‑specific tasks – for instance, a customer‑support bot, a code revie
75
  - Short context window (2,048 tokens).
76
  - Small size means it can make more factual mistakes than larger models.
77
 
78
- ## How to Get Started
79
-
80
- ```python
81
- from transformers import AutoTokenizer, AutoModelForCausalLM
82
-
83
- model_name = "OvercastLab/Quark-50m-Instruct"
84
-
85
- tokenizer = AutoTokenizer.from_pretrained(model_name)
86
- model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")
87
-
88
- messages = [
89
- {"role": "system", "content": "You are Quark, a helpful assistant."},
90
- {"role": "user", "content": "Explain group query attention in one sentence."}
91
- ]
92
-
93
- inputs = tokenizer.apply_chat_template(
94
- messages,
95
- tokenize=True,
96
- add_generation_prompt=True,
97
- return_tensors="pt"
98
- ).to(model.device)
99
-
100
- outputs = model.generate(inputs, max_new_tokens=128)
101
- print(tokenizer.decode(outputs[0], skip_special_tokens=True))
102
-
103
  ## Training Details
104
 
105
  ### Pretraining
@@ -136,4 +111,29 @@ All data was tokenised with the official [Cosmo2 tokenizer](https://huggingface.
136
  ### Instruction Fine‑tuning
137
 
138
  The base model was fine‑tuned on a curated set of instruction‑following data (details to be released).
139
- The fine‑tuning used **LoRA** with the same sequence length and a lower learning rate (1e‑4) for a few thousand steps.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
75
  - Short context window (2,048 tokens).
76
  - Small size means it can make more factual mistakes than larger models.
77
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
78
  ## Training Details
79
 
80
  ### Pretraining
 
111
  ### Instruction Fine‑tuning
112
 
113
  The base model was fine‑tuned on a curated set of instruction‑following data (details to be released).
114
+ The fine‑tuning used **LoRA** with the same sequence length and a lower learning rate (1e‑4) for a few thousand steps.
115
+
116
+ ## How to Use
117
+
118
+ ```python
119
+ from transformers import AutoTokenizer, AutoModelForCausalLM
120
+
121
+ model_name = "OvercastLab/Quark-50m-Instruct"
122
+
123
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
124
+ model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")
125
+
126
+ messages = [
127
+ {"role": "system", "content": "You are Quark, a helpful assistant."},
128
+ {"role": "user", "content": "Explain group query attention in one sentence."}
129
+ ]
130
+
131
+ inputs = tokenizer.apply_chat_template(
132
+ messages,
133
+ tokenize=True,
134
+ add_generation_prompt=True,
135
+ return_tensors="pt"
136
+ ).to(model.device)
137
+
138
+ outputs = model.generate(inputs, max_new_tokens=128)
139
+ print(tokenizer.decode(outputs[0], skip_special_tokens=True))