Nonovogo commited on
Commit
2d01520
·
verified ·
1 Parent(s): 32fd935

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +27 -0
README.md CHANGED
@@ -17,6 +17,33 @@ language:
17
  - **License:** apache-2.0
18
  - **Finetuned from model :** unsloth/gemma-3-270m-it
19
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
20
  This gemma3_text model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
21
 
22
  [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
17
  - **License:** apache-2.0
18
  - **Finetuned from model :** unsloth/gemma-3-270m-it
19
 
20
+ Use
21
+ ```
22
+ text = tokenizer.apply_chat_template(
23
+ messages,
24
+ tokenize = False,
25
+ add_generation_prompt = True
26
+ ).removeprefix('<bos>')
27
+
28
+ # This forces the model to enter "thinking mode" immediately.
29
+ text += "<think>\n"
30
+
31
+ # 3. Generate
32
+ _ = model.generate(
33
+ **tokenizer(text, return_tensors="pt").to("cuda"),
34
+ max_new_tokens=2048, # Don't let it ramble forever
35
+
36
+ # --- STABILITY SETTINGS ---
37
+ do_sample=True, # Enable sampling to break deterministic loops
38
+ temperature=0.1, # Very low temp (focused) but not zero
39
+ top_p=0.95, # Standard filtering
40
+ repetition_penalty=1.0, # CRITICAL: Disable penalty (1.0 = no penalty)
41
+
42
+ streamer=TextStreamer(tokenizer, skip_prompt=True),
43
+ eos_token_id=tokenizer.eos_token_id # Ensure it knows when to stop
44
+ )
45
+ ```
46
+ For better output
47
  This gemma3_text model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
48
 
49
  [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)