strykes commited on
Commit
13c1865
·
verified ·
1 Parent(s): 47cd4e3

upload README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -38
README.md CHANGED
@@ -1,18 +1,14 @@
1
  ---
2
  language:
3
  - en
4
- - zh
5
  license: apache-2.0
6
  tags:
7
  - transformers
8
  - safetensors
9
- - llama.cpp
10
  - gguf
11
  - peft
12
  - qlora
13
  - reasoning
14
- - math
15
- - code
16
  base_model:
17
  - Nanbeige/Nanbeige4.1-3B
18
  library_name: transformers
@@ -21,47 +17,27 @@ pipeline_tag: text-generation
21
 
22
  # EmberForge-3B-Reasoner
23
 
24
- EmberForge-3B-Reasoner is a private finetuned Nanbeige 4.1 3B reasoning model release by `strykes`.
25
 
26
- ## What is included
27
 
28
- This repo intentionally includes multiple artifact types:
29
-
30
- - **Merged full model (Safetensors)** at repo root (for Transformers / benchmark pipelines)
31
- - **LoRA adapter** in `adapter/`
32
- - **GGUF quants** in `gguf/`:
33
  - `Nanbeige4.1-3B-Q5_K_M.gguf`
34
  - `Nanbeige4.1-3B-Q4_K_M.gguf`
35
  - `Nanbeige4.1-3B-f16.gguf`
 
36
 
37
- ## Training summary
38
-
39
- - Base model: `Nanbeige/Nanbeige4.1-3B`
40
- - Method: QLoRA with Unsloth, merged to full weights
41
- - Dataset: synthetic reasoning instruction dataset (`3500` samples)
42
- - Epochs: `2`
43
- - Effective batch size: `16` (batch 1 x grad acc 16)
44
- - Max sequence length: `4096`
45
- - Learning rate: `1e-4` with cosine schedule
46
- - Final reported training loss: `~1.28`
47
-
48
- ## Quick usage (Transformers)
49
-
50
- ```python
51
- from transformers import AutoTokenizer, AutoModelForCausalLM
52
-
53
- model_id = "strykes/emberforge-3b-reasoner"
54
-
55
- tok = AutoTokenizer.from_pretrained(model_id)
56
- model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
57
- ```
58
-
59
- ## Quick usage (llama.cpp)
60
 
61
- Use files in `gguf/`, e.g. `Q5_K_M` for stronger quality or `Q4_K_M` for lower RAM.
 
 
 
 
62
 
63
  ## Notes
64
 
65
- - This is a finetuned model intended for research/benchmarking.
66
- - Follow upstream Nanbeige license and applicable usage policies.
67
- - Outputs can still contain errors; validate for critical tasks.
 
1
  ---
2
  language:
3
  - en
 
4
  license: apache-2.0
5
  tags:
6
  - transformers
7
  - safetensors
 
8
  - gguf
9
  - peft
10
  - qlora
11
  - reasoning
 
 
12
  base_model:
13
  - Nanbeige/Nanbeige4.1-3B
14
  library_name: transformers
 
17
 
18
  # EmberForge-3B-Reasoner
19
 
20
+ Private finetuned Nanbeige4.1-3B reasoning release by `strykes`.
21
 
22
+ ## Included Artifacts
23
 
24
+ - Merged full model (Safetensors) at repo root for HF benchmarking
25
+ - LoRA adapter in `adapter/`
26
+ - GGUF in `gguf/`:
 
 
27
  - `Nanbeige4.1-3B-Q5_K_M.gguf`
28
  - `Nanbeige4.1-3B-Q4_K_M.gguf`
29
  - `Nanbeige4.1-3B-f16.gguf`
30
+ - Optional archive in `archives/`
31
 
32
+ ## Training Snapshot
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
 
34
+ - Base: `Nanbeige/Nanbeige4.1-3B`
35
+ - Method: Unsloth QLoRA -> merged weights
36
+ - Data: ~3.5k synthetic reasoning samples
37
+ - Epochs: 2
38
+ - Sequence length: 4096
39
 
40
  ## Notes
41
 
42
+ - Intended for research and benchmarking.
43
+ - Validate outputs before critical use.