shc2012 commited on
Commit
56be81c
·
verified ·
1 Parent(s): 8bf7a8e

Add BF16 README

Browse files
Files changed (1) hide show
  1. bf16/README.md +50 -0
bf16/README.md ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # shenwen-coderV2 BF16 GGUF
2
+
3
+ This folder contains the **BF16 (BFloat16)** quantized version of [shenwenAI/shenwen-coderV2-Instruct](https://huggingface.co/shenwenAI/shenwen-coderV2-Instruct).
4
+
5
+ ## Model Info
6
+
7
+ - **Architecture**: Qwen2 (0.5B parameters)
8
+ - **Context Length**: 32,768 tokens
9
+ - **Quantization**: BF16 (BFloat16)
10
+ - **Original Precision**: bfloat16
11
+
12
+ ## Usage with swllm.cpp
13
+
14
+ We recommend using our optimized [swllm.cpp](https://github.com/shenwenAI/swllm.cpp) for best performance:
15
+
16
+ ```bash
17
+ # Build swllm.cpp (first time only)
18
+ cd swllm.cpp
19
+ mkdir build && cd build
20
+ cmake .. && make -j4
21
+
22
+ # Run inference
23
+ ./swllm -m shenwen-coderV2-BF16.gguf -p "Write a hello world in Python:" --top-p 0.9 -t 8
24
+ ```
25
+
26
+ ### Key Parameters
27
+
28
+ | Parameter | Description | Recommended |
29
+ |----------|-------------|-------------|
30
+ | `-m` | Model path | Required |
31
+ | `-t` | Thread count | CPU cores |
32
+ | `-p` | Prompt | Your code prompt |
33
+ | `--top-p` | Top-p sampling | 0.9 |
34
+ | `-n` | Max tokens | 512-1024 |
35
+
36
+ ## Model Files
37
+
38
+ | File | Size | Description |
39
+ |------|------|-------------|
40
+ | shenwen-coderV2-BF16.gguf | ~949 MB | Full BF16 precision |
41
+
42
+ ## Connect With Us
43
+
44
+ [![GitHub](https://img.shields.io/badge/GitHub-shenwenAI-blue?style=flat&logo=github)](https://github.com/shenwenAI)
45
+ [![Hugging Face](https://img.shields.io/badge/HuggingFace-shenwenAI-yellow?style=flat&logo=huggingface)](https://huggingface.co/shenwenAI)
46
+ [![Twitter](https://img.shields.io/badge/Twitter-shenwenai-1DA1F2?style=flat&logo=twitter)](https://x.com/shenwenai)
47
+
48
+ - GitHub: https://github.com/shenwenAI
49
+ - Hugging Face: https://huggingface.co/shenwenAI
50
+ - Twitter/X: https://x.com/shenwenai