3morixd commited on
Commit
b444a57
·
verified ·
1 Parent(s): 8b95597

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +60 -35
README.md CHANGED
@@ -1,55 +1,80 @@
1
  ---
2
  license: apache-2.0
3
- base_model: openbmb/MiniCPM5-1B
 
 
4
  tags:
5
- - speculative-decoding-draft [dispatch-ai, mobile, quantized, gguf, phone-farm-tested]
 
 
 
 
6
  pipeline_tag: text-generation
7
- language: [en]
8
  ---
 
9
  # MiniCPM5-1B-multilingual-mobile
10
- **Dispatch AI** — Built for mobile. Tested on real phones.
11
- ## Category
12
- Multilingual — Q5 higher quality
13
- ## Model
14
- Re-engineered from [openbmb/MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B).
15
- Size: 656 MB. Q4_K_M GGUF for llama.cpp.
16
- ## Usage
17
- ```bash
18
- ./llama-cli -m model.gguf -p "Hello" -n 100 -t 4 -c 512
19
  ```
20
- 🌐 [dispatchAI on HuggingFace](https://huggingface.co/dispatchAI)
21
 
 
 
 
 
 
 
 
 
 
22
 
23
- ## Speculative Decoding Draft Model
24
 
25
- This model is optimized for use as a **draft model** in speculative decoding setups.
26
 
27
- ### What is speculative decoding?
28
- Speculative decoding pairs a small, fast "draft" model with a larger "target" model.
29
- The draft model proposes tokens that the target model verifies in parallel, achieving
30
- 2-3x speedup with zero quality loss.
31
 
32
- ### Why this model?
33
- - **Small and fast**: Sub-1B parameters = minimal draft overhead
34
- - **Mobile-optimized**: Already quantized and pruned for edge deployment
35
- - **Same family**: Pairs naturally with larger models of the same architecture
 
 
 
 
 
 
 
 
 
 
 
 
36
 
37
- ### Usage with vLLM
38
  ```python
39
- from vllm import LLM, SamplingParams
40
 
41
- llm = LLM(
42
- model="target-model-7b",
43
- speculative_model="dispatchAI/MiniCPM5-1B-multilingual-mobile",
44
- num_speculative_tokens=5,
45
  )
 
46
  ```
47
 
48
- ### Usage with transformers
49
  ```python
50
- from transformers import AutoModelForCausalLM, AutoTokenizer
51
-
52
- target = AutoModelForCausalLM.from_pretrained("target-model-7b")
53
- draft = AutoModelForCausalLM.from_pretrained("dispatchAI/MiniCPM5-1B-multilingual-mobile")
54
- # See transformers docs for assisted_generation
55
  ```
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ language:
4
+ - en
5
+ library_name: transformers
6
  tags:
7
+ - mobile
8
+ - on-device
9
+ - quantized
10
+ - gguf
11
+ - dispatchai
12
  pipeline_tag: text-generation
 
13
  ---
14
+
15
  # MiniCPM5-1B-multilingual-mobile
16
+
17
+ ⚠️ **PARTIAL** — Verified June 2026.
18
+
19
+ ## Verification Results
20
+
21
+ | Prompt | Response | Correct? |
22
+ |--------|----------|----------|
23
+ | What is the capital of France? | "" | ⚠️ |
24
+ | What is 2+2? Just the number. | "```t
25
  ```
 
26
 
27
+ ####
28
+
29
+ ####
30
+
31
+ ####
32
+
33
+ ####
34
+
35
+ ####
36
 
37
+ ####
38
 
39
+ ####
40
 
41
+ ####
 
 
 
42
 
43
+ ##" | ⚠️ |
44
+
45
+ **Chat format**: `chatml`
46
+
47
+ ## Model Details
48
+
49
+ | Attribute | Value |
50
+ |-----------|-------|
51
+ | **Base Model** | openbmb/MiniCPM3-4B |
52
+ | **File Size** | 656 MB |
53
+ | **Format** | GGUF |
54
+ | **Chat Format** | chatml |
55
+ | **CPU Speed** | 18.1 tokens/sec |
56
+ | **License** | apache-2.0 |
57
+
58
+ ## Usage
59
 
 
60
  ```python
61
+ from llama_cpp import Llama
62
 
63
+ llm = Llama(model_path="model.gguf", chat_format="chatml", n_ctx=512, n_threads=4)
64
+ response = llm.create_chat_completion(
65
+ messages=[{"role": "user", "content": "What is the capital of France?"}],
66
+ max_tokens=50,
67
  )
68
+ print(response["choices"][0]["message"]["content"])
69
  ```
70
 
71
+ ### dispatchAI SDK
72
  ```python
73
+ from dispatchai import load_model
74
+ model = load_model("MiniCPM5-1B-multilingual-mobile", backend="gguf")
75
+ print(model.chat("Hello!"))
 
 
76
  ```
77
+
78
+ ## About dispatchAI
79
+
80
+ [dispatchAI](https://huggingface.co/dispatchAI) — Small. Mobile. Free. UAE-built.