File size: 1,692 Bytes
3c55341
 
 
 
 
 
 
 
 
 
 
 
 
 
cf41862
 
3c55341
 
 
 
 
 
 
 
99559ae
 
 
 
 
cf41862
 
99559ae
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
---
license: apache-2.0
base_model: openbmb/MiniCPM5-1B
tags:
- function-calling
- tool-use
- minicpm
- automaticity
---

# MiniCPM5 Automaticity V7

Public release of the MiniCPM5 automaticity V7 LoRA and merged GGUF exports.

The training data and benchmark repos are private because future rows/results may contain real tool-call traces.
The included HTML report and benchmark dataset repo compare:

- MiniCPM5_Base
- MiniCPM5_Base + AUTOMATICITY_V7_LORA_ADAPTER
- MiniCPM5_AUTOMATICITY_V7_Q4
- MiniCPM5_AUTOMATICITY_V7_Q8
- FunctionGemma_AUTOMATICITY_V7_Q8

MiniCPM tool-call fine-tuning targets MiniCPM XML function calls rather than JSON.

## Automaticity benchmark v1

Frozen local benchmark: `/home/turnercore/automaticity-benchmark-v1/automaticity-hard-v1.jsonl`

Canonical benchmark repo: `turnercore/automaticity-benchmark-v1` (private)

| Run | Exact | Tool name | Arguments | No-op recall | p50 latency | p95 latency |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| FunctionGemma_AUTOMATICITY_V7_Q8 | 82/92 (89.1%) | 96.7% | 90.2% | 94.7% | 180 ms | 568 ms |
| MiniCPM5_Base | 78/92 (84.8%) | 92.4% | 87.0% | 86.8% | 701 ms | 2,070 ms |
| MiniCPM5_Base + AUTOMATICITY_V7_LORA_ADAPTER | 59/92 (64.1%) | 67.4% | 75.0% | 21.1% | 316 ms | 478 ms |
| MiniCPM5_AUTOMATICITY_V7_Q8 | 60/92 (65.2%) | 69.6% | 75.0% | 26.3% | 151 ms | 248 ms |
| MiniCPM5_AUTOMATICITY_V7_Q4 | 54/92 (58.7%) | 64.1% | 67.4% | 13.2% | 141 ms | 226 ms |

The V7 MiniCPM fine-tune is not promoted. It learned the XML output shape and is fast when baked to GGUF, but it overcalls no-op, negated, hypothetical, deferred, and incomplete prompts. `AUTOMATICITY_V8` adds no-op hardening rows for the next run.