conceptnetUk commited on
Commit
ae0fed8
·
verified ·
1 Parent(s): acf2805

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +27 -9
README.md CHANGED
@@ -23,16 +23,34 @@ license: mit
23
 
24
  Fine-tuned `distilbert-base-multilingual-cased` on the ConceptNet 4-layer enterprise voice intent taxonomy.
25
 
 
 
 
 
 
 
 
26
  ## Accuracy
27
- - Overall: **98.6%**
28
- - Layer 3 Predictive: **100% precision**
29
- - Trained on 584 examples, tested on 146
 
 
 
 
 
 
 
 
 
 
 
30
 
31
- ## The 4-Layer Taxonomy
32
- - **Layer 1 — Basic:** "Do X" — immediate execution
33
- - **Layer 2 — Context-Aware:** "Do X when Y" — conditional execution
34
- - **Layer 3 — Predictive:** "Do X before Y" — proactive execution
35
- - **Layer 4 — Autonomous:** "Do X always" — persistent agent deployment
36
 
37
  ## Languages
38
  English · French · Spanish · German · Italian · Portuguese · Chinese · Arabic · Russian
@@ -49,4 +67,4 @@ classifier("Send the report when the contract is signed")
49
  - GitHub: https://github.com/wushu75/ConceptNet
50
  - Website: https://conceptnet.co.uk
51
 
52
- *Patents pending · © 2026 ConceptNet Ltd*
 
23
 
24
  Fine-tuned `distilbert-base-multilingual-cased` on the ConceptNet 4-layer enterprise voice intent taxonomy.
25
 
26
+ ## Independent Evaluation — August 2026
27
+
28
+ Independently tested by the Hugging Face community (John6666):
29
+ - Neural model: **99.315%** on reconstructed public test split
30
+ - Grouped lexical-family holdout: **99.78%**
31
+ - Conclusion: "The obvious train/test leakage explanation did not survive that check"
32
+
33
  ## Accuracy
34
+ - Fast-path classifier: **83%** · <5ms latency
35
+ - Neural model: **98.6%** (99.3% independently verified)
36
+ - L3 Predictive precision: **100%**
37
+ - Dataset: 757 examples across 9 languages (updated with L3 paraphrase diversity)
38
+
39
+ ## Cascade Performance
40
+ | Threshold | Fast coverage | Fast accuracy | Final accuracy |
41
+ |-----------|--------------|---------------|----------------|
42
+ | 0.50 | 69.2% | 95.0% | 95.9% |
43
+ | 0.55 | 60.3% | 98.9% | 98.6% |
44
+ | 0.65 | 43.2% | 100% | 99.3% |
45
+
46
+ ## Layer Precedence (mixed semantics)
47
+ When a command contains signals from multiple layers: **L4 > L3 > L2 > L1**
48
 
49
+ ## The 4 Layers
50
+ - **L1 Basic** "Do X" — immediate execution
51
+ - **L2 Context-Aware** "Do X when Y" — conditional
52
+ - **L3 Predictive** "Do X before/ahead of/prior to Y" — proactive
53
+ - **L4 Autonomous** "Do X always" — persistent agent
54
 
55
  ## Languages
56
  English · French · Spanish · German · Italian · Portuguese · Chinese · Arabic · Russian
 
67
  - GitHub: https://github.com/wushu75/ConceptNet
68
  - Website: https://conceptnet.co.uk
69
 
70
+ *© 2026 ConceptNet Ltd · Patents pending*