fredoline005 commited on
Commit
6bf7ea8
·
verified ·
1 Parent(s): 76a8dc6

Limitations: the undistilled-e5-base ablation has been run, it is in the table

Browse files

The limitations section still said the ablation was not yet run while the benchmark table above it reported that ablation. Corrected, and states plainly that the ablation beats this model on the 4-task average and loses on retrieval.

Files changed (1) hide show
  1. README.md +4 -2
README.md CHANGED
@@ -63,8 +63,10 @@ Use `query: ` / `passage: ` prefixes for retrieval (inherited from the e5 family
63
 
64
  ## Limitations (honest)
65
  - Benchmarked on **2 Turkish tasks, no confidence intervals** — margins are modest.
66
- - The win over e5-small is partly the larger base; an undistilled-e5-base ablation is
67
- not yet run.
 
 
68
  - Distillation is bounded by the teacher; the native-Turkish-tokenizer edge is v1.
69
  - Pin a corpus snapshot + add PII/dedup filtering for a production retrain.
70
 
 
63
 
64
  ## Limitations (honest)
65
  - Benchmarked on **2 Turkish tasks, no confidence intervals** — margins are modest.
66
+ - The win over e5-small is partly the larger base. The undistilled-e5-base ablation
67
+ HAS now been run and is in the table above: it beats this model on the 4-task
68
+ average (0.624 vs 0.613) and loses to it on retrieval (0.444 vs 0.465). Distillation
69
+ bought retrieval and cost general STS.
70
  - Distillation is bounded by the teacher; the native-Turkish-tokenizer edge is v1.
71
  - Pin a corpus snapshot + add PII/dedup filtering for a production retrain.
72