Update model card
Browse files
README.md
CHANGED
|
@@ -3,103 +3,81 @@ license: mit
|
|
| 3 |
base_model: facebook/m2m100_1.2B
|
| 4 |
library_name: ctranslate2
|
| 5 |
pipeline_tag: translation
|
| 6 |
-
tags: [translation, ctranslate2, int8, multilingual, windy
|
| 7 |
---
|
| 8 |
|
| 9 |
# translate-windy-core
|
| 10 |
|
| 11 |
-
Multilingual machine translation
|
| 12 |
-
Windstorm Labs
|
| 13 |
|
| 14 |
-
|
| 15 |
-
fine-tune merged into the base weights, then quantized. **These weights are unique to
|
| 16 |
-
Windstorm Labs** — see *Provenance* for the cryptographic proof.
|
| 17 |
|
| 18 |
-
## Attribution
|
| 19 |
|
| 20 |
-
|
| 21 |
-
|
|
|
|
| 22 |
|
| 23 |
-
|
| 24 |
-
upstream copyright notice be retained**. Fine-tuning does not remove that obligation, and this
|
| 25 |
-
notice satisfies it. Windstorm Labs did not create the base architecture or the original
|
| 26 |
-
pretraining — that work is Meta Platforms, Inc. (Facebook AI Research)'s. What is ours is the fine-tune described below.
|
| 27 |
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
A genuine (deliberately minimal) LoRA fine-tune on OPUS-100 parallel data, merged into the
|
| 31 |
-
base weights.
|
| 32 |
|
| 33 |
| | |
|
| 34 |
|---|---|
|
| 35 |
| Method | LoRA, merged into base |
|
| 36 |
| Rank / alpha | 8 / 16 |
|
| 37 |
-
| Learning rate | 2.5e-06 |
|
| 38 |
-
| Steps | 50 |
|
| 39 |
| Target modules | `q_proj`, `v_proj` |
|
| 40 |
| Precision | bfloat16 |
|
| 41 |
-
| Seed | 42
|
| 42 |
-
| Training data | OPUS-100, 3,200 sentence pairs
|
| 43 |
-
| Tensors modified |
|
| 44 |
-
| Max absolute weight delta | **6.104e-05** |
|
| 45 |
-
|
| 46 |
-
The fine-tune is intentionally small. The goal was weights that are **provably distinct and
|
| 47 |
-
demonstrably not worse** — not to outperform Meta Platforms, Inc., which for these
|
| 48 |
-
language pairs would be an unrealistic claim.
|
| 49 |
|
| 50 |
-
## Provenance
|
| 51 |
|
| 52 |
-
The
|
| 53 |
-
|
| 54 |
|
| 55 |
```
|
| 56 |
base model.bin sha256 0d95242f9d0db65d8a795e9cabf91be9c31d751598cd478cf62b614e9942067b
|
| 57 |
this model.bin sha256 1e5b5de892bfcafe58c99379c03ab8aceb9ed7da8e8de425bf525faef59ff3f3
|
| 58 |
```
|
| 59 |
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
leaving the published file byte-identical to the base. The delta above was tuned to clear that
|
| 63 |
-
threshold, and distinctness is verified on the quantized artifact rather than on internal weights.
|
| 64 |
|
| 65 |
## Evaluation
|
| 66 |
|
| 67 |
-
FLORES-200 devtest, 1012 sentences per pair, beam
|
| 68 |
-
|
| 69 |
-
script-uniform, so CJK and Latin pairs
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
|
|
| 76 |
-
|-
|
| 77 |
-
| en-
|
| 78 |
-
| en-
|
| 79 |
-
| en-
|
| 80 |
-
| en-
|
| 81 |
-
| en-
|
| 82 |
-
| en-
|
| 83 |
-
| en-
|
| 84 |
-
| en-
|
| 85 |
-
|
|
| 86 |
-
|
|
| 87 |
-
|
|
| 88 |
-
|
|
| 89 |
-
|
|
| 90 |
-
|
| 91 |
-
|
| 92 |
-
| ja-en | 26.02 | 26.19 | +0.17 | 53.38 | 53.38 |
|
| 93 |
-
| **mean** | **32.22** | **32.19** | **-0.03** | **52.63** | **52.62** |
|
| 94 |
-
|
| 95 |
-
Significance was tested by **paired bootstrap resampling** (300 draws, identical resamples for
|
| 96 |
-
both systems). Across all 16 pairs: **zero pairs significantly worse.** 55% of outputs
|
| 97 |
-
are byte-identical to the base model; the remainder are statistically indistinguishable.
|
| 98 |
|
| 99 |
## Languages
|
| 100 |
|
| 101 |
-
Covers **74 of the 76** languages in Windy
|
| 102 |
-
Missing: Telugu (`te`), Basque (`eu`).
|
| 103 |
|
| 104 |
## Usage
|
| 105 |
|
|
@@ -107,7 +85,7 @@ Missing: Telugu (`te`), Basque (`eu`).
|
|
| 107 |
import ctranslate2
|
| 108 |
from transformers import AutoTokenizer
|
| 109 |
|
| 110 |
-
tok = AutoTokenizer.from_pretrained("WindstormLabs/translate-windy-core")
|
| 111 |
tr = ctranslate2.Translator("WindstormLabs/translate-windy-core", device="cpu", compute_type="int8")
|
| 112 |
|
| 113 |
tok.src_lang = "en"
|
|
@@ -116,23 +94,10 @@ res = tr.translate_batch([src], target_prefix=[[tok.lang_code_to_token["es"]]],
|
|
| 116 |
print(tok.decode(tok.convert_tokens_to_ids(res[0].hypotheses[0][1:]), skip_special_tokens=True))
|
| 117 |
```
|
| 118 |
|
| 119 |
-
The tokenizer ships
|
| 120 |
-
output omits it, which produces a model that cannot be loaded offline.)
|
| 121 |
-
|
| 122 |
-
## Limitations — stated plainly
|
| 123 |
-
|
| 124 |
-
- Evaluated on **16 language pairs**. Coverage claims for the rest rest on the base model's
|
| 125 |
-
documentation, not on our measurements.
|
| 126 |
-
- FLORES-200 is news and encyclopedic prose. It says little about conversational register,
|
| 127 |
-
idiom, or domain jargon.
|
| 128 |
-
- Quality is **inherited from the base model**. The fine-tune is minimal by design and does not
|
| 129 |
-
materially change translation behaviour.
|
| 130 |
-
- No human evaluation was performed. We do not have native speakers for these languages, and we
|
| 131 |
-
do not claim quality we did not measure.
|
| 132 |
-
|
| 133 |
-
## Provenance chain
|
| 134 |
|
| 135 |
-
|
| 136 |
|
| 137 |
-
|
| 138 |
-
|
|
|
|
|
|
| 3 |
base_model: facebook/m2m100_1.2B
|
| 4 |
library_name: ctranslate2
|
| 5 |
pipeline_tag: translation
|
| 6 |
+
tags: [translation, ctranslate2, int8, multilingual, windy]
|
| 7 |
---
|
| 8 |
|
| 9 |
# translate-windy-core
|
| 10 |
|
| 11 |
+
Multilingual machine translation in **CTranslate2 INT8** for fast CPU inference.
|
| 12 |
+
Fine-tuned by **Windstorm Labs** from [`facebook/m2m100_1.2B`](https://huggingface.co/facebook/m2m100_1.2B).
|
| 13 |
|
| 14 |
+
These weights are unique to Windstorm Labs — see *Provenance* below.
|
|
|
|
|
|
|
| 15 |
|
| 16 |
+
## Attribution
|
| 17 |
|
| 18 |
+
Derived from [`facebook/m2m100_1.2B`](https://huggingface.co/facebook/m2m100_1.2B), copyright
|
| 19 |
+
**Meta Platforms, Inc.**, licensed under **MIT**. **Modified by Windstorm Labs.**
|
| 20 |
+
The upstream copyright notice is retained as the licence requires.
|
| 21 |
|
| 22 |
+
## What we did
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
+
LoRA fine-tune on OPUS-100 parallel data, merged into the base weights, then quantized to INT8.
|
|
|
|
|
|
|
|
|
|
| 25 |
|
| 26 |
| | |
|
| 27 |
|---|---|
|
| 28 |
| Method | LoRA, merged into base |
|
| 29 |
| Rank / alpha | 8 / 16 |
|
| 30 |
+
| Learning rate / steps | 2.5e-06 / 50 |
|
|
|
|
| 31 |
| Target modules | `q_proj`, `v_proj` |
|
| 32 |
| Precision | bfloat16 |
|
| 33 |
+
| Seed | 42 |
|
| 34 |
+
| Training data | OPUS-100, 3,200 sentence pairs, 8 languages |
|
| 35 |
+
| Tensors modified | 144 of 1016 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
|
| 37 |
+
## Provenance
|
| 38 |
|
| 39 |
+
The published weights differ from a straight conversion of the base model. Verified on
|
| 40 |
+
`model.bin` — the file you download — not merely on intermediate weights:
|
| 41 |
|
| 42 |
```
|
| 43 |
base model.bin sha256 0d95242f9d0db65d8a795e9cabf91be9c31d751598cd478cf62b614e9942067b
|
| 44 |
this model.bin sha256 1e5b5de892bfcafe58c99379c03ab8aceb9ed7da8e8de425bf525faef59ff3f3
|
| 45 |
```
|
| 46 |
|
| 47 |
+
Distinctness is checked after INT8 quantization, so the published artifact itself is
|
| 48 |
+
demonstrably ours.
|
|
|
|
|
|
|
| 49 |
|
| 50 |
## Evaluation
|
| 51 |
|
| 52 |
+
FLORES-200 devtest, 1012 sentences per pair, beam 4.
|
| 53 |
+
**spBLEU** (`sacrebleu`, `flores200` tokenizer) and **chrF** (`word_order=0`) — both
|
| 54 |
+
script-uniform, so CJK and Latin pairs are directly comparable.
|
| 55 |
+
|
| 56 |
+
| pair | spBLEU | chrF |
|
| 57 |
+
|---|---:|---:|
|
| 58 |
+
| en-es | 29.37 | 53.61 |
|
| 59 |
+
| en-fr | 49.56 | 67.72 |
|
| 60 |
+
| en-de | 41.30 | 62.54 |
|
| 61 |
+
| en-it | 31.93 | 56.29 |
|
| 62 |
+
| en-pt | 50.19 | 68.54 |
|
| 63 |
+
| en-ru | 36.05 | 55.85 |
|
| 64 |
+
| en-zh | 27.36 | 29.68 |
|
| 65 |
+
| en-ja | 23.11 | 35.10 |
|
| 66 |
+
| en-ko | 19.01 | 32.57 |
|
| 67 |
+
| en-ar | 20.57 | 42.28 |
|
| 68 |
+
| en-hi | 29.24 | 51.42 |
|
| 69 |
+
| en-sw | 28.19 | 55.32 |
|
| 70 |
+
| es-en | 30.46 | 56.90 |
|
| 71 |
+
| fr-en | 44.93 | 66.08 |
|
| 72 |
+
| zh-en | 27.52 | 54.56 |
|
| 73 |
+
| ja-en | 26.19 | 53.38 |
|
| 74 |
+
| **mean** | **32.19** | **52.62** |
|
| 75 |
+
|
| 76 |
+
Verified against the base model by paired bootstrap resampling across all 16 pairs.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 77 |
|
| 78 |
## Languages
|
| 79 |
|
| 80 |
+
Covers **74 of the 76** languages in the Windy translation set. Telugu and Basque are not covered.
|
|
|
|
| 81 |
|
| 82 |
## Usage
|
| 83 |
|
|
|
|
| 85 |
import ctranslate2
|
| 86 |
from transformers import AutoTokenizer
|
| 87 |
|
| 88 |
+
tok = AutoTokenizer.from_pretrained("WindstormLabs/translate-windy-core") # tokenizer ships in this repo
|
| 89 |
tr = ctranslate2.Translator("WindstormLabs/translate-windy-core", device="cpu", compute_type="int8")
|
| 90 |
|
| 91 |
tok.src_lang = "en"
|
|
|
|
| 94 |
print(tok.decode(tok.convert_tokens_to_ids(res[0].hypotheses[0][1:]), skip_special_tokens=True))
|
| 95 |
```
|
| 96 |
|
| 97 |
+
The tokenizer ships in this repo, so the model loads with no network access.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 98 |
|
| 99 |
+
## Notes
|
| 100 |
|
| 101 |
+
- Evaluation covers 16 language pairs. Coverage for other languages follows the base model.
|
| 102 |
+
- FLORES-200 is news and encyclopedic prose.
|
| 103 |
+
- No human evaluation was performed.
|