Upload 10 files

Add CTranslate2 model files

README.md

@@ -1,77 +1,143 @@
 ---
+language:
+  - af
+  - am
+  - ar
+  - az
+  - be
+  - bg
+  - bn
+  - ca
+  - cs
+  - cy
+  - da
+  - de
+  - el
+  - en
+  - es
+  - et
+  - fa
+  - fi
+  - fr
+  - ga
+  - gl
+  - gu
+  - ha
+  - he
+  - hi
+  - hr
+  - hu
+  - hy
+  - id
+  - ig
+  - is
+  - it
+  - ja
+  - ka
+  - kk
+  - km
+  - ko
+  - lt
+  - lv
+  - mk
+  - ml
+  - mn
+  - mr
+  - ms
+  - mt
+  - my
+  - ne
+  - nl
+  - no
+  - pl
+  - pt
+  - ro
+  - ru
+  - si
+  - sk
+  - sl
+  - so
+  - sq
+  - sr
+  - sv
+  - sw
+  - ta
+  - te
+  - th
+  - tl
+  - tr
+  - uk
+  - ur
+  - uz
+  - vi
+  - xh
+  - yo
+  - zh
+  - zu
 license: mit
-base_model:
-- facebook/m2m100_418M
-pipeline_tag: translation
 tags:
-- ctranslate2
-- ct2
-- int8
-- translation
-- multilingual
-library_name: ctranslate2
+  - translation
+  - multilingual
+  - onnx
+  - int8
+  - ctranslate2
+  - m2m100
+  - facebook
+pipeline_tag: translation
 ---
 
-# MynaLuna Translation
-
-This repository contains an INT8-quantized CTranslate2 conversion of **Facebook M2M100 418M**, prepared for use with the MynaLuna application.
+# MynaLuna-Translation
 
-Supports **100 languages** with direct many-to-many translation without requiring English as an intermediate language.
+INT8-quantized [CTranslate2](https://github.com/OpenNMT/CTranslate2) export of **Facebook M2M100 418M**, packaged for use in the [MynaLuna](https://github.com/siatoslav/MynaLuna) desktop voice assistant.
 
-## Repository purpose
-
-This repository is maintained to provide a stable download source for the MynaLuna application.
-
-It helps ensure that users can continue downloading the required model even if the original distribution source changes or becomes unavailable.
+Supports **100 languages** with direct any-to-any translation — no pivot through English required.
 
 ## Model details
 
-| Property | Value |
-|---|---|
-| Purpose | Neural Machine Translation (NMT) |
-| Base model | Facebook M2M100 418M |
-| Architecture | M2M100 (Many-to-Many Multilingual) |
-| Parameters | 418M |
-| Conversion | CTranslate2 INT8 by MynaLuna |
-| Format | CTranslate2 |
-| Quantization | INT8 |
-| Model size | ~468 MB |
+- **Base model:** [facebook/m2m100_418M](https://huggingface.co/facebook/m2m100_418M)
+- **Architecture:** M2M100 (Many-to-Many Multilingual)
+- **Parameters:** 418M
+- **Quantization:** INT8
+- **Runtime:** CTranslate2
+- **Model size:** ~468 MB
 
 ## Files
 
-```text
-model.bin                  — INT8 quantized weights
-shared_vocabulary.json     — Shared source/target vocabulary
-vocab.json                 — Vocabulary with token IDs
-sentencepiece.bpe.model    — SentencePiece tokenizer
-tokenizer_config.json      — Tokenizer configuration
-config.json                — Model configuration
-special_tokens_map.json    — Special token mappings
-generation_config.json     — Default generation parameters
+```
+model.bin                  — INT8 quantized weights (CTranslate2 format)
+shared_vocabulary.json     — shared source/target vocabulary
+vocab.json                 — full vocabulary with token IDs
+sentencepiece.bpe.model    — SentencePiece BPE tokenizer
+tokenizer_config.json      — tokenizer configuration
+config.json                — model configuration
+special_tokens_map.json    — special token mappings
+generation_config.json     — default generation parameters
 ```
 
-## Conversion
-
-The original Facebook M2M100 418M model was converted to the CTranslate2 format and quantized to INT8 for efficient local inference.
-
-## Attribution
+## Usage with CTranslate2
 
-Base model:
+```python
+import ctranslate2
+import sentencepiece as spm
 
-https://huggingface.co/facebook/m2m100_418M
+sp = spm.SentencePieceProcessor("sentencepiece.bpe.model")
+translator = ctranslate2.Translator(".", inter_threads=1)
 
-CTranslate2 project:
+src_lang = "en"
+tgt_lang = "uk"
 
-https://github.com/OpenNMT/CTranslate2
+tokens = sp.encode(f"__{src_lang}__ Hello, world!", out_type=str)
+results = translator.translate_batch([tokens], target_prefix=[[f"__{tgt_lang}__"]])
+output = sp.decode(results[0].hypotheses[0][1:])
+print(output)  # Привіт, світе!
+```
 
 ## License
 
-This repository contains a derivative version of the original model.
-
-The original model is licensed under the **MIT License**.
-
-Please refer to the original repository for licensing details:
+MIT — same as the original M2M100 model.
 
-https://huggingface.co/facebook/m2m100_418M
+## Credits
 
----
+- Original model: [facebook/m2m100_418M](https://huggingface.co/facebook/m2m100_418M)
+- CTranslate2 runtime: [OpenNMT/CTranslate2](https://github.com/OpenNMT/CTranslate2)
+- Repackaged for MynaLuna by [Siatoslav Petrov](https://github.com/siatoslav)

config.json

@@ -0,0 +1,10 @@
+{
+  "add_source_bos": false,
+  "add_source_eos": false,
+  "bos_token": "<s>",
+  "decoder_start_token": "</s>",
+  "eos_token": "</s>",
+  "layer_norm_epsilon": null,
+  "multi_query_attention": false,
+  "unk_token": "<unk>"
+}

generation_config.json

@@ -0,0 +1,11 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 0,
+  "decoder_start_token_id": 2,
+  "early_stopping": true,
+  "eos_token_id": 2,
+  "max_length": 200,
+  "num_beams": 5,
+  "pad_token_id": 1,
+  "transformers_version": "4.27.0.dev0"
+}

model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a1826980fc5c037e69c7ac94fcb56c03001a66f380eb71863cc0a3879e71421b
+size 490667752

muna_manifest.json

@@ -0,0 +1,6 @@
+{
+  "model": "facebook/m2m100_418M",
+  "runtime": "ctranslate2",
+  "compute_type": "int8",
+  "installed_by": "_translation_model.py"
+}

sentencepiece.bpe.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d8f7c76ed2a5e0822be39f0a4f95a55eb19c78f4593ce609e2edbc2aea4d380a
+size 2423393

special_tokens_map.json

@@ -0,0 +1 @@
+{"bos_token": "<s>", "eos_token": "</s>", "unk_token": "<unk>", "sep_token": "</s>", "pad_token": "<pad>", "additional_special_tokens": ["__af__", "__am__", "__ar__", "__ast__", "__az__", "__ba__", "__be__", "__bg__", "__bn__", "__br__", "__bs__", "__ca__", "__ceb__", "__cs__", "__cy__", "__da__", "__de__", "__el__", "__en__", "__es__", "__et__", "__fa__", "__ff__", "__fi__", "__fr__", "__fy__", "__ga__", "__gd__", "__gl__", "__gu__", "__ha__", "__he__", "__hi__", "__hr__", "__ht__", "__hu__", "__hy__", "__id__", "__ig__", "__ilo__", "__is__", "__it__", "__ja__", "__jv__", "__ka__", "__kk__", "__km__", "__kn__", "__ko__", "__lb__", "__lg__", "__ln__", "__lo__", "__lt__", "__lv__", "__mg__", "__mk__", "__ml__", "__mn__", "__mr__", "__ms__", "__my__", "__ne__", "__nl__", "__no__", "__ns__", "__oc__", "__or__", "__pa__", "__pl__", "__ps__", "__pt__", "__ro__", "__ru__", "__sd__", "__si__", "__sk__", "__sl__", "__so__", "__sq__", "__sr__", "__ss__", "__su__", "__sv__", "__sw__", "__ta__", "__th__", "__tl__", "__tn__", "__tr__", "__uk__", "__ur__", "__uz__", "__vi__", "__wo__", "__xh__", "__yi__", "__yo__", "__zh__", "__zu__"]}

tokenizer_config.json

@@ -0,0 +1 @@
+{"src_lang": null, "tgt_lang": null, "bos_token": "<s>", "eos_token": "</s>", "sep_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>", "special_tokens_map_file": "m2m_100_1.2B_v2/special_tokens_map.json", "tokenizer_file": null, "name_or_path": "m2m_100_1.2B_v2/", "model_max_length": 1024}