Upload README.md with huggingface_hub

README.md

@@ -1,3 +1,169 @@
----
-license: cc-by-nc-4.0
----
+---
+language:
+- multilingual
+- en
+- ru
+- de
+- fr
+- es
+- zh
+- ja
+- ko
+- ar
+license: cc-by-nc-4.0
+library_name: transformers
+tags:
+- sonar
+- sentence-embeddings
+- multilingual
+- translation
+- text-generation
+- text2text-generation
+base_model: facebook/nllb-200-distilled-1.3B
+pipeline_tag: text2text-generation
+---
+
+# SONAR 200 Text Decoder (HuggingFace Port)
+
+This is a port of [Meta's SONAR](https://github.com/facebookresearch/SONAR) text decoder from fairseq2 to HuggingFace Transformers format.
+
+## Model Description
+
+SONAR decoder converts 1024-dimensional sentence embeddings back to text. It supports 202 languages (same as NLLB-200).
+
+- **Original model:** [facebook/SONAR](https://huggingface.co/facebook/SONAR)
+- **Encoder port:** [cointegrated/SONAR_200_text_encoder](https://huggingface.co/cointegrated/SONAR_200_text_encoder)
+- **Code & Documentation:** [GitHub: sonar-transformers](https://github.com/raxtemur/sonar-transformers)
+
+## Usage
+
+### With sonar_transformers library (recommended)
+
+```bash
+pip install torch transformers sentencepiece
+```
+
+```python
+from sonar_transformers import SonarPipeline
+
+pipeline = SonarPipeline()
+
+# Translation
+result = pipeline.translate(
+    ["Hello, how are you?"],
+    source_lang="eng_Latn",
+    target_lang="rus_Cyrl"
+)
+print(result)  # ['Здравствуйте, как дела?']
+
+# Encode text to embeddings
+embeddings = pipeline.encode(["Hello world!"], source_lang="eng_Latn")
+print(embeddings.shape)  # torch.Size([1, 1024])
+
+# Decode embeddings back to text
+texts = pipeline.decode(embeddings, target_lang="eng_Latn")
+print(texts)  # ['Hello world!']
+```
+
+### Direct usage with transformers
+
+```python
+import torch
+from transformers import M2M100ForConditionalGeneration, NllbTokenizer
+from transformers.modeling_outputs import BaseModelOutput
+
+# Load model and tokenizer
+model = M2M100ForConditionalGeneration.from_pretrained("raxtemur/SONAR_200_text_decoder")
+tokenizer = NllbTokenizer.from_pretrained("raxtemur/SONAR_200_text_decoder")
+
+# Your embeddings from SONAR encoder (1024-dim vectors)
+embeddings = torch.randn(1, 1024)  # Replace with actual embeddings
+
+# Prepare encoder outputs
+encoder_outputs = BaseModelOutput(last_hidden_state=embeddings.unsqueeze(1))
+
+# Generate text
+target_lang = "eng_Latn"
+forced_bos_token_id = tokenizer.convert_tokens_to_ids(target_lang)
+
+generated_ids = model.generate(
+    encoder_outputs=encoder_outputs,
+    forced_bos_token_id=forced_bos_token_id,
+    max_length=128,
+    num_beams=5
+)
+
+text = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)
+print(text)
+```
+
+## Compatibility
+
+Tested against original fairseq2 SONAR:
+
+| Test | Result |
+|------|--------|
+| Encoder cosine similarity | **1.000000** |
+| Decoder output match | **Identical** |
+| Round-trip (encode→decode) | **Works** |
+| Translation | **Works** |
+
+Example outputs:
+- "Hello world!" → "Hello world!" ✓
+- "This is a test sentence." → "This is a test sentence." ✓
+- eng→rus: "Hello, how are you?" → "Здравствуйте, как дела?" ✓
+- eng→deu: "Machine learning is powerful." → "Maschinelles Lernen ist mächtig." ✓
+
+## Conversion Details
+
+This model was converted from the original fairseq2 checkpoint using the following key mappings:
+
+| fairseq2 | HuggingFace |
+|----------|-------------|
+| `decoder.decoder.layers.N.encoder_decoder_attn.*` | `model.decoder.layers.N.encoder_attn.*` |
+| `decoder.decoder.layers.N.ffn.inner_proj.*` | `model.decoder.layers.N.fc1.*` |
+| `decoder.decoder.layers.N.ffn.output_proj.*` | `model.decoder.layers.N.fc2.*` |
+| `decoder.decoder.layers.N.ffn_layer_norm.*` | `model.decoder.layers.N.final_layer_norm.*` |
+| `decoder.decoder_frontend.embed.weight` | `model.decoder.embed_tokens.weight` |
+| `decoder.final_proj.weight` | `lm_head.weight` |
+
+Special tokens were reordered:
+- fairseq2: `[pad=0, unk=1, bos=2, eos=3]`
+- HuggingFace: `[bos=0, pad=1, eos=2, unk=3]`
+
+## Language Codes (FLORES-200)
+
+Common codes:
+- `eng_Latn` - English
+- `rus_Cyrl` - Russian
+- `deu_Latn` - German
+- `fra_Latn` - French
+- `spa_Latn` - Spanish
+- `zho_Hans` - Chinese (Simplified)
+- `jpn_Jpan` - Japanese
+- `kor_Hang` - Korean
+- `arb_Arab` - Arabic
+
+Full list: 202 languages from FLORES-200.
+
+## Citation
+
+```bibtex
+@article{Duquenne:2023:sonar_arxiv,
+  author = {Duquenne, Paul-Ambroise and Schwenk, Holger and Balikas, Georgios and others},
+  title = {SONAR: Sentence-Level Multimodal and Language-Agnostic Representations},
+  journal = {arXiv preprint arXiv:2308.11466},
+  year = {2023},
+}
+```
+
+## License
+
+**CC-BY-NC-4.0** (inherited from original SONAR)
+
+The model weights are derived from [Meta's SONAR](https://github.com/facebookresearch/SONAR) and are licensed under CC-BY-NC-4.0. Commercial use is not permitted.
+
+## Acknowledgments
+
+- [Meta AI](https://github.com/facebookresearch/SONAR) - Original SONAR
+- [cointegrated](https://huggingface.co/cointegrated/SONAR_200_text_encoder) - Encoder conversion inspiration