update readme to match github

README.md

@@ -3,26 +3,30 @@ license: cc-by-nc-4.0
 ---
 
 # SONAR
-[[Paper]]()
+[[Paper]](https://fb.workplace.com/groups/831302610278251/permalink/9713798772028546) (TODO: change for external link once published) 
 [[Demo]](#usage)
 
-We introduce SONAR, a new multilingual and multimodal fixed-size sentence embedding space. Our single **text encoder, covering 200 languages**, substantially outperforms existing sentence embeddings such as LASER3 and LabSE on the xsim and xsim++ multilingual similarity search tasks. 
+We introduce SONAR, a new multilingual and multimodal fixed-size sentence embedding space, with a full suite of speech and text encoders and decoders. It substantially outperforms existing sentence embeddings such as LASER3 and LabSE on the xsim and xsim++ multilingual similarity search tasks. 
 
-Speech segments can be embedded in the same \sonar embedding space using language-specific speech encoders trained in a teacher-student setting on speech transcription data. Our encoders outperform existing speech encoders on similarity search tasks.
-We also provide a **text decoder for 200 languages**, which allows us to perform text-to-text and speech-to-text machine translation, including for zero-shot language and modality combinations. 
+Speech segments can be embedded in the same SONAR embedding space using language-specific speech encoders trained in a teacher-student setting on speech transcription data. We also provide a single text decoder, which allows us to perform text-to-text and speech-to-text machine translation, including for zero-shot language and modality combinations.
 
-Our text-to-text results are competitive compared to the state-of-the-art NLLB~1B model, despite the fixed-size bottleneck representation. Our zero-shot speech-to-text translation results compare favorably with strong supervised baselines such as Whisper.
+*SONAR* stands for **S**entence-level multim**O**dal and la**N**guage-**A**gnostic **R**epresentations
 
-
-Model inference support thanks [Fairseq2](https://github.com/facebookresearch/fairseq2)
+The full list of supported languages (along with download links) can be found here [below](#supported-languages-and-download-links).
 
 
 ## Installing
-
-See our github [repo](https://reimagined-broccoli-941276ee.pages.github.io/nightly/installation/from_source_conda)
+SONAR depends mainly on [Fairseq2](https://github.com/fairinternal/fairseq2) and can be installed using (tested with `python=3.8`)
+```bash
+pip install --upgrade pip
+pip config set global.extra-index-url https://test.pypi.org/simple/
+pip install -e .
+```
 
 ## Usage
-Compute text sentence embeddings:
+fairseq2 will automatically download models into your `$TORCH_HOME/hub` directory upon using the commands below.
+
+### Compute text sentence embeddings with SONAR:
 ```python
 from sonar.inference_pipelines.text import TextToEmbeddingModelPipeline
 t2vec_model = TextToEmbeddingModelPipeline(encoder="text_sonar_basic_encoder",
@@ -32,7 +36,7 @@ t2vec_model.predict(sentences, source_lang="eng_Latn").shape
 # torch.Size([2, 1024])
 ```
 
-Translate with SONAR
+### Translate text with SONAR
 ```python
 from sonar.inference_pipelines.text import TextToTextModelPipeline
 t2t_model = TextToTextModelPipeline(encoder="text_sonar_basic_encoder",
@@ -44,50 +48,47 @@ t2t_model.predict(sentences, source_lang="eng_Latn", target_lang="fra_Latn")
 # ['Mon nom est SONAR.', "Je peux intégrer les phrases dans l'espace vectoriel."]
 ```
 
-Compute speech sentence embeddings:
+### Compute speech sentence embeddings with SONAR
 ```python
-import torch
-from sonar.inference_pipelines.speech import SpeechToEmbeddingPipeline, SpeechInferenceParams
-
-speech_embedding_dp_builder = SpeechToEmbeddingPipeline.load_from_name("sonar_speech_encoder_eng")
+from sonar.inference_pipelines.speech import SpeechToEmbeddingModelPipeline
+s2vec_model = SpeechToEmbeddingModelPipeline(encoder="sonar_speech_encoder_eng")
 
-speech_ctx = SpeechInferenceParams(
-    data_file="..../test_fleurs_fra-eng.tsv",
-    audio_root_dir=".../audio_zips",
-    audio_path_index=2,
-    batch_size=4,
-)
+s2vec_model.predict(["./tests/integration_tests/data/audio_files/audio_1.wav",
+                     "./tests/integration_tests/data/audio_files/audio_2.wav"]).shape
+# torch.Size([2, 1024])
+import torchaudio
+inp, sr = torchaudio.load("./tests/integration_tests/data/audio_files/audio_1.wav")
+assert sr == 16000, "Sample rate should be 16kHz"
 
-speech_embedding_dp = speech_embedding_dp_builder.build_pipeline(speech_ctx)
-with torch.inference_mode():
-    speech_emb = next(iter(speech_embedding_dp))
-speech_emb["audio"]["data"].sentence_embeddings
+s2vec_model.predict([inp]).shape
+# torch.Size([1, 1024])
 ```
 
-
-Speech-to-text with SONAR
+### Speech-to-text translation with SONAR
 ```python
-import torch
-from sonar.inference_pipelines import SpeechToTextPipeline, SpeechInferenceParams
-
-speech_to_text_dp_builder = SpeechToTextPipeline.load_from_name(encoder_name="sonar_speech_encoder_eng", 
-                                                                decoder_name="text_sonar_basic_decoder")
-
-speech_ctx = SpeechInferenceParams(
-    data_file=".../test_fleurs_fra-eng.tsv",
-    audio_root_dir=".../audio_zips",
-    audio_path_index=2,
-    target_lang='fra_Latn',
-    batch_size=4,
-)
-speech_to_text_dp = speech_to_text_dp_builder.build_pipeline(speech_ctx)
-with torch.inference_mode():
-    speech_text_translation = next(iter(speech_to_text_dp))
-speech_text_translation
+from sonar.inference_pipelines.speech import SpeechToTextModelPipeline
+
+s2t_model = SpeechToTextModelPipeline(encoder="sonar_speech_encoder_eng",
+                                      decoder="text_sonar_basic_decoder",
+                                      tokenizer="text_sonar_basic_decoder")
+
+import torchaudio
+inp, sr = torchaudio.load("./tests/integration_tests/data/audio_files/audio_1.wav")
+assert sr == 16000, "Sample rate should be 16kHz"
+
+# passing loaded audio files
+s2t_model.predict([inp], target_lang="eng_Latn")
+# ['Television reports show white smoke coming from the plant.']
+
+# passing multiple wav files 
+s2t_model.predict(["./tests/integration_tests/data/audio_files/audio_1.wav",
+                   "./tests/integration_tests/data/audio_files/audio_2.wav"], target_lang="eng_Latn")
+# ['Television reports show white smoke coming from the plant.',
+# 'These couples may choose to make an adoption plan for their baby.']
 ```
 
-Predicting [cross-lingual semantic similarity](https://github.com/facebookresearch/fairseq/tree/nllb/examples/nllb/human_XSTS_eval) 
-with BLASER-2 models
+
+### Predicting [cross-lingual semantic similarity](https://github.com/facebookresearch/fairseq/tree/nllb/examples/nllb/human_XSTS_eval) with BLASER 2 models
 ```Python
 import torch
 from sonar.models.blaser.loader import load_blaser_model
@@ -102,6 +103,7 @@ print(blaser_qe(src=emb, mt=emb).item())  # 4.9819
 ```
 
 See more complete demo notebooks :
+
 * [sonar text2text similarity and translation](examples/sonar_text_demo.ipynb)
 * [sonar speech2text and other data pipeline examples](examples/inference_pipelines.ipynb)