Translation
Transformers
Safetensors
MLX
Japanese
English
mistral
text-generation
machine-translation
japanese
english
mlx-my-repo
text-generation-inference
4-bit precision
Instructions to use moutons/CAT-Translate-7b-mlx-4Bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use moutons/CAT-Translate-7b-mlx-4Bit with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "translation" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # 'pip install "transformers<5.0.0' from transformers import pipeline pipe = pipeline("translation", model="moutons/CAT-Translate-7b-mlx-4Bit")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("moutons/CAT-Translate-7b-mlx-4Bit") model = AutoModelForCausalLM.from_pretrained("moutons/CAT-Translate-7b-mlx-4Bit") - MLX
How to use moutons/CAT-Translate-7b-mlx-4Bit with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir CAT-Translate-7b-mlx-4Bit moutons/CAT-Translate-7b-mlx-4Bit
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
| from transformers import pipeline | |
| model_name = "CyberAgent/CAT-Translate-7b" | |
| chat_pipeline = pipeline("text-generation", model_name) | |
| prompt = "Translate the following {src_lang} text into {tgt_lang}.\n\n {src_text}" | |
| src_lang = "Japanese" | |
| tgt_lang = "English" | |
| src_text = "🐈はとてもかわいいの。おててがまるくてふわふわなの。" | |
| user_input = [{"role": "user", "content": prompt.format(src_lang=src_lang, tgt_lang=tgt_lang, src_text=src_text)}] | |
| response = chat_pipeline(user_input) | |
| print("-" * 20) | |
| print("Source Text:") | |
| print(src_text) | |
| print("Translation:") | |
| print(response[0]['generated_text'][-1]['content']) |