Translation
Transformers
Safetensors
MLX
Japanese
English
mistral
text-generation
machine-translation
japanese
english
mlx-my-repo
text-generation-inference
4-bit precision
Instructions to use moutons/CAT-Translate-7b-mlx-4Bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use moutons/CAT-Translate-7b-mlx-4Bit with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "translation" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # 'pip install "transformers<5.0.0' from transformers import pipeline pipe = pipeline("translation", model="moutons/CAT-Translate-7b-mlx-4Bit")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("moutons/CAT-Translate-7b-mlx-4Bit") model = AutoModelForCausalLM.from_pretrained("moutons/CAT-Translate-7b-mlx-4Bit") - MLX
How to use moutons/CAT-Translate-7b-mlx-4Bit with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir CAT-Translate-7b-mlx-4Bit moutons/CAT-Translate-7b-mlx-4Bit
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
File size: 637 Bytes
f79d2f3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | from transformers import pipeline
model_name = "CyberAgent/CAT-Translate-7b"
chat_pipeline = pipeline("text-generation", model_name)
prompt = "Translate the following {src_lang} text into {tgt_lang}.\n\n {src_text}"
src_lang = "Japanese"
tgt_lang = "English"
src_text = "🐈はとてもかわいいの。おててがまるくてふわふわなの。"
user_input = [{"role": "user", "content": prompt.format(src_lang=src_lang, tgt_lang=tgt_lang, src_text=src_text)}]
response = chat_pipeline(user_input)
print("-" * 20)
print("Source Text:")
print(src_text)
print("Translation:")
print(response[0]['generated_text'][-1]['content']) |