Image-Text-to-Text
MLX
Safetensors
English
qwen2_5_vl
multimodal
computer-use
cua
web-agent
agent
vision-language
conversational
8-bit precision
Instructions to use mlx-community/Fara-7B-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use mlx-community/Fara-7B-8bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("mlx-community/Fara-7B-8bit") config = load_config("mlx-community/Fara-7B-8bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
Add MLX 8-bit conversion of microsoft/Fara-7B with vision preserved and weight-level fidelity vs bf16
a39db61 verified | { | |
| "</tool_call>": 151658, | |
| "<tool_call>": 151657, | |
| "<|box_end|>": 151649, | |
| "<|box_start|>": 151648, | |
| "<|endoftext|>": 151643, | |
| "<|file_sep|>": 151664, | |
| "<|fim_middle|>": 151660, | |
| "<|fim_pad|>": 151662, | |
| "<|fim_prefix|>": 151659, | |
| "<|fim_suffix|>": 151661, | |
| "<|im_end|>": 151645, | |
| "<|im_start|>": 151644, | |
| "<|image_pad|>": 151655, | |
| "<|object_ref_end|>": 151647, | |
| "<|object_ref_start|>": 151646, | |
| "<|quad_end|>": 151651, | |
| "<|quad_start|>": 151650, | |
| "<|repo_name|>": 151663, | |
| "<|video_pad|>": 151656, | |
| "<|vision_end|>": 151653, | |
| "<|vision_pad|>": 151654, | |
| "<|vision_start|>": 151652 | |
| } | |