Instructions to use mlx-community/CodeFormulaV2-mlx-bf16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use mlx-community/CodeFormulaV2-mlx-bf16 with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("mlx-community/CodeFormulaV2-mlx-bf16") config = load_config("mlx-community/CodeFormulaV2-mlx-bf16") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps
- LM Studio
| language: en | |
| license: cdla-permissive-2.0 | |
| pipeline_tag: image-text-to-text | |
| library_name: mlx | |
| tags: | |
| - mlx | |
| - mlx-vlm | |
| - idefics3 | |
| base_model: docling-project/CodeFormulaV2 | |
| base_model_relation: quantized | |
| datasets: | |
| - ds4sd/SynthFormulaNet | |
| - ds4sd/SynthCodeNet | |
| # CodeFormulaV2-mlx-bf16 | |
| MLX bf16 conversion of [docling-project/CodeFormulaV2](https://huggingface.co/docling-project/CodeFormulaV2), produced with [`mlx_vlm.convert`](https://github.com/Blaizzy/mlx-vlm). Architecture, training data, and intended use are described on the upstream model page; this repo only changes the storage format (PyTorch safetensors → MLX safetensors, same bf16 precision). | |
| ## Usage | |
| ```bash | |
| pip install mlx-vlm | |
| ``` | |
| ```python | |
| from mlx_vlm import load, generate | |
| from mlx_vlm.prompt_utils import apply_chat_template | |
| model, processor = load("mlx-community/CodeFormulaV2-mlx-bf16") | |
| prompt = apply_chat_template(processor, model.config, "<formula>", num_images=1) | |
| result = generate( | |
| model, processor, | |
| prompt=prompt, | |
| image="path/to/image.png", | |
| temperature=0.0, | |
| ) | |
| print(result.text) | |
| ``` | |
| Use `"<formula>"` as the prompt for a math-expression image, `"<code>"` for a code-block image, per the upstream model card. | |
| ## License and attribution | |
| This is a derivative of [docling-project/CodeFormulaV2](https://huggingface.co/docling-project/CodeFormulaV2), redistributed under the same [Community Data License Agreement – Permissive 2.0 (CDLA-Permissive-2.0)](https://cdla.dev/permissive-2-0/). | |
| Please cite the upstream work: | |
| ```bibtex | |
| @techreport{Docling, | |
| author = {Deep Search Team}, | |
| month = {8}, | |
| title = {{Docling Technical Report}}, | |
| url = {https://arxiv.org/abs/2408.09869}, | |
| eprint = {2408.09869}, | |
| year = {2024} | |
| } | |
| ``` | |