| | --- |
| | license: gemma |
| | datasets: |
| | - NbAiLab/aurora-sft-2512-filtered |
| | language: |
| | - 'no' |
| | - nb |
| | - nn |
| | base_model: NbAiLab/borealis-12b-instruct-preview |
| | pipeline_tag: image-text-to-text |
| | library_name: transformers |
| | tags: |
| | - conversational |
| | - instruct |
| | - experimental |
| | - mlx |
| | - mlx-my-repo |
| | --- |
| | |
| | # Borealis 12B Instruct Preview MLX (8bit) |
| |
|
| | Converted to **MLX** from [NbAiLab/borealis-12b-instruct-preview](https://huggingface.co/NbAiLab/borealis-12b-instruct-preview) using `mlx-lm` **0.29.1**. |
| |
|
| | **Repo:** https://huggingface.co/NbAiLab/borealis-12b-instruct-preview-mlx-8bit |
| |
|
| | ## Use with mlx |
| |
|
| | ```bash |
| | pip install mlx-lm |
| | ``` |
| |
|
| | ```python |
| | from mlx_lm import load, generate |
| | |
| | model, tokenizer = load("versae/borealis-12b-instruct-preview-mlx-8bit") |
| | |
| | prompt = "hei :)" |
| | |
| | if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None: |
| | messages = [{"role": "user", "content": prompt}] |
| | prompt = tokenizer.apply_chat_template( |
| | messages, tokenize=False, add_generation_prompt=True |
| | ) |
| | |
| | response = generate(model, tokenizer, prompt=prompt, verbose=True) |
| | print(response) |
| | ``` |
| |
|
