Instructions to use Diabase/europa-9b-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Diabase/europa-9b-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="Diabase/europa-9b-GGUF",
	filename="gemma-2-9b-it.Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Diabase/europa-9b-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Diabase/europa-9b-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Diabase/europa-9b-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Diabase/europa-9b-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Diabase/europa-9b-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Diabase/europa-9b-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Diabase/europa-9b-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Diabase/europa-9b-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Diabase/europa-9b-GGUF:Q4_K_M

Use Docker

docker model run hf.co/Diabase/europa-9b-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use Diabase/europa-9b-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Diabase/europa-9b-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Diabase/europa-9b-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Diabase/europa-9b-GGUF:Q4_K_M

Ollama
How to use Diabase/europa-9b-GGUF with Ollama:
```
ollama run hf.co/Diabase/europa-9b-GGUF:Q4_K_M
```

Unsloth Studio

How to use Diabase/europa-9b-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Diabase/europa-9b-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Diabase/europa-9b-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Diabase/europa-9b-GGUF to start chatting

Atomic Chat new
Docker Model Runner
How to use Diabase/europa-9b-GGUF with Docker Model Runner:
```
docker model run hf.co/Diabase/europa-9b-GGUF:Q4_K_M
```

Lemonade

How to use Diabase/europa-9b-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Diabase/europa-9b-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.europa-9b-GGUF-Q4_K_M

List all available models

lemonade list

Diabase Europa 9B v1.0

A locally runnable, Swedish-enhanced European AI assistant built on Gemma 2 9B Instruct.

Diabase Europa 9B is a light fine-tune for Swedish and European enterprise use: clearer instruction following, better Swedish assistant behavior, and practical answers on AI, data, compliance, and sovereign deployment topics. It is designed to run on your own hardware without sending prompts to a third-party API.

This repository ships a merged GGUF (Q4_K_M) plus an Ollama Modelfile with the release system prompt baked in. One download, three commands, ready to chat.


Release	v1.0 (`v5.0-light` adapter, merged)
Base model	Gemma 2 9B Instruct
Format	GGUF Q4_K_M (~5.5 GB)
Context	4096 tokens
VRAM (GPU)	~6 GB+ recommended
RAM (CPU)	8 GB+ minimum, 16 GB recommended
LoRA adapter	diabase/europa-9b-v1.0-lora (advanced)

Quick start (Ollama, recommended)

Prerequisites: Ollama installed, Gemma license accepted on Hugging Face if required by your org.

# 1. Download model files from this repo
huggingface-cli download diabase/europa-9b-GGUF gemma-2-9b-it.Q4_K_M.gguf Modelfile

# 2. Create the local model (includes Diabase system prompt)
cd <download-folder>
ollama create diabase-europa-9b -f Modelfile

# 3. Run
ollama run diabase-europa-9b

Try it:

Svara på svenska: Vad är Diabase Europa och hur skiljer det sig från amerikanska AI-tjänster?

Expected behavior: third-person description, natural Swedish, uses öppna vikter rather than invented calques like öppenkälla.

What is in this repo

File	Purpose
`gemma-2-9b-it.Q4_K_M.gguf`	Quantized merged model (base + Diabase adapter)
`Modelfile`	Ollama config with Gemma chat template + bundled system prompt
`README.md`	This model card

The system prompt is intentionally public in Modelfile. It is not hidden during inspection (ollama show diabase-europa-9b --modelfile), but it is not shown in normal chat. You may customize it for your deployment.

What this model is

A Gemma 2 9B Instruct derivative with a light QLoRA SFT adapter merged in
Tuned for Swedish assistant behavior, European context, and enterprise AI topics
Open-weight and locally deployable on consumer hardware
Transparently evaluated on EuroEval Swedish benchmarks

What this model is not

Not trained from scratch on Swedish corpora
Not a frontier-scale model competing with GPT-4 class systems on raw capability
Not a safety-certified or compliance-certified product out of the box
Not a replacement for legal, medical, or security review by qualified professionals

Intended use

Swedish and multilingual assistant workflows in European organizations
On-prem or private cloud deployments where data residency matters
AI/ML, data engineering, compliance, and sovereign AI education and drafting
Prototyping European AI products before heavier customization

Out of scope

Fully autonomous decision-making without human review
High-stakes legal, medical, or financial advice without expert validation
Claims of automatic GDPR, NIS2, or EU AI Act compliance from model use alone
Environments requiring formal safety red-teaming or guaranteed factual accuracy

Evaluation (EuroEval Swedish)

Compared against the same base model (unsloth/gemma-2-9b-it-bnb-4bit) under identical eval settings (4-bit, vLLM, 4096 context, Triton).

Task	Metric	Base Gemma 2 9B	Diabase v1.0	Delta
IFEval-sv	accuracy	75.53	77.23	+1.70
HellaSwag-sv	MCC	56.91	57.45	+0.54
HellaSwag-sv	accuracy	67.30	67.77	+0.47
multi-wiki-qa-sv	F1	74.69	74.10	-0.59
multi-wiki-qa-sv	EM	55.02	54.59	-0.43
MMLU-sv	MCC	51.95	51.21	-0.74
MMLU-sv	accuracy	63.75	63.20	-0.55
SUC3 (NER)	micro-F1	32.68	33.41	+0.73
SUC3 (NER)	micro-F1-no-misc	50.69	51.77	+1.08
SweReC	MCC	74.11	73.34	-0.77
ScaLA-sv	MCC	56.35	56.23	-0.12

Summary: Knowledge and reasoning are preserved within noise. The clearest win is instruction following (+1.7 IFEval). This is a polish on a strong base, not a leaderboard revolution. That is by design: we optimized for a safe, deployable Swedish assistant rather than benchmark chasing at the cost of general capability.

Full baseline documented in the Diabase model lab release materials.

Example output

Prompt: Vad är Diabase Europa och hur skiljer det sig från amerikanska AI-tjänster?

Response (Ollama, v1.0):

Diabase Europa är en AI-assistent med öppna vikter, byggd för svenska och europeiska sammanhang. Den är anpassad för svenska språket och europeiska behov. Diabase Europa är utformad med fokus på integritet, praktisk användbarhet och lokal relevans.

Skillnaden mot många amerikanska AI-tjänster ligger i fokus. Diabase Europa är tänkt att användas i europeiska sammanhang med hänsyn till GDPR, dataresidency och andra europeiska krav...

Other ways to run

llama.cpp

./llama-cli \
  -m gemma-2-9b-it.Q4_K_M.gguf \
  -p "<start_of_turn>user\nHej! Förklara kort vad fine-tuning är.<end_of_turn>\n<start_of_turn>model\n" \
  -n 256 \
  --temp 0.7

For llama.cpp, supply the system prompt manually or via your client. See the SYSTEM block in Modelfile.

Transformers + LoRA (advanced, for fine-tuning)

Use the separate LoRA adapter repo if you want to continue training or serve via PEFT:

from pathlib import Path
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

BASE = "unsloth/gemma-2-9b-it-bnb-4bit"
ADAPTER = "diabase/europa-9b-v1.0-lora"

system = Path("prompts/diabase_europa_v1_system.txt").read_text(encoding="utf-8").strip()

tokenizer = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(
    BASE,
    quantization_config=BitsAndBytesConfig(load_in_4bit=True),
    device_map="auto",
    torch_dtype=torch.bfloat16 if torch.cuda.is_bf16_supported() else torch.float16,
)
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()

messages = [
    {"role": "system", "content": system},
    {"role": "user", "content": "Svara på svenska: Vad innebär digital suveränitet?"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=256, do_sample=False)

print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Training

Diabase Europa v1.0 uses a light SFT recipe directly on Gemma 2 9B Instruct. No continued pre-training. No aggressive full-model fine-tuning.

Parameter	Value
Method	QLoRA SFT (4-bit NF4)
LoRA rank / alpha	16 / 16
Target modules	Attention only (`q`, `k`, `v`, `o` projections)
Learning rate	5e-6
Epochs	1
Effective batch size	8
Max sequence length	2048
Trainable params	~0.35% of base

Training data (~575 synthetic instruction examples)

Category	~Examples	Focus
General AI/ML	150	Concepts, tools, practices
Coding	110	Python, TypeScript, ML infra
Reasoning	85	Diagnostics, analysis
Multilingual	90	Major European languages
Sovereign AI	80	Data sovereignty, compliance
Swedish	60	Dedicated Swedish scenarios

All examples are synthetically generated. No personal data or private information. Human-reviewed curation is planned for v1.1.

Limitations

Early-stage release. Strongest as a Swedish/European assistant polish on Gemma 2, not a from-scratch Swedish foundation model.
Small SFT set. Shapes behavior and tone; does not inject large amounts of new knowledge.
Hallucinations. Can produce plausible but incorrect answers. Verify important facts.
Terminology. Improves with the bundled system prompt; some edge cases remain. DPO and human-reviewed data planned for v1.1.
Safety. Inherits Gemma 2 safety training. No dedicated Diabase red-teaming yet.
Benchmark variance. Small regressions on some EuroEval tasks are within expected noise for light SFT.

Roadmap

Version	Focus
v1.1	Human-reviewed Swedish data, curator workflow, DPO for terminology
v1.2	Controlled targeted SFT (NER, strict QA) without general capability loss
v2	Newer ~12B base model, larger curated Swedish mix

About Diabase

Diabase builds European AI infrastructure for organizations that want open-weight models, local deployment, and transparent evaluation. We document what we train, what we measure, and what we do not claim.

Links

Website: diabase.ai
LoRA adapter: diabase/europa-9b-v1.0-lora

Citation

@misc{diabase_europa_9b_2026,
  author       = {Diabase AI},
  title        = {Diabase Europa 9B v1.0},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/diabase/europa-9b-GGUF}}
}

License

This model is derived from Gemma 2 9B IT and is subject to the Gemma Terms of Use.

Diabase fine-tuning artifacts, documentation, and the bundled system prompt are provided under the Apache 2.0 License where applicable. You must comply with both the Gemma license and applicable law when using, modifying, or redistributing this model.

Downloads last month: 100

GGUF

Model size

9B params

Architecture

gemma2

Hardware compatibility

4-bit

Model tree for Diabase/europa-9b-GGUF

Base model

google/gemma-2-9b

Finetuned

google/gemma-2-9b-it

Quantized

(168)

this model