Instructions to use janusz-bit/brulion-assistant-1.7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use janusz-bit/brulion-assistant-1.7b with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("LiquidAI/LFM3-1.7B") model = PeftModel.from_pretrained(base_model, "janusz-bit/brulion-assistant-1.7b") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use janusz-bit/brulion-assistant-1.7b with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf janusz-bit/brulion-assistant-1.7b:Q4_K_M # Run inference directly in the terminal: llama cli -hf janusz-bit/brulion-assistant-1.7b:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf janusz-bit/brulion-assistant-1.7b:Q4_K_M # Run inference directly in the terminal: llama cli -hf janusz-bit/brulion-assistant-1.7b:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf janusz-bit/brulion-assistant-1.7b:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf janusz-bit/brulion-assistant-1.7b:Q4_K_M
Use Docker
docker model run hf.co/janusz-bit/brulion-assistant-1.7b:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use janusz-bit/brulion-assistant-1.7b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "janusz-bit/brulion-assistant-1.7b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "janusz-bit/brulion-assistant-1.7b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/janusz-bit/brulion-assistant-1.7b:Q4_K_M
- Ollama
How to use janusz-bit/brulion-assistant-1.7b with Ollama:
ollama run hf.co/janusz-bit/brulion-assistant-1.7b:Q4_K_M
- Unsloth Desktop
- Pi
How to use janusz-bit/brulion-assistant-1.7b with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf janusz-bit/brulion-assistant-1.7b:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "janusz-bit/brulion-assistant-1.7b:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use janusz-bit/brulion-assistant-1.7b with Docker Model Runner:
docker model run hf.co/janusz-bit/brulion-assistant-1.7b:Q4_K_M
- Lemonade
How to use janusz-bit/brulion-assistant-1.7b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull janusz-bit/brulion-assistant-1.7b:Q4_K_M
Run and chat with the model
lemonade run user.brulion-assistant-1.7b-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use janusz-bit/brulion-assistant-1.7b with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf janusz-bit/brulion-assistant-1.7b:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default janusz-bit/brulion-assistant-1.7b:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use janusz-bit/brulion-assistant-1.7b with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf janusz-bit/brulion-assistant-1.7b:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "janusz-bit/brulion-assistant-1.7b:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Brulion Assistant 1.7B
Brulion Assistant 1.7B is a LoRA adapter for LiquidAI/LFM3-1.7B,
trained as the on-device assistant of Brulion, an offline-first Markdown notes app:
it searches, queries, reads and changes the user's notebook through the app's tools, in Polish and
English, and answers at once, without thinking. The app runs it over Liquid AI's own 4-bit file,
LFM3-1.7B-QAD-Q4_0.gguf: llama.cpp adds the adapter as it computes, and the 4-bit weights stay as they are.
A modified model. This repository is a Derivative Work of LFM3-1.7B by Liquid AI, Inc., distributed under the LFM Open License v1.0 (
LICENSE). The weights were changed by fine-tuning (NOTICE, "Training" below). It is not made, endorsed or supported by Liquid AI; "LFM" and "Liquid AI" appear here only to describe where the base model comes from. Commercial use is licensed only below US $10 million of annual revenue (below, "License").
Files
| File | Size | SHA-256 | What |
|---|---|---|---|
brulion-assistant-1.7b-lora.gguf |
96.7 MB | abd6fff7106c375c… |
the adapter for llama.cpp (--lora, f16) over LFM3-1.7B-QAD-Q4_0.gguf: what the app runs |
adapter_model.safetensors |
193.3 MB | 4963fd82b7daa62a… |
the same adapter for PEFT (rank 48, alpha 96, bf16), with adapter_config.json |
brulion-assistant-1.7b-BF16.gguf |
3.47 GB | 9a3214064b339cf5… |
the adapter merged into the bf16 weights, one file for other programs |
brulion-assistant-1.7b-Q4_0.gguf |
1.05 GB | 3a3cac5753757115… |
the merged weights quantized to Q4_0 by llama-quantize (not Liquid AI's QAD file) |
brulion-assistant-1.7b-Q4_K_M.gguf |
1.10 GB | 26ee42f6e4bc3640… |
the merged weights quantized to Q4_K_M by llama-quantize |
merged/ |
3.48 GB | (SHA256SUMS) | the adapter merged into the bf16 weights (safetensors), with the base model's config, tokenizer, chat template and code |
LICENSE |
10.6 kB | 4d28ca14dedc0b3d… |
LFM Open License v1.0, the base model's license, unchanged |
NOTICE |
1.89 kB | 4ab7a06c764d9710… |
what was changed, and by whom (license section 4) |
SHA256SUMS lists every file's SHA-256 (sha256sum -c SHA256SUMS).
Use with llama.cpp
In the app the model is two files: Liquid AI's LFM3-1.7B-QAD-Q4_0.gguf of
LiquidAI/LFM3-1.7B-GGUF
(1 049 615 680 bytes, SHA-256 43a7d09a0e9cba91b63eb7052488b9564569bddfd65754d2e426465e572a7452) and brulion-assistant-1.7b-lora.gguf
(96 654 336 bytes, SHA-256 abd6fff7106c375c6f1d28927f3e77c96be36f09211a58585d755e7d5777b452). Every result below was measured so. The
QAD file's weights were trained for 4 bits and lie on the 4-bit grid: merged in and quantized
again, the adapter would mostly round back to them, so it is added as the model computes. It is in
f16: llama.cpp has no fast path for bf16 on ARM.
hf download LiquidAI/LFM3-1.7B-GGUF LFM3-1.7B-QAD-Q4_0.gguf --revision cd0c74c575b5c416f3011b5b4c298077f26ca8c2 --local-dir .
hf download janusz-bit/brulion-assistant-1.7b brulion-assistant-1.7b-lora.gguf --local-dir .
llama-server -m LFM3-1.7B-QAD-Q4_0.gguf --lora brulion-assistant-1.7b-lora.gguf -c 16384 --reasoning off \
--temp 0.1 --top-k 50 --repeat-penalty 1.1 --repeat-last-n 64
--reasoning off makes the chat template open every answer with an empty <think></think>, as
the model was trained. Send the app's system text and tools (below) with /v1/chat/completions;
the tool calls come back as OpenAI tool_calls.
Speed. An adapter costs time where it runs. On a Pixel 9 Pro (Tensor G4; llama.cpp v0.9.0
built with the NDK, its android_armv8.6_1 CPU plugin, 4 threads on the big cores; 965 tokens
read and 128 written, the median of 3 rounds):
| Reading (tokens/s) | Writing (tokens/s) | |
|---|---|---|
LFM3-1.7B-QAD-Q4_0.gguf alone |
166.3 | 24.7 |
| with this adapter (rank 48, f16) | 110.6 (−33 %) | 21.2 (−14 %) |
On x86, counted in processor cycles per token, an earlier rank-48 adapter of this project added 9 % to reading and 15–18 % to writing.
Merged GGUF files
For programs that take a single GGUF file (brulion-assistant-1.7b-BF16.gguf, brulion-assistant-1.7b-Q4_0.gguf, brulion-assistant-1.7b-Q4_K_M.gguf): the adapter merged into the
bf16 weights and converted by llama.cpp's scripts; the 4-bit files are quantized from the BF16 one
by llama-quantize, the usual way, not trained for 4 bits as Liquid AI's QAD file is. The app does
not use them, and they were not measured with the bench: measure them before relying on them. For
an earlier adapter of this project (one trained with thinking, measured with an older bench) the
bench gave 184 of the 245 test tasks over the QAD file, 176 merged into Q4_0 and 191 merged into
Q4_K_M.
llama-server -m brulion-assistant-1.7b-Q4_K_M.gguf -c 16384 --reasoning off \
--temp 0.1 --top-k 50 --repeat-penalty 1.1 --repeat-last-n 64
Use with transformers and PEFT
adapter_config.json and adapter_model.safetensors are the same adapter in PEFT's format, for
further training or for transformers:
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("LiquidAI/LFM3-1.7B", revision="1da06bec6a819da10b54c67e5c90972b46d81871",
dtype=torch.bfloat16, trust_remote_code=True)
model = PeftModel.from_pretrained(base, "janusz-bit/brulion-assistant-1.7b")
tokenizer = AutoTokenizer.from_pretrained("LiquidAI/LFM3-1.7B", revision="1da06bec6a819da10b54c67e5c90972b46d81871")
messages = [{"role": "system", "content": system_text}, # the app's (below, "Prompt format")
{"role": "user", "content": "Today is Wednesday, 2026-11-18.\n\nWhat's on my shopping list?"}]
prompt = tokenizer.apply_chat_template(messages, tools=tools, # the app's twelve tools, as JSON schemas
add_generation_prompt=True, enable_thinking=False, tokenize=False)
The bf16 model with the adapter was not measured; the results below are over the 4-bit file.
merged/ holds the adapter merged into the bf16 weights (safetensors), with the base model's
config, tokenizer, chat template and model code (trust_remote_code=True); not measured either.
Prompt format
LFM3's chat template with tools, thinking off. The app writes the prompt itself
(local::prompt() in src/core/ai/local.cpp, after the model's
chat_template.jinja; a test compares the two), so llama.cpp's chat template with
--reasoning off gives the same tokens. A conversation of the training data — one question, one
search and the answer:
<|startoftext|><|im_start|>system
You are the assistant in the notes app Brulion: you search, read and change the user's notebook with your tools.
Notes are Markdown: items "- ", tasks "- [ ]" (done "- [x]", in progress "- [/]"), …
Rules:
- …
<tools>
{"name": "search_notes", "description": "Full-text search over every note of the notebook: …", "parameters": {"properties": {"limit": {"description": "At most this many matches (default 15, at most 50)", "type": "integer"}, "query": {"description": "Words to look for", "type": "string"}}, "required": ["query"], "type": "object"}}
… one line per tool …
</tools><|im_end|>
<|im_start|>user
Today is Tuesday, 2027-02-09.
Which film did Sophie recommend?<|im_end|>
<|im_start|>assistant
<think></think><tool_call>
<function=search_notes>
<parameter=query>
Sophie
</parameter>
</function>
</tool_call><|im_end|>
<|im_start|>tool
<tool_response>
- [[2027-02-08]], line 1: Sophie recommended the film "Up". (ref journals/2027-02-08.md#L1@eb9a97eb)
</tool_response><|im_end|>
<|im_start|>assistant
<think></think>"Up".<|im_end|>
- System text and tools. The model was trained with the assistant's own system text — English
or Polish, the language of the app and the notebook — and its twelve tools:
search_notes,query_notes,list_pages,read_page,append_to_journal,append_to_page,write_page,update_block,look_at_picture,extension_reference,check_extension,write_extension. Both are in src/core/extensions/builtin/asystent.luau (WRITTEN,TOOLS); another prompt or other tools are untested. - The first message starts with the day (
Today is Wednesday, 2026-11-18.,Dziś jest środa, 2026-11-18.) and, when the chat is opened from a page, that page's text (The conversation is about the page [[Title]]: … ---). - Tool results come one
toolturn each, in the order of the calls; a failed one starts withError:. Every turn of the assistant starts with an empty<think></think>. - The loop. At most 16 turns per question; a reading call repeated with the same input is not run again, nor a change already made (the result says so); from the eighth turn on, the last result says which turn of 16 it is. Every change to a note is shown to the user, who approves or declines it.
- Sampling, as the app does: temperature 0.1, top-k 50, repetition penalty 1.1 over the last 64 tokens, a fixed seed; 16 384 tokens of context, at most 2 048 tokens per answer.
Intended use
- The Brulion app's assistant on the device (Android, Linux) over the user's own notebook, every change approved by the user.
- A small on-device model for research on tool use, or a starting point for training another app's assistant on tasks of its own.
Out of scope: a general chat assistant; facts from the model's memory (1.7 B parameters); other prompts, tools or languages than the ones above without measuring them first; thinking mode (not trained); unattended changes to notes; anything where a wrong answer harms someone (health, money, safety).
Limitations
- Measured on made-up notebooks of 8 to 18 pages each; real notebooks are larger and less tidy.
- The weakest kinds of task on the generated test set (a run's mean of 4):
multi(14 of 18),no_tool(15.75 of 18),journal(16.5 of 18). - Of the 23 tasks it failed in all four runs (11 of the 245 generated, 12 of the 88 hand-written), 10 need arithmetic or counting over what it read (a week's spending, hours at a rate, a recipe for twelve, a budget's rest), 5 a day worked out from words, all of them Polish ("pojutrze", "w przyszły wtorek", "wczoraj", "na piątek", "w przyszłym tygodniu"), 4 a fact asked for in other words than the note's ("kosze" for the bin collection, "numer indeksu" for the note's "numer albumu"), 4 something else (a query for tomorrow's reminders written wrong, a note to compose written into the journal, the wrong one of a recipe's ingredients, an invoice's total given for a rate).
- It is a 1.7 B model: it can still state a wrong fact. Its answers link the pages they draw on
(
[[Page]]) — check them. - Trained on notebooks of a generator (home, car, contacts, shopping, recipes, a reading list, projects, journal days): other kinds of notes are less well covered.
Training
Two stages, as Liquid AI fine-tunes LFM3 for an app's tools — supervised fine-tuning on the solutions' steps without thinking, then GRPO with a reward for the right call or answer — and the two adapters combined into one (finetune/, docs/FINETUNE.md):
- Supervised fine-tuning (
finetune/sft.py) of the base model's bf16 weights: LoRA rank 16, alpha 32, dropout 0.05 — the adapter of step 316; 3 h 51 min on one NVIDIA GeForce RTX 5060 Laptop GPU (8 GB).- Data: the dataset's
training/taught-training.jsonl,training/guided-training.jsonl,training/taught2-training.jsonl,training/corrected1-training.jsonl,training/corrected2-training.jsonl. Of each task's passed attempts the best 2 (no failed tool call first, then the fewest tokens written): 5 056 sequences to learn from, 281 more, of the tasks held out, for a validation loss. - Learning: the loss on the model's turns only, each sequence's mean weighing the same, a short answer as much as a long one; AdamW, learning rate 0.0002, warm-up 3 %, cosine; 16 sequences a step, one epoch. Validation loss 0.993 → 0.144.
- Data: the dataset's
- GRPO (
finetune/grpo.py) over the base model with the adapter above merged into its weights: a second LoRA, rank 32, alpha 64 — the adapter of step 75 of 100; 3 h 12 min on one NVIDIA GeForce RTX 5060 Laptop GPU (8 GB).- Prompts: the steps of the
grposplit's solutions (the dataset'straining/taught3-training.jsonl: each answer of a sequence, with everything before it), less those whose own solution the reward does not take. Only the 1 803 steps the model it starts from does sometimes are drawn (it answered every step 8 times before: 3 021 steps scored 1 every time, 36 never), in proportion to p(1 − p), p the share of its answers that scored 1; a step that answers at once (no tool needed, or asking back) twice as often. For each prompt 8 answers at temperature 1.0, at most 256 tokens. - The reward of a step that calls tools: 1.0 for the solution's calls — the tool and each argument compared by its kind: a page's name, a query's filters in any order, a ref or a status exactly, the words a change writes with some tolerance but its dates with their markers (📅 due, ⏰ reminder…), times, task boxes and numbers as the solution's — or for other calls the task names as right at that step (a retry); else 0.2 × the F1 of the tools' names.
- The reward of an answer: 1.0 when it calls no tool, passes the task's checks of the answer, states no date, time or number of three digits or more that its prompt does not show (unless worked out with the working shown, or, in a task that works its answer out, a value its check asks for) and says nothing untrue of the changes (a change claimed when none was made, the run's own change said to have been so already); else 0.
- Learning: each answer's advantage is its reward against its group's mean, over the group's standard deviation; a group whose answers all score the same is skipped, and a step draws prompts (at most 8) until 4 groups tell answers apart. A KL penalty of 0.04 towards the model it starts from; AdamW, learning rate 1e-5, 5 steps of warm-up.
- Prompts: the steps of the
- One adapter: the two summed as one LoRA of rank 48, alpha 96, on
in_proj,k_proj,o_proj,out_proj,q_proj,v_proj,w1,w2,w3(finetune/combine.py: their A stacked, their B side by side, all at the scale alpha / r = 2) — the same sum of weights the two adapters make.
The data are the dataset's sft and grpo splits — made-up notebooks and tasks written by
finetune/generate.py, each task with a solution — and the training sequences the bench made of
them (the dataset's training/ files), in the tokens of the prompts the app writes: the solutions'
steps written as the model's turns, the solutions' calls with the base model's own answers, and an
earlier fine-tune's first steps (read, not learnt) with the solution going on from there. The
hand-written tasks (hand) and the generated test split were never trained on.
Evaluation
brulion-assistantbench (tests/assistantbench) runs the
app's real assistant — its Luau extension, system text, tools and the notebook index — on a copy
of each task's notebook, the model answering every request as the app's runner does, and checks
the outcome: the lines that must be in the notes afterwards, no change nobody asked for, the facts
of the answer, the tools called. A task passes only when every check holds.
Measured 2026-11-18 with the bench's llama-server runner (llama.cpp on a GPU), every model over LFM3-1.7B-QAD-Q4_0.gguf (the fine-tuned ones with their adapter added by --lora): the app's prompt and sampling (temperature 0.1, top-k 50, repetition penalty 1.1 over 64 tokens), 16 384 tokens of context, at most 2 048 tokens per answer. Brulion Assistant 1.7B and SFT stage alone (rank 16) ran every task four times (seeds 2986414103–2986414106: the app's and the next three); the others once (seed 2986414103). Where a model ran several times, its cells are a run's mean. Requests, failed tool calls and tokens written are of the test set.
| Model | Runs | 88 hand-written | 245 generated (test) | Polish (test, 124) | English (test, 121) | Requests per task | Failed tool calls per run | Tokens written per task |
|---|---|---|---|---|---|---|---|---|
| Brulion Assistant 1.7B | 4 | 74.25 (84.4 %) | 232.25 (94.8 %) | 113.75 | 118.5 | 2.1 | 0 | 57 |
| SFT stage alone (rank 16) | 4 | 74.5 (84.7 %) | 228.5 (93.3 %) | 110.75 | 117.75 | 2.3 | 2 | 67 |
| LFM3-1.7B, thinking | 1 | 74 (84.1 %) | 218 (89.0 %) | 103 | 115 | 4.3 | 19 | 1310 |
| LFM3-1.7B, no thinking | 1 | 54 (61.4 %) | 174 (71.0 %) | 80 | 94 | 4.9 | 108 | 187 |
Brulion Assistant 1.7B, its 4 runs: 232, 233, 231, 233 of the 245 generated; 75, 74, 76, 72 of the 88 hand-written. SFT stage alone (rank 16), its 4 runs: 227, 228, 229, 230 of the 245 generated; 75, 75, 75, 73 of the 88 hand-written. Runs of the same model differ by up to four of the hand-written tasks (72 to 76 in four runs), so the two fine-tuned models ran four times. Run for run (the same seeds), GRPO's stage did 5, 5, 2 and 3 more of the generated tasks than the SFT stage alone (3.75 on average; 12 of the 15 Polish ones) and none more of the hand-written.
By category, the generated test set (a run's mean):
| Category (tasks) | Brulion Assistant 1.7B | SFT stage alone (rank 16) | LFM3-1.7B, thinking | LFM3-1.7B, no thinking |
|---|---|---|---|---|
search (64) |
61 | 60.75 | 56 | 57 |
query (24) |
24 | 24 | 21 | 17 |
read (23) |
23 | 21.25 | 20 | 19 |
append (28) |
27 | 27 | 25 | 24 |
journal (18) |
16.5 | 17 | 16 | 13 |
update (34) |
33 | 31.5 | 30 | 19 |
write (6) |
6 | 6 | 6 | 6 |
multi (18) |
14 | 15 | 18 | 9 |
no_tool (18) |
15.75 | 14 | 15 | 4 |
clarify (6) |
6 | 6 | 6 | 2 |
decline (6) |
6 | 6 | 5 | 4 |
License
LFM Open License v1.0, the base model's license; this repository adds no terms of its own. In short — the license itself is what counts:
- You may use, change and redistribute the model. Commercial use is licensed only as long as you, together with the entities that control you, that you control or that are under common control with you, have less than US $10 million of annual revenue; above that it is not licensed. The limit does not apply to a qualified non-profit's non-commercial or research use.
- Redistributing it or a model made from it: include
LICENSE, keep the notices (NOTICE), and mark the files you change. - No trademark rights: do not name your model or product after LFM or Liquid AI beyond describing where it comes from.
- No warranty; the license ends for anyone who breaks its terms.
The app, the bench and the training code are AGPL-3.0 (the Brulion repository); the
tasks are the dataset janusz-bit/brulion-assistant-tasks
(CC BY 4.0). The model card describes commit 226302bd94.
- Downloads last month
- 254