Brulion Assistant 1.7B

Brulion Assistant 1.7B is a LoRA adapter for LiquidAI/LFM3-1.7B, trained as the on-device assistant of Brulion, an offline-first Markdown notes app: it searches, queries, reads and changes the user's notebook through the app's tools, in Polish and English, and answers at once, without thinking. The app runs it over Liquid AI's own 4-bit file, LFM3-1.7B-QAD-Q4_0.gguf: llama.cpp adds the adapter as it computes, and the 4-bit weights stay as they are.

A modified model. This repository is a Derivative Work of LFM3-1.7B by Liquid AI, Inc., distributed under the LFM Open License v1.0 (LICENSE). The weights were changed by fine-tuning (NOTICE, "Training" below). It is not made, endorsed or supported by Liquid AI; "LFM" and "Liquid AI" appear here only to describe where the base model comes from. Commercial use is licensed only below US $10 million of annual revenue (below, "License").

Files

File Size SHA-256 What
brulion-assistant-1.7b-lora.gguf 96.7 MB abd6fff7106c375c… the adapter for llama.cpp (--lora, f16) over LFM3-1.7B-QAD-Q4_0.gguf: what the app runs
adapter_model.safetensors 193.3 MB 4963fd82b7daa62a… the same adapter for PEFT (rank 48, alpha 96, bf16), with adapter_config.json
brulion-assistant-1.7b-BF16.gguf 3.47 GB 9a3214064b339cf5… the adapter merged into the bf16 weights, one file for other programs
brulion-assistant-1.7b-Q4_0.gguf 1.05 GB 3a3cac5753757115… the merged weights quantized to Q4_0 by llama-quantize (not Liquid AI's QAD file)
brulion-assistant-1.7b-Q4_K_M.gguf 1.10 GB 26ee42f6e4bc3640… the merged weights quantized to Q4_K_M by llama-quantize
merged/ 3.48 GB (SHA256SUMS) the adapter merged into the bf16 weights (safetensors), with the base model's config, tokenizer, chat template and code
LICENSE 10.6 kB 4d28ca14dedc0b3d… LFM Open License v1.0, the base model's license, unchanged
NOTICE 1.89 kB 4ab7a06c764d9710… what was changed, and by whom (license section 4)

SHA256SUMS lists every file's SHA-256 (sha256sum -c SHA256SUMS).

Use with llama.cpp

In the app the model is two files: Liquid AI's LFM3-1.7B-QAD-Q4_0.gguf of LiquidAI/LFM3-1.7B-GGUF (1 049 615 680 bytes, SHA-256 43a7d09a0e9cba91b63eb7052488b9564569bddfd65754d2e426465e572a7452) and brulion-assistant-1.7b-lora.gguf (96 654 336 bytes, SHA-256 abd6fff7106c375c6f1d28927f3e77c96be36f09211a58585d755e7d5777b452). Every result below was measured so. The QAD file's weights were trained for 4 bits and lie on the 4-bit grid: merged in and quantized again, the adapter would mostly round back to them, so it is added as the model computes. It is in f16: llama.cpp has no fast path for bf16 on ARM.

hf download LiquidAI/LFM3-1.7B-GGUF LFM3-1.7B-QAD-Q4_0.gguf --revision cd0c74c575b5c416f3011b5b4c298077f26ca8c2 --local-dir .
hf download janusz-bit/brulion-assistant-1.7b brulion-assistant-1.7b-lora.gguf --local-dir .
llama-server -m LFM3-1.7B-QAD-Q4_0.gguf --lora brulion-assistant-1.7b-lora.gguf -c 16384 --reasoning off \
    --temp 0.1 --top-k 50 --repeat-penalty 1.1 --repeat-last-n 64

--reasoning off makes the chat template open every answer with an empty <think></think>, as the model was trained. Send the app's system text and tools (below) with /v1/chat/completions; the tool calls come back as OpenAI tool_calls.

Speed. An adapter costs time where it runs. On a Pixel 9 Pro (Tensor G4; llama.cpp v0.9.0 built with the NDK, its android_armv8.6_1 CPU plugin, 4 threads on the big cores; 965 tokens read and 128 written, the median of 3 rounds):

Reading (tokens/s) Writing (tokens/s)
LFM3-1.7B-QAD-Q4_0.gguf alone 166.3 24.7
with this adapter (rank 48, f16) 110.6 (−33 %) 21.2 (−14 %)

On x86, counted in processor cycles per token, an earlier rank-48 adapter of this project added 9 % to reading and 15–18 % to writing.

Merged GGUF files

For programs that take a single GGUF file (brulion-assistant-1.7b-BF16.gguf, brulion-assistant-1.7b-Q4_0.gguf, brulion-assistant-1.7b-Q4_K_M.gguf): the adapter merged into the bf16 weights and converted by llama.cpp's scripts; the 4-bit files are quantized from the BF16 one by llama-quantize, the usual way, not trained for 4 bits as Liquid AI's QAD file is. The app does not use them, and they were not measured with the bench: measure them before relying on them. For an earlier adapter of this project (one trained with thinking, measured with an older bench) the bench gave 184 of the 245 test tasks over the QAD file, 176 merged into Q4_0 and 191 merged into Q4_K_M.

llama-server -m brulion-assistant-1.7b-Q4_K_M.gguf -c 16384 --reasoning off \
    --temp 0.1 --top-k 50 --repeat-penalty 1.1 --repeat-last-n 64

Use with transformers and PEFT

adapter_config.json and adapter_model.safetensors are the same adapter in PEFT's format, for further training or for transformers:

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("LiquidAI/LFM3-1.7B", revision="1da06bec6a819da10b54c67e5c90972b46d81871",
                                            dtype=torch.bfloat16, trust_remote_code=True)
model = PeftModel.from_pretrained(base, "janusz-bit/brulion-assistant-1.7b")
tokenizer = AutoTokenizer.from_pretrained("LiquidAI/LFM3-1.7B", revision="1da06bec6a819da10b54c67e5c90972b46d81871")
messages = [{"role": "system", "content": system_text},  # the app's (below, "Prompt format")
            {"role": "user", "content": "Today is Wednesday, 2026-11-18.\n\nWhat's on my shopping list?"}]
prompt = tokenizer.apply_chat_template(messages, tools=tools,  # the app's twelve tools, as JSON schemas
                                       add_generation_prompt=True, enable_thinking=False, tokenize=False)

The bf16 model with the adapter was not measured; the results below are over the 4-bit file.

merged/ holds the adapter merged into the bf16 weights (safetensors), with the base model's config, tokenizer, chat template and model code (trust_remote_code=True); not measured either.

Prompt format

LFM3's chat template with tools, thinking off. The app writes the prompt itself (local::prompt() in src/core/ai/local.cpp, after the model's chat_template.jinja; a test compares the two), so llama.cpp's chat template with --reasoning off gives the same tokens. A conversation of the training data — one question, one search and the answer:

<|startoftext|><|im_start|>system
You are the assistant in the notes app Brulion: you search, read and change the user's notebook with your tools.

Notes are Markdown: items "- ", tasks "- [ ]" (done "- [x]", in progress "- [/]"), …

Rules:
- …

<tools>
{"name": "search_notes", "description": "Full-text search over every note of the notebook: …", "parameters": {"properties": {"limit": {"description": "At most this many matches (default 15, at most 50)", "type": "integer"}, "query": {"description": "Words to look for", "type": "string"}}, "required": ["query"], "type": "object"}}
… one line per tool …
</tools><|im_end|>
<|im_start|>user
Today is Tuesday, 2027-02-09.

Which film did Sophie recommend?<|im_end|>
<|im_start|>assistant
<think></think><tool_call>
<function=search_notes>
<parameter=query>
Sophie
</parameter>
</function>
</tool_call><|im_end|>
<|im_start|>tool
<tool_response>
- [[2027-02-08]], line 1: Sophie recommended the film "Up". (ref journals/2027-02-08.md#L1@eb9a97eb)
</tool_response><|im_end|>
<|im_start|>assistant
<think></think>"Up".<|im_end|>
  • System text and tools. The model was trained with the assistant's own system text — English or Polish, the language of the app and the notebook — and its twelve tools: search_notes, query_notes, list_pages, read_page, append_to_journal, append_to_page, write_page, update_block, look_at_picture, extension_reference, check_extension, write_extension. Both are in src/core/extensions/builtin/asystent.luau (WRITTEN, TOOLS); another prompt or other tools are untested.
  • The first message starts with the day (Today is Wednesday, 2026-11-18., Dziś jest środa, 2026-11-18.) and, when the chat is opened from a page, that page's text (The conversation is about the page [[Title]]: … ---).
  • Tool results come one tool turn each, in the order of the calls; a failed one starts with Error: . Every turn of the assistant starts with an empty <think></think>.
  • The loop. At most 16 turns per question; a reading call repeated with the same input is not run again, nor a change already made (the result says so); from the eighth turn on, the last result says which turn of 16 it is. Every change to a note is shown to the user, who approves or declines it.
  • Sampling, as the app does: temperature 0.1, top-k 50, repetition penalty 1.1 over the last 64 tokens, a fixed seed; 16 384 tokens of context, at most 2 048 tokens per answer.

Intended use

  • The Brulion app's assistant on the device (Android, Linux) over the user's own notebook, every change approved by the user.
  • A small on-device model for research on tool use, or a starting point for training another app's assistant on tasks of its own.

Out of scope: a general chat assistant; facts from the model's memory (1.7 B parameters); other prompts, tools or languages than the ones above without measuring them first; thinking mode (not trained); unattended changes to notes; anything where a wrong answer harms someone (health, money, safety).

Limitations

  • Measured on made-up notebooks of 8 to 18 pages each; real notebooks are larger and less tidy.
  • The weakest kinds of task on the generated test set (a run's mean of 4): multi (14 of 18), no_tool (15.75 of 18), journal (16.5 of 18).
  • Of the 23 tasks it failed in all four runs (11 of the 245 generated, 12 of the 88 hand-written), 10 need arithmetic or counting over what it read (a week's spending, hours at a rate, a recipe for twelve, a budget's rest), 5 a day worked out from words, all of them Polish ("pojutrze", "w przyszły wtorek", "wczoraj", "na piątek", "w przyszłym tygodniu"), 4 a fact asked for in other words than the note's ("kosze" for the bin collection, "numer indeksu" for the note's "numer albumu"), 4 something else (a query for tomorrow's reminders written wrong, a note to compose written into the journal, the wrong one of a recipe's ingredients, an invoice's total given for a rate).
  • It is a 1.7 B model: it can still state a wrong fact. Its answers link the pages they draw on ([[Page]]) — check them.
  • Trained on notebooks of a generator (home, car, contacts, shopping, recipes, a reading list, projects, journal days): other kinds of notes are less well covered.

Training

Two stages, as Liquid AI fine-tunes LFM3 for an app's tools — supervised fine-tuning on the solutions' steps without thinking, then GRPO with a reward for the right call or answer — and the two adapters combined into one (finetune/, docs/FINETUNE.md):

  1. Supervised fine-tuning (finetune/sft.py) of the base model's bf16 weights: LoRA rank 16, alpha 32, dropout 0.05 — the adapter of step 316; 3 h 51 min on one NVIDIA GeForce RTX 5060 Laptop GPU (8 GB).
    • Data: the dataset's training/taught-training.jsonl, training/guided-training.jsonl, training/taught2-training.jsonl, training/corrected1-training.jsonl, training/corrected2-training.jsonl. Of each task's passed attempts the best 2 (no failed tool call first, then the fewest tokens written): 5 056 sequences to learn from, 281 more, of the tasks held out, for a validation loss.
    • Learning: the loss on the model's turns only, each sequence's mean weighing the same, a short answer as much as a long one; AdamW, learning rate 0.0002, warm-up 3 %, cosine; 16 sequences a step, one epoch. Validation loss 0.993 → 0.144.
  2. GRPO (finetune/grpo.py) over the base model with the adapter above merged into its weights: a second LoRA, rank 32, alpha 64 — the adapter of step 75 of 100; 3 h 12 min on one NVIDIA GeForce RTX 5060 Laptop GPU (8 GB).
    • Prompts: the steps of the grpo split's solutions (the dataset's training/taught3-training.jsonl: each answer of a sequence, with everything before it), less those whose own solution the reward does not take. Only the 1 803 steps the model it starts from does sometimes are drawn (it answered every step 8 times before: 3 021 steps scored 1 every time, 36 never), in proportion to p(1 − p), p the share of its answers that scored 1; a step that answers at once (no tool needed, or asking back) twice as often. For each prompt 8 answers at temperature 1.0, at most 256 tokens.
    • The reward of a step that calls tools: 1.0 for the solution's calls — the tool and each argument compared by its kind: a page's name, a query's filters in any order, a ref or a status exactly, the words a change writes with some tolerance but its dates with their markers (📅 due, ⏰ reminder…), times, task boxes and numbers as the solution's — or for other calls the task names as right at that step (a retry); else 0.2 × the F1 of the tools' names.
    • The reward of an answer: 1.0 when it calls no tool, passes the task's checks of the answer, states no date, time or number of three digits or more that its prompt does not show (unless worked out with the working shown, or, in a task that works its answer out, a value its check asks for) and says nothing untrue of the changes (a change claimed when none was made, the run's own change said to have been so already); else 0.
    • Learning: each answer's advantage is its reward against its group's mean, over the group's standard deviation; a group whose answers all score the same is skipped, and a step draws prompts (at most 8) until 4 groups tell answers apart. A KL penalty of 0.04 towards the model it starts from; AdamW, learning rate 1e-5, 5 steps of warm-up.
  3. One adapter: the two summed as one LoRA of rank 48, alpha 96, on in_proj, k_proj, o_proj, out_proj, q_proj, v_proj, w1, w2, w3 (finetune/combine.py: their A stacked, their B side by side, all at the scale alpha / r = 2) — the same sum of weights the two adapters make.

The data are the dataset's sft and grpo splits — made-up notebooks and tasks written by finetune/generate.py, each task with a solution — and the training sequences the bench made of them (the dataset's training/ files), in the tokens of the prompts the app writes: the solutions' steps written as the model's turns, the solutions' calls with the base model's own answers, and an earlier fine-tune's first steps (read, not learnt) with the solution going on from there. The hand-written tasks (hand) and the generated test split were never trained on.

Evaluation

brulion-assistantbench (tests/assistantbench) runs the app's real assistant — its Luau extension, system text, tools and the notebook index — on a copy of each task's notebook, the model answering every request as the app's runner does, and checks the outcome: the lines that must be in the notes afterwards, no change nobody asked for, the facts of the answer, the tools called. A task passes only when every check holds.

Measured 2026-11-18 with the bench's llama-server runner (llama.cpp on a GPU), every model over LFM3-1.7B-QAD-Q4_0.gguf (the fine-tuned ones with their adapter added by --lora): the app's prompt and sampling (temperature 0.1, top-k 50, repetition penalty 1.1 over 64 tokens), 16 384 tokens of context, at most 2 048 tokens per answer. Brulion Assistant 1.7B and SFT stage alone (rank 16) ran every task four times (seeds 2986414103–2986414106: the app's and the next three); the others once (seed 2986414103). Where a model ran several times, its cells are a run's mean. Requests, failed tool calls and tokens written are of the test set.

Model Runs 88 hand-written 245 generated (test) Polish (test, 124) English (test, 121) Requests per task Failed tool calls per run Tokens written per task
Brulion Assistant 1.7B 4 74.25 (84.4 %) 232.25 (94.8 %) 113.75 118.5 2.1 0 57
SFT stage alone (rank 16) 4 74.5 (84.7 %) 228.5 (93.3 %) 110.75 117.75 2.3 2 67
LFM3-1.7B, thinking 1 74 (84.1 %) 218 (89.0 %) 103 115 4.3 19 1310
LFM3-1.7B, no thinking 1 54 (61.4 %) 174 (71.0 %) 80 94 4.9 108 187

Brulion Assistant 1.7B, its 4 runs: 232, 233, 231, 233 of the 245 generated; 75, 74, 76, 72 of the 88 hand-written. SFT stage alone (rank 16), its 4 runs: 227, 228, 229, 230 of the 245 generated; 75, 75, 75, 73 of the 88 hand-written. Runs of the same model differ by up to four of the hand-written tasks (72 to 76 in four runs), so the two fine-tuned models ran four times. Run for run (the same seeds), GRPO's stage did 5, 5, 2 and 3 more of the generated tasks than the SFT stage alone (3.75 on average; 12 of the 15 Polish ones) and none more of the hand-written.

By category, the generated test set (a run's mean):

Category (tasks) Brulion Assistant 1.7B SFT stage alone (rank 16) LFM3-1.7B, thinking LFM3-1.7B, no thinking
search (64) 61 60.75 56 57
query (24) 24 24 21 17
read (23) 23 21.25 20 19
append (28) 27 27 25 24
journal (18) 16.5 17 16 13
update (34) 33 31.5 30 19
write (6) 6 6 6 6
multi (18) 14 15 18 9
no_tool (18) 15.75 14 15 4
clarify (6) 6 6 6 2
decline (6) 6 6 5 4

License

LFM Open License v1.0, the base model's license; this repository adds no terms of its own. In short — the license itself is what counts:

  • You may use, change and redistribute the model. Commercial use is licensed only as long as you, together with the entities that control you, that you control or that are under common control with you, have less than US $10 million of annual revenue; above that it is not licensed. The limit does not apply to a qualified non-profit's non-commercial or research use.
  • Redistributing it or a model made from it: include LICENSE, keep the notices (NOTICE), and mark the files you change.
  • No trademark rights: do not name your model or product after LFM or Liquid AI beyond describing where it comes from.
  • No warranty; the license ends for anyone who breaks its terms.

The app, the bench and the training code are AGPL-3.0 (the Brulion repository); the tasks are the dataset janusz-bit/brulion-assistant-tasks (CC BY 4.0). The model card describes commit 226302bd94.

Downloads last month
254
GGUF
Model size
2B params
Architecture
lfm3
Hardware compatibility
Log In to add your hardware

4-bit

16-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for janusz-bit/brulion-assistant-1.7b

Finetuned
(89)
this model

Dataset used to train janusz-bit/brulion-assistant-1.7b