Instructions to use sharp-pixel/opensearch-compact-variables-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use sharp-pixel/opensearch-compact-variables-v2 with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("mistralai/Ministral-3-8B-Instruct-2512-BF16") model = PeftModel.from_pretrained(base_model, "sharp-pixel/opensearch-compact-variables-v2") - Notebooks
- Google Colab
- Kaggle
OpenSearch compact variables v2
A text-only LoRA adapter for generating compact JSON inputs for a server-side
OpenSearch query template. It is specialized for English shopper requests in a
second-hand luxury catalog. This is an adapter, not a full model or a general
purpose text-to-DSL model. Base weights and the tokenizer are downloaded
separately from Ministral 3 8B Instruct BF16,
pinned to revision 06cc81bfd6e45321d8fc8f816576c5b6ac67ec22.
Input and output
Use the included system prompt and user prompt template, with the base model's chat template. The user template accepts the shopper request and trusted application context, plus the index mapping and query fields as double-encoded JSON strings. example.py constructs this input and runs the adapter with PEFT. It requires PyTorch, Transformers, PEFT, and a CUDA GPU with enough memory for the BF16 base model; training used PEFT 0.19.1. For inference, vLLM 0.33 has also been tested with the BF16 base and this adapter.
The model returns one object with exactly filters, op, price, and q.
For example, a dress-watch request with a 20,000 ceiling and no required
category may produce:
{"filters":[["category","watches"]],"op":"or","price":20000,"q":"watch"}
The response is not executable OpenSearch DSL. The backend validates the variables and combines them with trusted controls, including required filters, size, sort, hit tracking, and optional persona scoring, before rendering the included v2 template, which includes the shared template. The application must validate the rendered DSL against its own mapping and policy before execution. These templates are coupled to the prototype catalog's fields and ranking; they are not a generic OpenSearch query renderer. The included render.py shows the compact-to-template conversion and requires Jinja2 and Pydantic 2. Application policy enforcement remains the caller's responsibility. Prompt instructions alone do not provide a security boundary: fail closed on malformed output and independently enforce required filters, price limits, authorization, and resource limits. Do not accept trusted controls from the shopper or model.
To try inference after downloading this repository, run python example.py
from its directory. The example prints the compact output; it does not submit a
search request. To render the printed output, call render_model_output(completion, context)
from render.py, using the same server-owned context. Set ADAPTER_ID if the
adapter is hosted separately from the example. The exact base and tokenizer revisions are pinned in the example and
adapter configuration.
Training
The adapter was trained with completion-only supervised fine-tuning on 4,640 synthetic examples, with 1,160 grouped held-out examples. Related scenarios were kept in the same split. The dataset is not included in this package.
| Setting | Value |
|---|---|
| Base | Ministral 3 8B Instruct BF16, pinned revision above |
| Method | QLoRA, 4-bit NF4 with BF16 compute |
| Trainable layers | Language-model attention and MLP projections |
| LoRA rank / alpha / dropout | 16 / 32 / 0.05 |
| Epochs / sequence length | 2 / 2,112 tokens |
| Learning rate / effective batch size | 0.0001 / 16 |
| GPU / training time | RTX PRO 6000 Blackwell / 68.4 minutes |
| Training loss | 0.010969 |
Evaluation and limitations
On 1,160 held-out synthetic examples generated with 4-bit NF4 inference,
1,160 outputs parsed and passed the evaluated DSL policy, required-constraint,
and semantic checks. The rendered request matched the reference exactly in
1,013 cases (87.33%). All 147 differences involved the core multi-match and /
or operator; these differences can affect retrieval recall. These checks do
not establish correctness for arbitrary inputs or real-world retrieval quality.
A separate BF16 vLLM 0.33 run on 40 stratified held-out examples produced 40 parseable, policy- and semantic-valid requests, with 35 exact matches. This is a small validation sample, not a production quality or latency guarantee. Test against your actual catalog, queries, users, and evaluation policy before using the model. The model may make unsupported inferences or fail to respect instructions; results can reflect biases in the synthetic examples.
License and attribution
The adapter and model-card documentation are prepared under Apache 2.0, following the prior La Trouvaille adapter release; see LICENSE and NOTICE. The upstream model is also Apache 2.0. The bundled prompts, templates, and renderer originate from the application and retain their respective source licenses. Training data and other application source are not included.
- Downloads last month
- 17
Model tree for sharp-pixel/opensearch-compact-variables-v2
Base model
mistralai/Ministral-3-8B-Base-2512