sharp-pixel's picture
Upload compact v2 adapter
0b0888d verified
|
Raw History Blame Contribute Delete
5.21 kB
---
base_model: mistralai/Ministral-3-8B-Instruct-2512-BF16
base_model_relation: adapter
model_name: opensearch-compact-variables-v2
library_name: peft
pipeline_tag: text-generation
license: apache-2.0
language:
- en
tags:
- lora
- qlora
- sft
- opensearch
- json
---
# OpenSearch compact variables v2
A text-only LoRA adapter for generating compact JSON inputs for a server-side
OpenSearch query template. It is specialized for English shopper requests in a
second-hand luxury catalog. This is an adapter, not a full model or a general
purpose text-to-DSL model. Base weights and the tokenizer are downloaded
separately from [Ministral 3 8B Instruct BF16](https://huggingface.co/mistralai/Ministral-3-8B-Instruct-2512-BF16),
pinned to revision `06cc81bfd6e45321d8fc8f816576c5b6ac67ec22`.
## Input and output
Use the included [system prompt](prompts/system.txt) and
[user prompt template](prompts/user.txt), with the base model's chat template.
The user template accepts the shopper request and trusted application context,
plus the index mapping and query fields as **double-encoded JSON strings**.
[example.py](example.py) constructs this input and runs the adapter with PEFT.
It requires PyTorch, Transformers, PEFT, and a CUDA GPU with enough memory for
the BF16 base model; training used PEFT 0.19.1. For inference, vLLM 0.33 has also
been tested with the BF16 base and this adapter.
The model returns one object with exactly `filters`, `op`, `price`, and `q`.
For example, a dress-watch request with a 20,000 ceiling and no required
category may produce:
```json
{"filters":[["category","watches"]],"op":"or","price":20000,"q":"watch"}
```
The response is **not executable OpenSearch DSL**. The backend validates the
variables and combines them with trusted controls, including required filters,
size, sort, hit tracking, and optional persona scoring, before rendering the
included [v2 template](templates/optimized-query-v2.json.j2), which includes
[the shared template](templates/optimized-query.json.j2). The application must
validate the rendered DSL against its own mapping and policy before execution.
These templates are coupled to the prototype catalog's fields and ranking; they
are not a generic OpenSearch query renderer. The included [render.py](render.py) shows the compact-to-template conversion
and requires Jinja2 and Pydantic 2. Application policy enforcement remains
the caller's responsibility. Prompt instructions
alone do not provide a security boundary: fail closed on malformed output and
independently enforce required filters, price limits, authorization, and
resource limits. Do not accept trusted controls from the shopper or model.
To try inference after downloading this repository, run `python example.py`
from its directory. The example prints the compact output; it does not submit a
search request. To render the printed output, call `render_model_output(completion, context)`
from `render.py`, using the same server-owned context. Set `ADAPTER_ID` if the
adapter is hosted separately from the example. The exact base and tokenizer revisions are pinned in the example and
adapter configuration.
## Training
The adapter was trained with completion-only supervised fine-tuning on 4,640
synthetic examples, with 1,160 grouped held-out examples. Related scenarios were
kept in the same split. The dataset is not included in this package.
| Setting | Value |
| --- | --- |
| Base | Ministral 3 8B Instruct BF16, pinned revision above |
| Method | QLoRA, 4-bit NF4 with BF16 compute |
| Trainable layers | Language-model attention and MLP projections |
| LoRA rank / alpha / dropout | 16 / 32 / 0.05 |
| Epochs / sequence length | 2 / 2,112 tokens |
| Learning rate / effective batch size | 0.0001 / 16 |
| GPU / training time | RTX PRO 6000 Blackwell / 68.4 minutes |
| Training loss | 0.010969 |
## Evaluation and limitations
On 1,160 held-out synthetic examples generated with 4-bit NF4 inference,
1,160 outputs parsed and passed the evaluated DSL policy, required-constraint,
and semantic checks. The rendered request matched the reference exactly in
1,013 cases (87.33%). All 147 differences involved the core multi-match `and` /
`or` operator; these differences can affect retrieval recall. These checks do
not establish correctness for arbitrary inputs or real-world retrieval quality.
A separate BF16 vLLM 0.33 run on 40 stratified held-out examples produced 40
parseable, policy- and semantic-valid requests, with 35 exact matches. This is
a small validation sample, not a production quality or latency guarantee.
Test against your actual catalog, queries, users, and evaluation policy before
using the model. The model may make unsupported inferences or fail to respect
instructions; results can reflect biases in the synthetic examples.
## License and attribution
The adapter and model-card documentation are prepared under Apache 2.0,
following the prior La Trouvaille adapter release; see [LICENSE](LICENSE) and
[NOTICE](NOTICE). The upstream model is also Apache 2.0. The bundled prompts, templates, and renderer originate from the application
and retain their respective source licenses. Training data and other application source are not included.