sharp-pixel's picture
Upload compact v2 adapter
0b0888d verified
|
Raw History Blame Contribute Delete
5.21 kB
metadata
base_model: mistralai/Ministral-3-8B-Instruct-2512-BF16
base_model_relation: adapter
model_name: opensearch-compact-variables-v2
library_name: peft
pipeline_tag: text-generation
license: apache-2.0
language:
  - en
tags:
  - lora
  - qlora
  - sft
  - opensearch
  - json

OpenSearch compact variables v2

A text-only LoRA adapter for generating compact JSON inputs for a server-side OpenSearch query template. It is specialized for English shopper requests in a second-hand luxury catalog. This is an adapter, not a full model or a general purpose text-to-DSL model. Base weights and the tokenizer are downloaded separately from Ministral 3 8B Instruct BF16, pinned to revision 06cc81bfd6e45321d8fc8f816576c5b6ac67ec22.

Input and output

Use the included system prompt and user prompt template, with the base model's chat template. The user template accepts the shopper request and trusted application context, plus the index mapping and query fields as double-encoded JSON strings. example.py constructs this input and runs the adapter with PEFT. It requires PyTorch, Transformers, PEFT, and a CUDA GPU with enough memory for the BF16 base model; training used PEFT 0.19.1. For inference, vLLM 0.33 has also been tested with the BF16 base and this adapter.

The model returns one object with exactly filters, op, price, and q. For example, a dress-watch request with a 20,000 ceiling and no required category may produce:

{"filters":[["category","watches"]],"op":"or","price":20000,"q":"watch"}

The response is not executable OpenSearch DSL. The backend validates the variables and combines them with trusted controls, including required filters, size, sort, hit tracking, and optional persona scoring, before rendering the included v2 template, which includes the shared template. The application must validate the rendered DSL against its own mapping and policy before execution. These templates are coupled to the prototype catalog's fields and ranking; they are not a generic OpenSearch query renderer. The included render.py shows the compact-to-template conversion and requires Jinja2 and Pydantic 2. Application policy enforcement remains the caller's responsibility. Prompt instructions alone do not provide a security boundary: fail closed on malformed output and independently enforce required filters, price limits, authorization, and resource limits. Do not accept trusted controls from the shopper or model.

To try inference after downloading this repository, run python example.py from its directory. The example prints the compact output; it does not submit a search request. To render the printed output, call render_model_output(completion, context) from render.py, using the same server-owned context. Set ADAPTER_ID if the adapter is hosted separately from the example. The exact base and tokenizer revisions are pinned in the example and adapter configuration.

Training

The adapter was trained with completion-only supervised fine-tuning on 4,640 synthetic examples, with 1,160 grouped held-out examples. Related scenarios were kept in the same split. The dataset is not included in this package.

Setting Value
Base Ministral 3 8B Instruct BF16, pinned revision above
Method QLoRA, 4-bit NF4 with BF16 compute
Trainable layers Language-model attention and MLP projections
LoRA rank / alpha / dropout 16 / 32 / 0.05
Epochs / sequence length 2 / 2,112 tokens
Learning rate / effective batch size 0.0001 / 16
GPU / training time RTX PRO 6000 Blackwell / 68.4 minutes
Training loss 0.010969

Evaluation and limitations

On 1,160 held-out synthetic examples generated with 4-bit NF4 inference, 1,160 outputs parsed and passed the evaluated DSL policy, required-constraint, and semantic checks. The rendered request matched the reference exactly in 1,013 cases (87.33%). All 147 differences involved the core multi-match and / or operator; these differences can affect retrieval recall. These checks do not establish correctness for arbitrary inputs or real-world retrieval quality.

A separate BF16 vLLM 0.33 run on 40 stratified held-out examples produced 40 parseable, policy- and semantic-valid requests, with 35 exact matches. This is a small validation sample, not a production quality or latency guarantee. Test against your actual catalog, queries, users, and evaluation policy before using the model. The model may make unsupported inferences or fail to respect instructions; results can reflect biases in the synthetic examples.

License and attribution

The adapter and model-card documentation are prepared under Apache 2.0, following the prior La Trouvaille adapter release; see LICENSE and NOTICE. The upstream model is also Apache 2.0. The bundled prompts, templates, and renderer originate from the application and retain their respective source licenses. Training data and other application source are not included.