irisaparina/llama-3-8b-instruct-infilling-lora

Model Card

Model Details

Model Description

This is a LoRA adapter for the meta-llama/Llama-3.1-8B-Instruct model, fine-tuned for the task of interpretation infilling in ambiguous text-to-SQL semantic parsing. The adapter is trained on the AmbiQT and Ambrosia* datasets, as described in the paper:

Disambiguate First, Parse Later: Generating Interpretations for Ambiguity Resolution in Semantic Parsing
Irina Saparina, Mirella Lapata
Findings of ACL 2025

The model is designed to take an ambiguous natural language question, database context, and a set of initial interpretations, and generate any missing interpretations in natural language. This modular approach improves coverage and recall for ambiguous queries in text-to-SQL tasks.

More details on how to train and evaluate are in this repo.

• Developed by: Irina Saparina, Mirella Lapata
• Model type: LoRA adapter for Llama 3.1 8B Instruct
• Language(s): English
• License: CC BY-NC-SA 4.0
• Finetuned from model: meta-llama/Llama-3.1-8B-Instruct

Model Sources

• Repository: https://github.com/saparina/disambiguate-then-parse
• Paper: Disambiguate First, Parse Later: Generating Interpretations for Ambiguity Resolution in Semantic Parsing (Saparina & Lapata, Findings 2025)

Uses

Direct Use

This adapter is intended for use in modular semantic parsing pipelines, specifically for the interpretation infilling step. Given a database context, an ambiguous question, and a set of initial interpretations, the model generates any missing interpretations in natural language.

Example prompt:

The task is to review the provided context, question, and existing interpretations, and determine if any additional interpretations are missing. If there are missing interpretations, list them on separate lines without explanations. If all interpretations have already been covered, simply state: "All possible interpretations are covered."

Given the following context:

{}

Question:

{}

Existing interpretations:

{}

Provide any missing interpretations or confirm that all possible interpretations are covered.

How to Get Started with the Model

You can load the adapter using the PEFT library and Hugging Face Transformers:

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel, PeftConfig

base_model = "meta-llama/Llama-3.1-8B-Instruct"
adapter_path = "path/to/this/adapter"

tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(base_model, torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, adapter_path)

The model should be combined with the generator of the initial interpretations (e.g., Llama 3.1 8B Instruct) and text-to-SQL parser (e.g., Qwen2.5 Coder 32B Instruct). See more details in the repo.

Training Details

Training Data

• Dataset: AmbiQT, Ambrosia (ambiguous text-to-SQL questions with multiple interpretations)
• Data preprocessing: See code repository for details.

Training Procedure

• Base model: meta-llama/Llama-3.1-8B-Instruct
• Adapter: LoRA (PEFT)
• LoRA rank: 16 (α=16)
• Noise: NEFTune (α=5)
• Epochs: 15
• Batch size: 8
• Learning rate: 5e-5 (cosine schedule, warmup ratio 0.01)
• Weight decay: 0.01
• Gradient clipping: 0.3
• Hardware: NVIDIA A100 GPU
• Framework: PEFT 0.14.0

Evaluation

Testing Data, Factors & Metrics

• Test sets: AmbiQT, Ambrosia (re-splitted)
• Metrics: Single and full interpretation coverage, recall, precision (see paper for details)

Results

See Table 9 in the paper for detailed results. The proposed system achieves state-of-the-art coverage and recall for ambiguous question interpretation infilling.

Technical Specifications

• Architecture: Llama 3.1 8B Instruct + LoRA adapter (rank 16)
• Objective: Supervised fine-tuning for interpretation infilling
• Hardware Type: NVIDIA A100 GPU
• Hours used: <10 hours per run

Citation

If you use this model, please cite:

BibTeX:

@inproceedings{saparina2025disambiguate,
  title={Disambiguate First, Parse Later: Generating Interpretations for Ambiguity Resolution in Semantic Parsing},
  author={Saparina, Irina and Lapata, Mirella},
  booktitle={Findings of ACL},
  year={2025},
  url={https://aclanthology.org/2025.findings-acl.863/}
}

Model Card Contact

• Irina Saparina

---

For more details, see the code repository and the paper.

---