sqlcoder-7b-2-ONNX / README.md
PY-AI-Dev's picture
Add ONNX export for sqlcoder-7b-2
f60cd0a verified
|
Raw History Blame Contribute Delete
1.89 kB
metadata
license: other
base_model: defog/sqlcoder-7b-2
base_model_relation: quantized
library_name: onnx
pipeline_tag: text-generation
tags:
  - onnx
  - onnxruntime
  - quantized
quantized_by: liodon-ai

sqlcoder-7b-2 — ONNX

ONNX export of defog/sqlcoder-7b-2, published by Liodon AI. Exported with optimum (optimum.exporters.onnx.main_export, task text-generation-with-past, so the graph exposes past-key-value inputs/outputs for KV-cached autoregressive decoding).

Files

File Size Notes
model.onnx 26.96 GB FP32, full precision
model_fp16.onnx 14.55 GB FP16, for GPU execution providers

Quick Start

import onnxruntime as ort
from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("liodon-ai/sqlcoder-7b-2-ONNX")
sess = ort.InferenceSession("model_quantized.onnx", providers=["CPUExecutionProvider"])
# past_key_values.*.key / .value inputs must be supplied (zero-length tensors
# for the first forward pass) -- see optimum's ORTModelForCausalLM for a
# ready-made wrapper that handles KV-cache bookkeeping automatically:
#   from optimum.onnxruntime import ORTModelForCausalLM
#   model = ORTModelForCausalLM.from_pretrained("liodon-ai/sqlcoder-7b-2-ONNX", file_name="model_quantized.onnx")

Source

Citation

@misc{liodonai_sqlcoder_7b_2_onnx,
  title        = {sqlcoder-7b-2 — ONNX},
  author       = {{Liodon AI}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/liodon-ai/sqlcoder-7b-2-ONNX}},
  note         = {ONNX export of defog/sqlcoder-7b-2}
}

Exported by Liodon AI