dsuman/gemma3-12b-GPTQ-RTN

Gemma 3 12B — GPTQ + RTN (W8A8)

Compression methods employed

GPTQ-based weight quantization combined with naive round-to-nearest (RTN) activation quantization, producing a W8A8 (8-bit weight, 8-bit activation) compressed model, with inference executed directly in 8-bit precision.

1. Set up the environment

bash setup.sh

2. Run inference

bash run.sh \
  --lang-pair ces-deu \
  --batch-size 512 \
  --input input.txt \
  --output output.txt

Arguments:

Flag	Description
--lang-pair	Language pair to translate. This submission only supports ces-deu (Czech → German).
--batch-size	Number of sequences processed per batch (max_num_seqs in vLLM).
--input	Path to a plain-text file with one source sentence per line.
--output	Path to write translations to — one line per input line, in the same order.