Instructions to use safffrron/25M2111-Week02-Track1-20-Submission01 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use safffrron/25M2111-Week02-Track1-20-Submission01 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="safffrron/25M2111-Week02-Track1-20-Submission01")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("safffrron/25M2111-Week02-Track1-20-Submission01", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use safffrron/25M2111-Week02-Track1-20-Submission01 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "safffrron/25M2111-Week02-Track1-20-Submission01"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "safffrron/25M2111-Week02-Track1-20-Submission01",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/safffrron/25M2111-Week02-Track1-20-Submission01

SGLang

How to use safffrron/25M2111-Week02-Track1-20-Submission01 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "safffrron/25M2111-Week02-Track1-20-Submission01" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "safffrron/25M2111-Week02-Track1-20-Submission01",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "safffrron/25M2111-Week02-Track1-20-Submission01" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "safffrron/25M2111-Week02-Track1-20-Submission01",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use safffrron/25M2111-Week02-Track1-20-Submission01 with Docker Model Runner:
```
docker model run hf.co/safffrron/25M2111-Week02-Track1-20-Submission01
```

Week 02 — Track 1 — 20% target

This is the complete evaluation and reproduction package for enrollment 25M2111. It contains a losslessly packed W3/W4/W8 base plus a physically charged rank-8 grouped-JSD residual.

Item	Value
Base model	`Qwen/Qwen3.5-4B`
Base artifact	`week02_20_base.ptz`
Retained vocabulary	`week02_20_keep_ids.json` (30,000 output IDs)
Residual	`adapter/` (rank 8, 400 tensors)
Exact charged size	1,657,231,034 bytes
20% ceiling margin	25,168,966 bytes
Base SHA256	`2b63ba3af765f54baf78536c0b46b4a8ccf9f6ac2b294001b7653e9069410091`
Week-2 matched checkpoint	0.871 accuracy on 560 problems
GitHub	https://github.com/safffrron/CS6013/tree/main/25M2111/Week02/Track1_20/Submission01
Hugging Face	https://huggingface.co/safffrron/25M2111-Week02-Track1-20-Submission01

Evaluate the uploaded checkpoint

Run these commands from this Submission01 directory. A Conda environment name is not assumed.

python -m venv .venv
source .venv/bin/activate
python -m pip install -e '.[eval]'

hf download safffrron/25M2111-Week02-Track1-20-Submission01 \
  --local-dir hf_checkpoint
sha256sum hf_checkpoint/week02_20_base.ptz

EAIMATH_RESTORE_DEVICE=cuda:0 python dequantize_to_bf16.py \
  Qwen/Qwen3.5-4B hf_checkpoint restored_hf

python inference/smoke.py restored_hf
./evaluation/run_public_eval.sh \
  restored_hf 0 restored_hf/keep_ids.json

The digest must match the table. The dequantizer restores the token-predicted BF16 base, safely merges the submitted adapter, and writes an ordinary Hugging Face checkpoint. The evaluation defaults to max_new_tokens=65536 and repetition penalty 1.20; its artifact evidence used 32,768 tokens.

Required conversion API

code.py provides:

convert_from_hf_checkpoint(model_name, output_path, sparsity=None)
convert_to_hf_checkpoint(model_name, checkpoint_path, output_path)

The optional sparsity value exists only for starter-code compatibility and is ignored: this submission does not use pruning. checkpoint_path may be the download directory or its week02_20_base.ptz file. Missing companion files are fetched automatically from the submission's Hugging Face repository.

Reproduce compression and recovery

The optional full rebuild uses the verified trace corpus and a GPU. First run training/reproduce_source.sh, then compression/reproduce_gptq.sh, then training/reproduce_adapter.sh. Point the three EAIMATH_* variables printed by those scripts at the resulting GPTQ source, trace corpus, and JSD adapter, then run:

python -m pip install -e '.[compress,eval]'
python convert_from_hf_checkpoint.py Qwen/Qwen3.5-4B reproduced_week02_20

The method uses group-128 calibrated GPTQ (3-bit MLP, 4-bit Gated DeltaNet, 8-bit full attention), 30,000 selected 8-bit tied vocabulary rows, deterministic token-string prediction for omitted input rows, and a rank-8 residual trained after quantization against grouped Jensen–Shannon teacher distributions. Machine-readable settings and matched results are in configs/ and results/.

Downloads last month: -; Downloads are not tracked for this model. How to track

Model tree for safffrron/25M2111-Week02-Track1-20-Submission01

Base model

Qwen/Qwen3.5-4B-Base

Finetuned

Qwen/Qwen3.5-4B

Finetuned

(514)

this model