Instructions to use safffrron/25M2111-Week01-Track2-20-Submission01 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use safffrron/25M2111-Week01-Track2-20-Submission01 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="safffrron/25M2111-Week01-Track2-20-Submission01")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("safffrron/25M2111-Week01-Track2-20-Submission01", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use safffrron/25M2111-Week01-Track2-20-Submission01 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "safffrron/25M2111-Week01-Track2-20-Submission01"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "safffrron/25M2111-Week01-Track2-20-Submission01",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/safffrron/25M2111-Week01-Track2-20-Submission01

SGLang

How to use safffrron/25M2111-Week01-Track2-20-Submission01 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "safffrron/25M2111-Week01-Track2-20-Submission01" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "safffrron/25M2111-Week01-Track2-20-Submission01",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "safffrron/25M2111-Week01-Track2-20-Submission01" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "safffrron/25M2111-Week01-Track2-20-Submission01",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use safffrron/25M2111-Week01-Track2-20-Submission01 with Docker Model Runner:
```
docker model run hf.co/safffrron/25M2111-Week01-Track2-20-Submission01
```

Week 01 — Track 2 — 20% target

This directory is a complete evaluator and reproduction package for enrollment 25M2111.

Item	Value
Base model	`Qwen/Qwen3.5-4B`
Compressed artifact	`r13_short_b_m3l4a8e8_body_token_zlib.ptz`
Exact size	1,626,481,790 bytes
SHA256	`2b63ba3af765f54baf78536c0b46b4a8ccf9f6ac2b294001b7653e9069410091`
Recorded checkpoint accuracy	0.879 on 560 problems
GitHub	https://github.com/safffrron/CS6013/tree/main/25M2111/Week01/Track2_20/Submission01
Hugging Face	https://huggingface.co/safffrron/25M2111-Week01-Track2-20-Submission01

Evaluate the submitted artifact

The following is the shortest complete evaluator path. Run it from this Submission01 directory. The compressed artifact is restored first; do not pass the .ptz file directly to Transformers or vLLM.

1. Install

Python 3.11 or later and a CUDA-capable machine are required. Any Conda or virtual environment may be used; its name is irrelevant. The following creates an isolated environment for a clean evaluation. If an equivalent environment is already active, skip the two environment-creation lines. If starting from a fresh machine, clone the private submission repository using an account that has been added as a collaborator:

git clone https://github.com/safffrron/CS6013.git
cd CS6013/25M2111/Week01/Track2_20/Submission01
python -m venv .venv
source .venv/bin/activate
python --version
python -m pip install -e '.[eval]'

2. Download and verify the checkpoint

mkdir -p hf_checkpoint
hf download safffrron/25M2111-Week01-Track2-20-Submission01 \
  r13_short_b_m3l4a8e8_body_token_zlib.ptz \
  r13_short_b_m3l4a8e8_body_token_zlib.ptz.json \
  r13_short_b_m3l4a8e8_body_token_zlib.ptz.keep_ids.json \
  --local-dir hf_checkpoint
sha256sum hf_checkpoint/r13_short_b_m3l4a8e8_body_token_zlib.ptz

The printed digest must equal the SHA256 in the table above.

3. Restore an ordinary BF16 Hugging Face checkpoint

python dequantize_to_bf16.py \
  Qwen/Qwen3.5-4B \
  hf_checkpoint/r13_short_b_m3l4a8e8_body_token_zlib.ptz \
  restored_hf

Restoration is deterministic. restored_hf/submission_report.json records the restoration details.

4. Smoke-test and evaluate

python inference/smoke.py restored_hf
./evaluation/run_public_eval.sh \
  restored_hf \
  0 \
  hf_checkpoint/r13_short_b_m3l4a8e8_body_token_zlib.ptz.keep_ids.json

The 0 is the physical GPU id. Multiple GPUs may be supplied as 0,1. The evaluation script clears any inherited CUDA mask, runs the checkpoint suite with max_new_tokens=65536 and repetition penalty 1.20, then merges shards. The keep_ids file is required because this submission evaluates logits over the 30,000 retained original output-token ids.

Course conversion API

code.py provides both required functions:

convert_from_hf_checkpoint(model_name, output_path, sparsity)
convert_to_hf_checkpoint(model_name, checkpoint_path, output_path)

The sparsity argument is accepted only for compatibility with the supplied course evaluator and is ignored by this compression method.

The command-line wrappers are convert_from_hf_checkpoint.py and dequantize_to_bf16.py. Evaluating the uploaded artifact requires only the dequantization wrapper. Rebuilding the artifact from the base model is the longer optional process below.

Rebuild the compressed artifact from source

This reproduces the learned short-reasoning source, calibrated GPTQ weights, vocabulary selection, token-string predictor, and final lossless package. It requires the verified training/calibration trace JSONL and a GPU.

python -m pip install -e '.[compress,eval]'
./training/reproduce_source.sh data/traces.jsonl 0 work/short2500
./compression/reproduce_gptq.sh \
  work/short2500 data/traces.jsonl 0 work/b20
export EAIMATH_B20_GPTQ_SOURCE="$PWD/work/b20/gptq_m3l4a8e8"
export EAIMATH_VOCAB_CORPUS="$PWD/data/traces.jsonl"
export EAIMATH_PREDICTOR_DEVICE=cuda:0
python convert_from_hf_checkpoint.py \
  Qwen/Qwen3.5-4B reproduced.ptz

The produced sidecars are reproduced.ptz.json and reproduced.ptz.keep_ids.json.

Method and evidence

The learned source is compressed with group-128 calibrated GPTQ: 3-bit MLP, 4-bit Gated DeltaNet projections, and 8-bit full attention. Sensitive recurrent controls and small tensors remain BF16. The tied embedding/output table stores 30,000 selected rows at 8 bits, and a 701,760-byte deterministic token-string predictor reconstructs omitted input rows. Zlib then losslessly reduces the serialized artifact from 1,790,943,184 to 1,626,481,790 bytes.

The physical restoration scored 0.879 accuracy, 0.970 parse rate, and 0.037 truncation rate on the 560-problem checkpoint suite at a recorded 32,768-token cap. Machine-readable configuration and evidence are in configs/ and results/ in this GitHub submission.

Track 2 CUDA rationale

This is byte-for-byte the same artifact as the corresponding Track 1 model. Its packed W3/W4/W8 weight representation can be executed as weight-only WnA16 matrix multiplication without first materializing all BF16 weights. The compact vocabulary also permits a gathered output projection over retained rows. CUDA_ACCELERATION.md documents applicable kernels, Qwen3.5 shape constraints, and the boundary between the submitted BF16 restoration path and the possible direct CUDA path. No unimplemented speedup is reported as measured.

Downloads last month: -; Downloads are not tracked for this model. How to track