NorthernTribe-Research/UMSR-Reasoner-7B-INT8

UMSR-Reasoner-7B (INT8)

8-bit bitsandbytes runtime quantization profile.

Model Tree

• Source model: https://huggingface.co/NorthernTribe-Research/UMSR-Reasoner-7B
• Variant repo: https://huggingface.co/NorthernTribe-Research/UMSR-Reasoner-7B-INT8
• Dataset: https://huggingface.co/datasets/NorthernTribe-Research/UMSR-v1

Quantization Profile

• Variant: int8
• Scheme: bitsandbytes int8
• Materialized weights in this repo: False

Usage

This variant repository is a profile card. Load weights from the source model id.

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig

model_id = "NorthernTribe-Research/UMSR-Reasoner-7B"
bnb_config = BitsAndBytesConfig(load_in_8bit=True)

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    quantization_config=bnb_config,
)

Output Contract

<final_answer>...</final_answer>