Instructions to use slopops/BigBang-v1-int4-AutoRound-SAR with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use slopops/BigBang-v1-int4-AutoRound-SAR with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="slopops/BigBang-v1-int4-AutoRound-SAR")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("slopops/BigBang-v1-int4-AutoRound-SAR")
model = AutoModelForMultimodalLM.from_pretrained("slopops/BigBang-v1-int4-AutoRound-SAR", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use slopops/BigBang-v1-int4-AutoRound-SAR with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "slopops/BigBang-v1-int4-AutoRound-SAR"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "slopops/BigBang-v1-int4-AutoRound-SAR",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/slopops/BigBang-v1-int4-AutoRound-SAR

SGLang

How to use slopops/BigBang-v1-int4-AutoRound-SAR with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "slopops/BigBang-v1-int4-AutoRound-SAR" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "slopops/BigBang-v1-int4-AutoRound-SAR",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "slopops/BigBang-v1-int4-AutoRound-SAR" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "slopops/BigBang-v1-int4-AutoRound-SAR",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use slopops/BigBang-v1-int4-AutoRound-SAR with Docker Model Runner:
```
docker model run hf.co/slopops/BigBang-v1-int4-AutoRound-SAR
```

BigBang-v1-int4-AutoRound-SAR

File size: 5,149 Bytes

2e6eea6

block_name,cosine_sim,psnr_db,init_loss,best_loss,best_iter,total_iters,passed
model.language_model.layers.0,0.999398365849629,62.74563344285527,1.0857593224500306e-05,5.764709953837155e-07,939,940,True
model.language_model.layers.1,0.9994502658955753,64.82866918092782,3.423117959755473e-06,7.940604405121121e-07,946,947,True
model.language_model.layers.2,0.9994643313111737,66.56976112856243,6.7428313741402235e-06,1.4173290310282027e-06,962,963,True
model.language_model.layers.3,0.9991424762411043,63.01271217243214,8.369693205168005e-06,2.329857125005219e-06,757,758,True
model.language_model.layers.4,0.9990625486243516,62.46786011859052,6.1128548622946255e-06,2.9815180369041627e-06,930,931,True
model.language_model.layers.5,0.9988802372245118,61.774415982165415,8.27899930300191e-06,3.915218258043751e-06,949,950,True
model.language_model.layers.6,0.998835175880231,72.19373069245982,9.441095244255848e-06,4.567613814288052e-06,873,874,True
model.language_model.layers.7,0.9981491551734507,68.46034901115699,1.567434628668707e-05,6.877670784888323e-06,913,914,True
model.language_model.layers.8,0.9980537934461609,67.27181583739291,1.3707451216760091e-05,7.847530469007324e-06,989,990,True
model.language_model.layers.9,0.9979800393339247,66.42990535001184,1.5406149032060057e-05,9.470289114688057e-06,974,975,True
model.language_model.layers.10,0.998262589215301,80.91685851492745,2.555710671003908e-05,1.2048375538142864e-05,841,842,True
model.language_model.layers.11,0.9977714883862063,79.93861522324254,3.196421675966121e-05,1.5139507013373077e-05,994,995,True
model.language_model.layers.12,0.9974960462423041,79.76714673313498,2.5527971956762485e-05,1.554579648654908e-05,945,946,True
model.language_model.layers.13,0.9971242630854249,78.81509765782293,2.8832459065597504e-05,1.8955182895297185e-05,977,978,True
model.language_model.layers.14,0.9973322114674374,78.2770185795565,3.5891942388843745e-05,2.208765181421768e-05,975,976,True
model.language_model.layers.15,0.9957852772204205,76.69351559956701,5.0056758482242e-05,3.0643073841929436e-05,673,674,True
model.language_model.layers.16,0.9954372909851372,75.96600612157582,5.6429245887557045e-05,3.477871359791607e-05,885,886,True
model.language_model.layers.17,0.9947229368845001,75.40505509164578,6.142834899947047e-05,3.867841951432638e-05,466,467,True
model.language_model.layers.18,0.9944830744061619,74.19776407711409,7.809165981598198e-05,5.206267815083265e-05,838,839,True
model.language_model.layers.19,0.9924883618950844,72.61358713131442,0.00011300867481622845,7.609669410157949e-05,840,841,True
model.language_model.layers.20,0.992287234403193,72.48704695706869,0.0001036642788676545,7.987261051312089e-05,686,687,True
model.language_model.layers.21,0.9919371472205967,72.14505604576752,0.00011655662092380226,8.175231778295711e-05,823,824,True
model.language_model.layers.22,0.9912291804794222,70.7923832687534,0.00016804040933493525,0.00010687758913263679,791,792,True
model.language_model.layers.23,0.9888476736377925,69.86910853504358,0.0001921755902003497,0.00013590486196335405,585,586,False
model.language_model.layers.24,0.9879246136406437,69.77261830277699,0.00017378482152707875,0.00013574025069829077,681,682,False
model.language_model.layers.25,0.9870466706342995,69.10043018728072,0.00021691163419745862,0.0001530000299680978,414,415,False
model.language_model.layers.26,0.9880332896718755,68.83073395801314,0.00022121460642665625,0.0001747026399243623,581,582,False
model.language_model.layers.27,0.9858953485963866,67.9221492210291,0.00028360538999550045,0.00021544296760112047,974,975,False
model.language_model.layers.28,0.9868189338594675,67.65130797069699,0.0003018546849489212,0.0002305347443325445,690,691,False
model.language_model.layers.29,0.9869267698377371,67.66501271352898,0.00031459511956200004,0.0002227950608357787,827,828,False
model.language_model.layers.30,0.9873259431915358,66.92135852615802,0.0003490559756755829,0.0002749512204900384,793,794,False
model.language_model.layers.31,0.9854578910162672,49.17874055336315,0.00046271103201434016,0.00034802505979314446,949,950,False
model.language_model.layers.32,0.985219054738991,47.734857879451155,0.0005040763644501567,0.0003757431695703417,782,783,False
model.language_model.layers.33,0.9869168716249987,47.92476671071529,0.0006758735980838537,0.00045035238144919276,230,231,False
model.language_model.layers.34,0.9876180860446766,59.0211915817988,0.0007961574010550976,0.0005668293451890349,840,841,False
model.language_model.layers.35,0.9868915913393721,47.60994672390391,0.0011157216504216194,0.0007601645775139332,813,814,False
model.language_model.layers.36,0.9868664080277085,46.926808972687255,0.0012197608593851328,0.0008616059785708785,912,913,False
model.language_model.layers.37,0.9884766169125214,47.350775089610906,0.0016283139120787382,0.0010883102659136057,819,820,False
model.language_model.layers.38,0.9898017558734864,57.04915977406314,0.0022520131897181273,0.0015673397574573755,717,718,False
model.language_model.layers.39,0.9893610094441101,54.98738274052245,0.0031894505955278873,0.0020158421248197556,581,582,False