Hugging Face's logo

solidrust
/
Nutopia-7B-AWQ

Text Generation
Transformers
Safetensors
mistral
mergekit
Merge
4-bit precision
AWQ
text-generation-inference
awq
Model card Files
xet
 Community
How to use from
vLLM
Install from pip and serve model 
# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "solidrust/Nutopia-7B-AWQ"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "solidrust/Nutopia-7B-AWQ",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'
Use Docker 
docker model run hf.co/solidrust/Nutopia-7B-AWQ
Quick Links

Alsebay/Nutopia-7B AWQ

Model Summary

Testing purpose only, seem it not good in Roleplaying 😢

This model was merged using the SLERP merge method.

The following models were included in the merge:

Downloads last month
4
Safetensors
Model size
7B params
Tensor type
I32
·
F16
·
Inference Providers NEW
Text Generation
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including solidrust/Nutopia-7B-AWQ