Instructions to use liuda1/Mistral-7B-golden with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use liuda1/Mistral-7B-golden with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="liuda1/Mistral-7B-golden")

# Load model directly
from transformers import AutoTokenizer, AutoModelForMultimodalLM

tokenizer = AutoTokenizer.from_pretrained("liuda1/Mistral-7B-golden")
model = AutoModelForMultimodalLM.from_pretrained("liuda1/Mistral-7B-golden")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use liuda1/Mistral-7B-golden with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "liuda1/Mistral-7B-golden"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "liuda1/Mistral-7B-golden",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/liuda1/Mistral-7B-golden

SGLang

How to use liuda1/Mistral-7B-golden with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "liuda1/Mistral-7B-golden" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "liuda1/Mistral-7B-golden",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "liuda1/Mistral-7B-golden" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "liuda1/Mistral-7B-golden",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use liuda1/Mistral-7B-golden with Docker Model Runner:
```
docker model run hf.co/liuda1/Mistral-7B-golden
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Our model is based on Mistral-7B-v0.1 as the basic model, with English chat dataset added for fine-tuning training, and further reinforcement training based on specific datasets. The trained model has a certain level of chat ability, which was found to be enhanced during self testing. We will continue to train the model in the future to improve our Chinese chat ability

Downloads last month: 517

Model tree for liuda1/Mistral-7B-golden

Quantizations

1 model

liuda1
/

Mistral-7B-golden

Model tree for liuda1/Mistral-7B-golden

Spaces using liuda1/Mistral-7B-golden 9