Text Generation
Transformers
PyTorch
French
mistral
text-generation-inference
8-bit precision
bitsandbytes
Instructions to use Pclanglais/Larth-Mistral with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Pclanglais/Larth-Mistral with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Pclanglais/Larth-Mistral")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Pclanglais/Larth-Mistral") model = AutoModelForCausalLM.from_pretrained("Pclanglais/Larth-Mistral") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use Pclanglais/Larth-Mistral with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Pclanglais/Larth-Mistral" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Pclanglais/Larth-Mistral", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Pclanglais/Larth-Mistral
- SGLang
How to use Pclanglais/Larth-Mistral with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Pclanglais/Larth-Mistral" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Pclanglais/Larth-Mistral", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Pclanglais/Larth-Mistral" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Pclanglais/Larth-Mistral", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Pclanglais/Larth-Mistral with Docker Model Runner:
docker model run hf.co/Pclanglais/Larth-Mistral
Commit History
Upload MistralForCausalLM 82d20bb
Update README.md 4698fc7
Upload tokenizer f50aeee
Upload MistralForCausalLM d9ab070
Update README.md 1a87941
Update README.md f767e32
Update README.md 55910c1
Update README.md 92de4e2
Update README.md edf934e
Upload tokenizer 9df50cc
Upload MistralForCausalLM 4f7dab8
Delete pytorch_model.bin a07c684
Upload tokenizer 2d7b752
Upload MistralForCausalLM da4d0d3
initial commit 331a611
Pierre-Carl Langlais commited on