Instructions to use pcalhoun/Mistral-7B-v0.3-JonathanSwift with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use pcalhoun/Mistral-7B-v0.3-JonathanSwift with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="pcalhoun/Mistral-7B-v0.3-JonathanSwift")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("pcalhoun/Mistral-7B-v0.3-JonathanSwift") model = AutoModelForCausalLM.from_pretrained("pcalhoun/Mistral-7B-v0.3-JonathanSwift") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use pcalhoun/Mistral-7B-v0.3-JonathanSwift with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "pcalhoun/Mistral-7B-v0.3-JonathanSwift" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "pcalhoun/Mistral-7B-v0.3-JonathanSwift", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/pcalhoun/Mistral-7B-v0.3-JonathanSwift
- SGLang
How to use pcalhoun/Mistral-7B-v0.3-JonathanSwift with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "pcalhoun/Mistral-7B-v0.3-JonathanSwift" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "pcalhoun/Mistral-7B-v0.3-JonathanSwift", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "pcalhoun/Mistral-7B-v0.3-JonathanSwift" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "pcalhoun/Mistral-7B-v0.3-JonathanSwift", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use pcalhoun/Mistral-7B-v0.3-JonathanSwift with Docker Model Runner:
docker model run hf.co/pcalhoun/Mistral-7B-v0.3-JonathanSwift
Format:
<|context|> context goes here <|plain|> plain text to convert to Jon Swift goes here <|swiftify|> Jon Swift version of plain text is generated here <|context|> completely new context goes here (unrelated to prior) <|plain|> plain text to convert to Jon Swift goes here <|swiftify|> Jon Swift version of plain text is generated here <|context|> etc...
Results:
<|context|> He came up with what he called the super colossal test, and he was able to conceptualize it and design it. But he wasn’t able to prove it mathematically because, although he was a skilled mathematician, he said, “I’m not there. I need a little bit of a better mathematician.” He ended up bringing in Abraham Wald to finalize the results of the idea they had come up with, and this became known as Sequential Analysis and ended up being a huge boon to the US military in terms of enabling them to test their ordnance and move forward. <|plain|> It’s fascinating in terms of Friedman’s later life — his earlier career, his intellectual discoveries are very much embedded in a growing federal government that’s growing because of the Great Depression, that’s growing because of World War II. This is what enables him to develop that statistical prowess. It gives him a very strong reputation, a field of economics. <|swiftify|> And how Friedman's after life hath been all employed, his whole future destiny engaged, in the very bowels of this ever swelling Federal leviathan, engendered by the great depression and the second war; and how it hath produced in him this unparalleled power of numbers; and given him an entire reputation, a very great name in that country of political economy!
- Downloads last month
- 2