Instructions to use FPHam/Writing_Partner_Mistral_7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use FPHam/Writing_Partner_Mistral_7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="FPHam/Writing_Partner_Mistral_7B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("FPHam/Writing_Partner_Mistral_7B")
model = AutoModelForCausalLM.from_pretrained("FPHam/Writing_Partner_Mistral_7B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use FPHam/Writing_Partner_Mistral_7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FPHam/Writing_Partner_Mistral_7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FPHam/Writing_Partner_Mistral_7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/FPHam/Writing_Partner_Mistral_7B

SGLang

How to use FPHam/Writing_Partner_Mistral_7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "FPHam/Writing_Partner_Mistral_7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FPHam/Writing_Partner_Mistral_7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "FPHam/Writing_Partner_Mistral_7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FPHam/Writing_Partner_Mistral_7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use FPHam/Writing_Partner_Mistral_7B with Docker Model Runner:
```
docker model run hf.co/FPHam/Writing_Partner_Mistral_7B
```

FPHam commited on Nov 13, 2023

Commit

16766cf

1 Parent(s): 606f30f

Update README.md

Browse files

Files changed (1) hide show

README.md +27 -0

README.md CHANGED Viewed

	@@ -59,5 +59,32 @@ In the distant reaches of the cosmos, where starlight weaves a tapestry of unkno
59	>"Thank you, Ensign," Alara replied, not taking her eyes off the swirling vortex ahead. "Let's see what lies beyond."
60
61



























62
63

 >"Thank you, Ensign," Alara replied, not taking her eyes off the swirling vortex ahead. "Let's see what lies beyond."
+Example 2:
+How to write proper dialogue?
+>Dialogue is a very important part of writing fiction. It's the way characters speak, and it helps define who they are. It also moves the story forward and adds tension, humor, or emotion. Dialogue must be realistic and natural-sounding, but it doesn't have to be boring. You want your readers to pay attention to what the characters are saying, so make sure that each line of dialogue counts.
+>
+>Here are some tips on how to write good dialogue:
+>
+>    Keep it short. Long speeches get tedious quickly. If one character talks for more than two sentences, break up his speech with an action or interjection from another character.
+>
+>    Use contractions (I'm, he'll). They sound more like real conversation than formal language does.
+>
+>    Vary sentence structure. This makes dialogue easier to read and understand.
+>
+>    Don't use slang unless it fits the setting or character perfectly. Slang dates easily and may not mean anything to future generations.
+>
+>    Avoid using too many adverbs in dialogue tags. Instead of "he said loudly," try something like "he bellowed." Or "she whispered."
+>
+>    Be careful not to give away too much information in dialogue tags. For example, if you say "Sarah laughed nervously," you're telling the reader she was nervous rather than showing her anxiety through her words or actions.
+>
+>    Make sure your characters don't all talk alike. Each person should have a unique voice, based on his background, education, age, gender, etc.
+>
+>    Don't forget about body language. The way someone holds himself, sits, stands, fidgets, or looks at others can add depth to a scene.
+>
+>    Use dialogue to reveal character traits and motivations. Listen carefully to people around you, and then translate those conversations onto paper.
+>
+>    Finally, remember that dialogue needs to serve a purpose in your story. Whether it advances the plot, sets the mood, or creates conflict, every word spoken by your characters should move the narrative along.