Instructions to use TroyDoesAI/MermaidMistral with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TroyDoesAI/MermaidMistral with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="TroyDoesAI/MermaidMistral")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("TroyDoesAI/MermaidMistral")
model = AutoModelForCausalLM.from_pretrained("TroyDoesAI/MermaidMistral")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use TroyDoesAI/MermaidMistral with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TroyDoesAI/MermaidMistral"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TroyDoesAI/MermaidMistral",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/TroyDoesAI/MermaidMistral

SGLang

How to use TroyDoesAI/MermaidMistral with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "TroyDoesAI/MermaidMistral" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TroyDoesAI/MermaidMistral",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "TroyDoesAI/MermaidMistral" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TroyDoesAI/MermaidMistral",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use TroyDoesAI/MermaidMistral with Docker Model Runner:
```
docker model run hf.co/TroyDoesAI/MermaidMistral
```

TroyDoesAI commited on Jan 9, 2024

Commit

14b5793

1 Parent(s): 4069ae8

Update README.md

Browse files

Added Photos of generations and link to implement mermaid js diagrams to images.

Files changed (1) hide show

README.md +17 -0

README.md CHANGED Viewed

@@ -27,6 +27,17 @@ MermaidMistral is not a "Chatty Kathy" and should only respond with a mermaid co
 - Adaptable and efficient in resource utilization
 - Due to hardware constraints this fine tune has a token limit of 2048.
 ### Collaboration:
 MermaidMistral is open to collaboration to further strengthen its capabilities. The dataset, formatted in Alpaca, provides a unique foundation for understanding Python intricacies. If you're interested in contributing or collaborating to enhance the model's performance, feel free to reach out to [troydoesai@gmail.com](mailto:troydoesai@gmail.com). Your expertise could play a pivotal role in refining MermaidMistral.
@@ -52,6 +63,12 @@ MermaidMistral is open to collaboration to further strengthen its capabilities.
 MermaidMistral proves that innovation thrives in compact packages, delivering exceptional performance across diverse applications. Its adaptability and efficiency showcase the potential for groundbreaking results even in resource-constrained environments.
 ### Example Story -> Flow
 https://chat.openai.com/share/e3163857-981b-4968-b2db-98ad869c9259

 - Adaptable and efficient in resource utilization
 - Due to hardware constraints this fine tune has a token limit of 2048.
+Certainly! Here are the updated links in markdown:
+1. Mermaid Mistral Generation 1:
+   ![Mermaid Mistral Generation 1](https://cdn.discordapp.com/attachments/1022059414280228875/1194088817284612178/MermaidMistral_Generation1.png?ex=65af14df&is=659c9fdf&hm=dc1b8ddeb658de80c1646838fbd3c15a46a1d8efefec2b609a1bb48aaa238174&)
+2. Mermaid Mistral Generation 2:
+   ![Mermaid Mistral Generation 2](https://cdn.discordapp.com/attachments/1022059414280228875/1194088816416399462/MermaidMistral_Generation2.png?ex=65af14de&is=659c9fde&hm=c259ee6ad27824a0b932a936699d276d5f1ffd0418ac608de6cd96784c46aba1&)
+3. ChatGPT Generation:
+   ![ChatGPT Generation](https://cdn.discordapp.com/attachments/1022059414280228875/1194088816823242812/ChatGPT_Generation1.png?ex=65af14df&is=659c9fdf&hm=b5e2c75286ea7a1313ae455402e14cf2bb7e82c5fb66d92acc542909cf2a6352&)
 ### Collaboration:
 MermaidMistral is open to collaboration to further strengthen its capabilities. The dataset, formatted in Alpaca, provides a unique foundation for understanding Python intricacies. If you're interested in contributing or collaborating to enhance the model's performance, feel free to reach out to [troydoesai@gmail.com](mailto:troydoesai@gmail.com). Your expertise could play a pivotal role in refining MermaidMistral.
 MermaidMistral proves that innovation thrives in compact packages, delivering exceptional performance across diverse applications. Its adaptability and efficiency showcase the potential for groundbreaking results even in resource-constrained environments.
+These mermaid codeblocks can be converted directly into images using mermaid cli tool found here: https://github.com/mermaid-js/mermaid-cli
+I plan to release my working proof of concept VSCode Extension that currently displays the ```Live Flow Map``` every time a user stops typing for more than 10 seconds.
+Stay tuned.
 ### Example Story -> Flow
 https://chat.openai.com/share/e3163857-981b-4968-b2db-98ad869c9259