Instructions to use phanerozoic/MistralPirate-7b-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use phanerozoic/MistralPirate-7b-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="phanerozoic/MistralPirate-7b-v1")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("phanerozoic/MistralPirate-7b-v1")
model = AutoModelForCausalLM.from_pretrained("phanerozoic/MistralPirate-7b-v1", device_map="auto")

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use phanerozoic/MistralPirate-7b-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "phanerozoic/MistralPirate-7b-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "phanerozoic/MistralPirate-7b-v1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/phanerozoic/MistralPirate-7b-v1

SGLang

How to use phanerozoic/MistralPirate-7b-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "phanerozoic/MistralPirate-7b-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "phanerozoic/MistralPirate-7b-v1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "phanerozoic/MistralPirate-7b-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "phanerozoic/MistralPirate-7b-v1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use phanerozoic/MistralPirate-7b-v1 with Docker Model Runner:
```
docker model run hf.co/phanerozoic/MistralPirate-7b-v1
```

Overview:

This repository hosts the MistralPirate model, an advanced language model fine-tuned on the Mistral architecture using a pirate-themed dataset. Building on the learnings from the PirateTalk-13b-v1 model, which was based on the Llama 2 Chat model, the MistralPirate represents our efforts to explore the capabilities of the Mistral model for domain-specific tasks.

Objective:

Our primary goal is consistent: to create a model adept at understanding and generating content in the pirate dialect. The MistralPirate initiative aimed to test the adaptability and efficacy of fine-tuning a leading-edge model like Mistral with a niche dialect, pushing the envelope of domain adaptation in neural language models.

Base Model:

For this iteration, we transitioned from the Llama 2 Chat model to the more advanced Mistral architecture. The transition was driven by a strategic decision to leverage the potential advancements inherent in the Mistral framework.

Dataset:

The dataset remains unchanged from our previous venture, maintaining a rich collection of pirate-themed content. This ensures the model's output is grounded in authentic pirate lexicon and idiom.

Performance Insights:

The MistralPirate model exhibits strong adherence to pirate dialect, but it presented some challenges in terms of response length and overall efficacy compared to the Llama 2 Chat iteration. Notably, this project was not just about linguistic performance but also served as an experimental proof-of-concept. Crafting a custom fine-tuning process for Mistral, without the conventional web-UI, was a significant undertaking. Our collaboration with GPT-4 was pivotal in developing the fine-tuning mechanism. A notable technical limitation was our inability to integrate stop tokens effectively, occasionally resulting in more extended responses.

Technical Specifications:

In a departure from our prior methodology, MistralPirate underwent training in full precision as an experimental decision. It's worth noting that the stochastic nature and noise in fp16 training might offer certain advantages in model generalization, which could have impacted our results.

Future Directions:

There are clear avenues for enhancement. The fine-tuning process, dataset improvements, and the integration of stop tokens are among the immediate areas of focus. Our commitment to refining the MistralPirate and similar models remains unwavering.

Acknowledgments:

We extend our gratitude to MistralAI for providing the base model. We also acknowledge all contributors who played a pivotal role in the development and fine-tuning of MistralPirate.

Downloads last month: 13