Instructions to use jtatman/tinymistral-v2-pycoder-instruct-248m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jtatman/tinymistral-v2-pycoder-instruct-248m with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="jtatman/tinymistral-v2-pycoder-instruct-248m")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("jtatman/tinymistral-v2-pycoder-instruct-248m")
model = AutoModelForCausalLM.from_pretrained("jtatman/tinymistral-v2-pycoder-instruct-248m")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use jtatman/tinymistral-v2-pycoder-instruct-248m with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jtatman/tinymistral-v2-pycoder-instruct-248m"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jtatman/tinymistral-v2-pycoder-instruct-248m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/jtatman/tinymistral-v2-pycoder-instruct-248m

SGLang

How to use jtatman/tinymistral-v2-pycoder-instruct-248m with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jtatman/tinymistral-v2-pycoder-instruct-248m" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jtatman/tinymistral-v2-pycoder-instruct-248m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jtatman/tinymistral-v2-pycoder-instruct-248m" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jtatman/tinymistral-v2-pycoder-instruct-248m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use jtatman/tinymistral-v2-pycoder-instruct-248m with Docker Model Runner:
```
docker model run hf.co/jtatman/tinymistral-v2-pycoder-instruct-248m
```

Model Card for tinymistral-v2-pycoder-instruct-248m

This modelcard is for tinymistral-v2-pycoder-instruct, a python-specific code generation model on top of Locutusque/TinyMistral-248M-v2-Instruct.

Model Details

This instruct model follows the original in using ChatML format.

An empty prompt will return various information from the base model, but using the instruct format will deliver python code of varying quality.

Model Description

Model is in active development, base model is in active development, and all should be treated with caution.

Developed by: [Locutusque and M4ai]
Funded by: [Lint from a corner pocket]
Shared by: jtatman
Model type: MistralForCausalLM
License: [MIT]
**Finetuned from model Locutusque/TinyMistral-248M-v2

Uses

Generate python code.

Direct Use

Probably could be fine tuned with a more comprehensive dataset. Experiments are in progress.

How to Get Started with the Model

Use the prompt format below to get started with the model.

Training Details

Training Data

Custom formatted existing python data from:

Training Procedure

Repeat training depending on compute budget.

Preprocessing

Conversion to alpaca/instruct format.

Training Hyperparameters

Training regime: fp16, merge of parameter fine-tune adapters when necessary and helpful.

Evaluation

Metrics

Latest metrics:

epoch: 4.87
global_step: 220
learning_rate: 0.00006713780918727916
loss: 2.3736

Downloads last month: 14

Safetensors

Model size

0.2B params

Tensor type

F32

Model tree for jtatman/tinymistral-v2-pycoder-instruct-248m

Merges

1 model

jtatman
/

tinymistral-v2-pycoder-instruct-248m