Instructions to use imda-lseokmin/testfinetunedmodel with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use imda-lseokmin/testfinetunedmodel with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="imda-lseokmin/testfinetunedmodel")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("imda-lseokmin/testfinetunedmodel")
model = AutoModelForCausalLM.from_pretrained("imda-lseokmin/testfinetunedmodel")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use imda-lseokmin/testfinetunedmodel with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "imda-lseokmin/testfinetunedmodel"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "imda-lseokmin/testfinetunedmodel",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/imda-lseokmin/testfinetunedmodel

SGLang

How to use imda-lseokmin/testfinetunedmodel with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "imda-lseokmin/testfinetunedmodel" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "imda-lseokmin/testfinetunedmodel",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "imda-lseokmin/testfinetunedmodel" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "imda-lseokmin/testfinetunedmodel",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use imda-lseokmin/testfinetunedmodel with Docker Model Runner:
```
docker model run hf.co/imda-lseokmin/testfinetunedmodel
```

testfinetunedmodel

Commit History

Adding Evaluation Results (#1)

865f3fb
verified

leaderboard-pr-bot commited on Apr 17, 2024

Upload 66 files

9efeae0

SM commited on Dec 27, 2023

With better accuracy

c3eb26d

SM commited on Dec 27, 2023

Retrain with the proper data file.

90db77c

SM commited on Dec 27, 2023

Upload 14 files

ba705df

SM commited on Dec 26, 2023

initial commit

8e4ca04

imda-lseokmin commited on Dec 26, 2023

Commit History

Adding Evaluation Results (#1) 865f3fb verified

Upload 66 files 9efeae0

With better accuracy c3eb26d

Retrain with the proper data file. 90db77c

Upload 14 files ba705df

initial commit 8e4ca04

Adding Evaluation Results (#1)

865f3fb
verified

Upload 66 files

9efeae0

With better accuracy

c3eb26d

Retrain with the proper data file.

90db77c

Upload 14 files

ba705df

initial commit

8e4ca04