Instructions to use MSey/tiny_CaLL_r1_O1_f1_LT_checkpoint-511 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use MSey/tiny_CaLL_r1_O1_f1_LT_checkpoint-511 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="MSey/tiny_CaLL_r1_O1_f1_LT_checkpoint-511")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("MSey/tiny_CaLL_r1_O1_f1_LT_checkpoint-511")
model = AutoModelForCausalLM.from_pretrained("MSey/tiny_CaLL_r1_O1_f1_LT_checkpoint-511")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use MSey/tiny_CaLL_r1_O1_f1_LT_checkpoint-511 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "MSey/tiny_CaLL_r1_O1_f1_LT_checkpoint-511"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MSey/tiny_CaLL_r1_O1_f1_LT_checkpoint-511",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/MSey/tiny_CaLL_r1_O1_f1_LT_checkpoint-511

SGLang

How to use MSey/tiny_CaLL_r1_O1_f1_LT_checkpoint-511 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "MSey/tiny_CaLL_r1_O1_f1_LT_checkpoint-511" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MSey/tiny_CaLL_r1_O1_f1_LT_checkpoint-511",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "MSey/tiny_CaLL_r1_O1_f1_LT_checkpoint-511" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MSey/tiny_CaLL_r1_O1_f1_LT_checkpoint-511",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use MSey/tiny_CaLL_r1_O1_f1_LT_checkpoint-511 with Docker Model Runner:
```
docker model run hf.co/MSey/tiny_CaLL_r1_O1_f1_LT_checkpoint-511
```

Model Card for Model ID

We fine-tuned our base model for 21 epochs on the Ca dataset, epoch 1 showed the best macro average f1 score on the evaluation dataset.

Context format

"### Context\n\nText to analyse.\n\n###Answer"

Metric

eval_AVGf1 0.9102075019834961

eval_DIAGNOSIS.f1 0.8808602150537634

eval_DIAGNOSIS.precision 0.8943231441048035

eval_DIAGNOSIS.recall 0.8677966101694915

eval_DIAGNOSTIC.f1 0.9472166137871358

eval_DIAGNOSTIC.precision 0.9624853458382181

eval_DIAGNOSTIC.recall 0.9324247586598523

eval_DRUG.f1 0.9440145653163405

eval_DRUG.precision 0.9792256846081209

eval_DRUG.recall 0.9112478031634447

eval_MEDICAL_FINDING.f1 0.9092427259297321

eval_MEDICAL_FINDING.precision 0.9073195744135367

eval_MEDICAL_FINDING.recall 0.9111740473738414

eval_THERAPY.f1 0.8697033898305084

eval_THERAPY.precision 0.8729399255715046

eval_THERAPY.recall 0.8664907651715039

eval_accuracy 0.9618960382191458

eval_f1 0.7632318301785055

eval_loss 0.006697072647511959

eval_model_preparation_time 0

eval_precision 0.6619246861924686

eval_recall 0.9011526605012733

eval_runtime 341.5967

eval_samples_per_second 23.952

eval_steps_per_second 5.99

test_AVGf1 0.8676664044743045

test_DIAGNOSIS.f1 0.7754658946987515

test_DIAGNOSIS.precision 0.7846942511900403

test_DIAGNOSIS.recall 0.7664520743919886

test_DIAGNOSTIC.f1 0.9211950129381322

test_DIAGNOSTIC.precision 0.9346062052505967

test_DIAGNOSTIC.recall 0.9081632653061225

test_DRUG.f1 0.9448028673835126

test_DRUG.precision 0.9835820895522388

test_DRUG.recall 0.9089655172413793

test_MEDICAL_FINDING.f1 0.879590997238056

test_MEDICAL_FINDING.precision 0.8656025907934305

test_MEDICAL_FINDING.recall 0.8940389439732409

test_THERAPY.f1 0.8172772501130711

test_THERAPY.precision 0.8187584956955143

test_THERAPY.recall 0.8158013544018059

test_accuracy 0.9665184459433998

test_f1 0.7391588362393848

test_loss 0.009836438111960888

test_model_preparation_time 0

test_precision 0.6447795213465416

test_recall 0.865905344949376

test_runtime 394.9961

test_samples_per_second 24.023

test_steps_per_second 6.008

Downloads last month: 5

Safetensors

Model size

1B params

Tensor type

F32

Model tree for MSey/tiny_CaLL_r1_O1_f1_LT_checkpoint-511

Base model

TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T

Finetuned

(105)

this model

Collection including MSey/tiny_CaLL_r1_O1_f1_LT_checkpoint-511

Ca-Test

Collection

6 items • Updated Mar 2