Instructions to use lmqg/bart-large-tweetqa-qag with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lmqg/bart-large-tweetqa-qag with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="lmqg/bart-large-tweetqa-qag")

# Load model directly
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("lmqg/bart-large-tweetqa-qag")
model = AutoModelForSeq2SeqLM.from_pretrained("lmqg/bart-large-tweetqa-qag")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use lmqg/bart-large-tweetqa-qag with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lmqg/bart-large-tweetqa-qag"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lmqg/bart-large-tweetqa-qag",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/lmqg/bart-large-tweetqa-qag

SGLang

How to use lmqg/bart-large-tweetqa-qag with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lmqg/bart-large-tweetqa-qag" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lmqg/bart-large-tweetqa-qag",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lmqg/bart-large-tweetqa-qag" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lmqg/bart-large-tweetqa-qag",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use lmqg/bart-large-tweetqa-qag with Docker Model Runner:
```
docker model run hf.co/lmqg/bart-large-tweetqa-qag
```

asahi417 commited on Nov 24, 2022

Commit

2bfc8de

1 Parent(s): 35ed466

add model

Browse files

Files changed (2) hide show

eval/metric.first.answer.paragraph.questions_answers.lmqg_qag_tweetqa.default.json +1 -1
pytorch_model.bin +2 -2

eval/metric.first.answer.paragraph.questions_answers.lmqg_qag_tweetqa.default.json CHANGED Viewed

@@ -1 +1 @@

- {"validation": {"QAAlignedF1Score (MoverScore)": 0.645408905658979, "QAAlignedF1Score (BERTScore)": 0.9172234166447675, "MoverScore": 0.6290265297078133, "BERTScore": 0.9110018365583534, "ROUGE_L": 0.37165690881702945, "METEOR": 0.3154636133794367, "Bleu_1": 0.43188759926692244, "Bleu_2": 0.30011779847872594, "Bleu_3": 0.20914603335217866, "Bleu_4": 0.14555259594468786}, "test": {"QAAlignedF1Score (MoverScore)": 0.6465994118897864, "QAAlignedF1Score (BERTScore)": 0.9246851189868012, "MoverScore": 0.62254961921224, "BERTScore": 0.9126712930345617, "ROUGE_L": 0.34985377392591416, "METEOR": 0.2790788570524155, "Bleu_1": 0.44545454545451546, "Bleu_2": 0.3115127823844136, "Bleu_3": 0.21584316619841207, "Bleu_4": 0.15175643909660202}}

+ {"validation": {"QAAlignedF1Score (MoverScore)": 0.645408905658979, "QAAlignedF1Score (BERTScore)": 0.9172234166447675, "MoverScore": 0.6290265297078133, "BERTScore": 0.9110018365583534, "ROUGE_L": 0.37165690881702945, "METEOR": 0.3154636133794367, "Bleu_1": 0.43188759926692244, "Bleu_2": 0.30011779847872594, "Bleu_3": 0.20914603335217866, "Bleu_4": 0.14555259594468786, "QAAlignedRecall (BERTScore)": 0.9157387038140109, "QAAlignedPrecision (BERTScore)": 0.9188391534582828, "QAAlignedRecall (MoverScore)": 0.6406859290174065, "QAAlignedPrecision (MoverScore)": 0.6509416186712476}, "test": {"QAAlignedF1Score (MoverScore)": 0.6465994118897864, "QAAlignedF1Score (BERTScore)": 0.9246851189868012, "MoverScore": 0.62254961921224, "BERTScore": 0.9126712930345617, "ROUGE_L": 0.34985377392591416, "METEOR": 0.2790788570524155, "Bleu_1": 0.44545454545451546, "Bleu_2": 0.3115127823844136, "Bleu_3": 0.21584316619841207, "Bleu_4": 0.15175643909660202, "QAAlignedRecall (BERTScore)": 0.9221427509841271, "QAAlignedPrecision (BERTScore)": 0.9273816504057235, "QAAlignedRecall (MoverScore)": 0.6403168610145709, "QAAlignedPrecision (MoverScore)": 0.6538693732277723}}

pytorch_model.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:855c8b431a9bf3463503050051839b61bcd0770a89f9ee6d930a281fb7e5e17d
-size 1625534721

 version https://git-lfs.github.com/spec/v1
+oid sha256:ee3751a58d92217290411059f9294970dca2ea5aa03917d252005c90fa42db2e
+size 1625538317