Text Generation
Transformers
PyTorch
TensorFlow
Safetensors
t5
text2text-generation
generated_from_keras_callback
text-generation-inference
Instructions to use readerbench/QAll-Flan-xl with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use readerbench/QAll-Flan-xl with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="readerbench/QAll-Flan-xl")# Load model directly from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("readerbench/QAll-Flan-xl") model = AutoModelForSeq2SeqLM.from_pretrained("readerbench/QAll-Flan-xl") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use readerbench/QAll-Flan-xl with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "readerbench/QAll-Flan-xl" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "readerbench/QAll-Flan-xl", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/readerbench/QAll-Flan-xl
- SGLang
How to use readerbench/QAll-Flan-xl with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "readerbench/QAll-Flan-xl" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "readerbench/QAll-Flan-xl", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "readerbench/QAll-Flan-xl" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "readerbench/QAll-Flan-xl", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use readerbench/QAll-Flan-xl with Docker Model Runner:
docker model run hf.co/readerbench/QAll-Flan-xl
Commit ·
5bc622b
1
Parent(s): 052539a
Update README.md
Browse files
README.md
CHANGED
|
@@ -11,6 +11,11 @@ widget:
|
|
| 11 |
example_title: Question Answering
|
| 12 |
- text: "Generate a question based on the context and the answer.\nContext: The Review of Politics was founded in 1939 by Gurian, modeled after German Catholic journals. It quickly emerged as part of an international Catholic intellectual revival, offering an alternative vision to positivist philosophy. For 44 years, the Review was edited by Gurian, Matthew Fitzsimons, Frederick Crosson, and Thomas Stritch. Intellectual leaders included Gurian, Jacques Maritain, Frank O'Malley, Leo Richard Ward, F. A. Hermens, and John U. Nef. It became a major forum for political ideas and modern political concerns, especially from a Catholic and scholastic tradition.\nAnswer: Thomas Stritch"
|
| 13 |
example_title: Question Generation
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
---
|
| 15 |
|
| 16 |
<!-- This model card has been generated automatically according to the information Keras had access to. You should
|
|
|
|
| 11 |
example_title: Question Answering
|
| 12 |
- text: "Generate a question based on the context and the answer.\nContext: The Review of Politics was founded in 1939 by Gurian, modeled after German Catholic journals. It quickly emerged as part of an international Catholic intellectual revival, offering an alternative vision to positivist philosophy. For 44 years, the Review was edited by Gurian, Matthew Fitzsimons, Frederick Crosson, and Thomas Stritch. Intellectual leaders included Gurian, Jacques Maritain, Frank O'Malley, Leo Richard Ward, F. A. Hermens, and John U. Nef. It became a major forum for political ideas and modern political concerns, especially from a Catholic and scholastic tradition.\nAnswer: Thomas Stritch"
|
| 13 |
example_title: Question Generation
|
| 14 |
+
- text: "Classify the complexity of the following question based on the text as either deep or shallow.\nText: Many people who live in California are very concerned about the earth. They are worried that the earth is warming too much.\nQuestion: How can people in California and outside of California prevent warming?"
|
| 15 |
+
example_title: Question Complexity
|
| 16 |
+
- text: "Classify the complexity of the following question based on the text as either deep or shallow.\nText: Many people who live in California are very concerned about the earth. They are worried that the earth is warming too much.\nQuestion: What does it mean to warm the earth too much?"
|
| 17 |
+
example_title: Question Category
|
| 18 |
+
|
| 19 |
---
|
| 20 |
|
| 21 |
<!-- This model card has been generated automatically according to the information Keras had access to. You should
|