Instructions to use fairdataihub/Llama-3.1-8B-Poster-Extraction with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use fairdataihub/Llama-3.1-8B-Poster-Extraction with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="fairdataihub/Llama-3.1-8B-Poster-Extraction")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("fairdataihub/Llama-3.1-8B-Poster-Extraction")
model = AutoModelForCausalLM.from_pretrained("fairdataihub/Llama-3.1-8B-Poster-Extraction")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use fairdataihub/Llama-3.1-8B-Poster-Extraction with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "fairdataihub/Llama-3.1-8B-Poster-Extraction"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fairdataihub/Llama-3.1-8B-Poster-Extraction",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/fairdataihub/Llama-3.1-8B-Poster-Extraction

SGLang

How to use fairdataihub/Llama-3.1-8B-Poster-Extraction with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "fairdataihub/Llama-3.1-8B-Poster-Extraction" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fairdataihub/Llama-3.1-8B-Poster-Extraction",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "fairdataihub/Llama-3.1-8B-Poster-Extraction" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fairdataihub/Llama-3.1-8B-Poster-Extraction",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use fairdataihub/Llama-3.1-8B-Poster-Extraction with Docker Model Runner:
```
docker model run hf.co/fairdataihub/Llama-3.1-8B-Poster-Extraction
```

Llama-3.1-8B-Poster-Extraction

Commit History

Update README.md

425fc29
verified

jimnoneill commited on Mar 3

Update README.md

7fee7da
verified

jimnoneill commited on Feb 9

Update model card with accurate schema, FAIR Data Innovations Hub, and posters.science context

1fe7e5e

Jamey O'Neill commited on Feb 7

Add poster2json library links (PyPI, docs, GitHub)

1f6dce7

Jamey O'Neill commited on Feb 7

Upload Llama 3.1 8B for Machine Actionable Poster Extraction

b90ce33
verified

jimnoneill commited on Jan 7

initial commit

6f0baa9
verified

jimnoneill commited on Jan 7

Commit History

Update README.md 425fc29 verified

Update README.md 7fee7da verified

Update model card with accurate schema, FAIR Data Innovations Hub, and posters.science context 1fe7e5e

Add poster2json library links (PyPI, docs, GitHub) 1f6dce7

Upload Llama 3.1 8B for Machine Actionable Poster Extraction b90ce33 verified