Instructions to use luke-thorburn/suggest-objections-soft with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use luke-thorburn/suggest-objections-soft with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="luke-thorburn/suggest-objections-soft")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("luke-thorburn/suggest-objections-soft")
model = AutoModelForCausalLM.from_pretrained("luke-thorburn/suggest-objections-soft")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use luke-thorburn/suggest-objections-soft with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "luke-thorburn/suggest-objections-soft"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "luke-thorburn/suggest-objections-soft",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/luke-thorburn/suggest-objections-soft

SGLang

How to use luke-thorburn/suggest-objections-soft with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "luke-thorburn/suggest-objections-soft" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "luke-thorburn/suggest-objections-soft",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "luke-thorburn/suggest-objections-soft" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "luke-thorburn/suggest-objections-soft",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use luke-thorburn/suggest-objections-soft with Docker Model Runner:
```
docker model run hf.co/luke-thorburn/suggest-objections-soft
```

Generate objections to a claim

This model has the same model parameters as gpt-neo-2.7B, but with an additional soft prompt which has been optimized on the task of generating the objections to a claim, optionally given some example objections to that claim. It was trained as part of a University of Melbourne research project evaluating how large language models can best be optimized to perform argumentative reasoning tasks.

Code used for optimization and evaluation can be found in the project GitHub repository. A paper reporting on model evaluation is currently under review.

Prompt Template

[prepended soft prompt][original claim]

Cons:
- [objection 1]
- [objection 2]
...
- [objection n]
- [generated objection]

Dataset

The soft prompt was trained using argument maps scraped from the crowdsourced argument-mapping platform Kialo.

Limitations and Biases

The model is a finetuned version of gpt-neo-2.7B, so likely has many of the same limitations and biases. Additionally, note that while the goal of the model is to produce coherent and valid reasoning, many generated model outputs will be illogical or nonsensical and should not be relied upon.

Acknowledgements

This research was funded by the Australian Department of Defence and the Office of National Intelligence under the AI for Decision Making Program, delivered in partnership with the Defence Science Institute in Victoria, Australia.

Downloads last month: 3