Instructions to use open-thoughts/OpenThinkerAgent-8B-RL with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use open-thoughts/OpenThinkerAgent-8B-RL with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="open-thoughts/OpenThinkerAgent-8B-RL")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("open-thoughts/OpenThinkerAgent-8B-RL")
model = AutoModelForCausalLM.from_pretrained("open-thoughts/OpenThinkerAgent-8B-RL")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use open-thoughts/OpenThinkerAgent-8B-RL with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "open-thoughts/OpenThinkerAgent-8B-RL"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "open-thoughts/OpenThinkerAgent-8B-RL",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/open-thoughts/OpenThinkerAgent-8B-RL

SGLang

How to use open-thoughts/OpenThinkerAgent-8B-RL with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "open-thoughts/OpenThinkerAgent-8B-RL" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "open-thoughts/OpenThinkerAgent-8B-RL",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "open-thoughts/OpenThinkerAgent-8B-RL" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "open-thoughts/OpenThinkerAgent-8B-RL",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use open-thoughts/OpenThinkerAgent-8B-RL with Docker Model Runner:
```
docker model run hf.co/open-thoughts/OpenThinkerAgent-8B-RL
```

OpenThinkerAgent-8B-RL / swesmith-fixthink-pymethods2test_rl_config.json

penfever

Clone content from laion/rl_swesmith-fixthink-pymethods2test-45

d7167bc verified about 1 month ago

Raw

History Blame Contribute Delete

7.85 kB

	{
	"job_name": "swesmith-fixthink-pymethods2test",
	"experiments_dir": "/pscratch/sd/p/penfever/OpenThoughts-Agent/experiments/swesmith-fixthink-pymethods2test",
	"cluster_name": "perlmutter",
	"skyrl_entrypoint": "examples.terminal_bench.entrypoints.main_tbench",
	"skyrl_hydra_args": [
	"+terminal_bench_config=terminal_bench",
	"trainer.strategy=fsdp2",
	"trainer.algorithm.advantage_estimator=rloo_n",
	"trainer.algorithm.use_kl_loss=false",
	"trainer.algorithm.kl_loss_coef=0.0",
	"trainer.algorithm.eps_clip_low=0.2",
	"trainer.algorithm.eps_clip_high=0.2",
	"trainer.algorithm.loss_reduction=token_mean",
	"trainer.epochs=2",
	"trainer.max_steps=60",
	"trainer.update_epochs_per_batch=1",
	"trainer.train_batch_size=64",
	"trainer.policy_mini_batch_size=64",
	"trainer.eval_batch_size=64",
	"trainer.micro_forward_batch_size_per_gpu=4",
	"trainer.micro_train_batch_size_per_gpu=1",
	"trainer.max_prompt_length=999999",
	"trainer.eval_interval=999999",
	"trainer.eval_before_train=false",
	"trainer.ckpt_interval=999999",
	"trainer.resume_mode=latest",
	"trainer.hf_save_interval=5",
	"++trainer.hf_hub_repo_id=laion/swesmith-fixthink-pymethods2test",
	"++trainer.hf_hub_private=false",
	"++trainer.hf_hub_revision=main",
	"++trainer.enable_db_registration=true",
	"trainer.project_name=OpenThoughts-Agent",
	"trainer.log_level=INFO",
	"trainer.tracker_commit_each_step=true",
	"trainer.run_name=swesmith-fixthink-pymethods2test",
	"trainer.ckpt_path=experiments/swesmith-fixthink-pymethods2test/swesmith-fixthink-pymethods2test/checkpoints",
	"trainer.export_path=experiments/swesmith-fixthink-pymethods2test/swesmith-fixthink-pymethods2test/exports",
	"trainer.policy.optimizer_config.lr=5e-6",
	"trainer.policy.optimizer_config.weight_decay=0.0",
	"trainer.policy.optimizer_config.adam_betas=[0.9,0.999]",
	"trainer.policy.fsdp_config.cpu_offload=true",
	"trainer.policy.fsdp_config.reshard_after_forward=true",
	"trainer.policy.fsdp_config.fsdp_size=4",
	"trainer.policy.model.path=/pscratch/sd/p/penfever/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6",
	"trainer.ref.fsdp_config.cpu_offload=true",
	"trainer.ref.fsdp_config.reshard_after_forward=true",
	"trainer.ref.fsdp_config.fsdp_size=4",
	"trainer.placement.colocate_all=false",
	"trainer.placement.policy_num_nodes=2",
	"trainer.placement.ref_num_nodes=2",
	"trainer.placement.policy_num_gpus_per_node=4",
	"trainer.placement.ref_num_gpus_per_node=4",
	"trainer.fully_async.max_staleness_steps=16",
	"trainer.fully_async.num_parallel_generation_workers=768",
	"generator.backend=vllm",
	"generator.timeout_multiplier=1.0",
	"generator.model_dtype=bfloat16",
	"generator.inference_engine_tensor_parallel_size=1",
	"generator.num_inference_engines=16",
	"generator.n_samples_per_prompt=8",
	"generator.eval_n_samples_per_prompt=8",
	"generator.gpu_memory_utilization=0.9",
	"generator.max_num_seqs=24",
	"generator.max_num_batched_tokens=16384",
	"generator.enable_prefix_caching=true",
	"generator.enable_chunked_prefill=true",
	"generator.run_engines_locally=true",
	"generator.weight_sync_backend=nccl",
	"generator.async_engine=true",
	"generator.batched=false",
	"generator.enable_http_endpoint=true",
	"generator.enable_ray_prometheus_stats=false",
	"generator.vllm_stats_interval=1",
	"generator.append_eos_token_after_stop_str_in_multi_turn=true",
	"generator.max_turns=999999",
	"generator.sampling_params.max_generate_length=4096",
	"generator.sampling_params.temperature=0.7",
	"generator.sampling_params.top_p=0.95",
	"generator.sampling_params.top_k=20",
	"++generator.engine_init_kwargs={max_model_len: 32768, custom_chat_template_chat_completion_path: chat_templates/qwen3_thinking_acc.jinja2, served_model_name: 0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6}",
	"data.train_data=[\"/pscratch/sd/p/penfever/tasks/exp_rpt_pymethods2test-large\"]",
	"data.val_data=[\"/pscratch/sd/p/penfever/tasks/OpenThoughts-TB-dev\"]",
	"+terminal_bench_config.trials_dir=experiments/swesmith-fixthink-pymethods2test/swesmith-fixthink-pymethods2test/trace_jobs",
	"+terminal_bench_config.harbor.name=terminus-2",
	"+terminal_bench_config.harbor.max_episodes=999999",
	"+terminal_bench_config.harbor.enable_summarize=false",
	"+terminal_bench_config.harbor.store_all_messages=true",
	"+terminal_bench_config.harbor.enable_episode_logging=false",
	"+terminal_bench_config.harbor.record_terminal_session=false",
	"+terminal_bench_config.harbor.enable_pane_logging=false",
	"+terminal_bench_config.harbor.strict_json_parser=true",
	"+terminal_bench_config.harbor.interleaved_thinking=true",
	"+terminal_bench_config.harbor.extra_body.chat_template_kwargs={enable_thinking: true}",
	"+terminal_bench_config.harbor.override_timeout_sec=1800",
	"+terminal_bench_config.harbor.override_cpus=1",
	"+terminal_bench_config.harbor.override_memory_mb=2048",
	"+terminal_bench_config.harbor.override_storage_mb=2048",
	"+terminal_bench_config.harbor.auto_snapshot=true",
	"+terminal_bench_config.harbor.verifier_override_timeout_sec=120",
	"+terminal_bench_config.harbor.max_retries=3",
	"+terminal_bench_config.harbor.min_wait_sec=60.0",
	"+terminal_bench_config.harbor.max_wait_sec=600.0",
	"+terminal_bench_config.harbor.wait_multiplier=2.0",
	"+terminal_bench_config.harbor.exclude_exceptions=[\"AgentTimeoutError\",\"VerifierTimeoutError\",\"RewardFileNotFoundError\",\"RewardFileEmptyError\",\"VerifierOutputParseError\",\"ContextLengthExceededError\"]",
	"+terminal_bench_config.harbor.n_concurrent_trials=280",
	"+terminal_bench_config.harbor.log_level=INFO",
	"+terminal_bench_config.harbor.enable_reward_shaping=false",
	"+terminal_bench_config.harbor.enable_error_classification=true",
	"+terminal_bench_config.harbor.mask_exceptions=[\"DaytonaError\",\"EnvironmentStartTimeoutError\",\"NetworkError\",\"ConnectionError\",\"RewardFileNotFoundError\",\"RewardFileEmptyError\",\"AgentEnvironmentTimeoutError\",\"AgentTimeoutError\",\"ContextLengthExceededError\"]",
	"+terminal_bench_config.harbor.default_error_treatment=zero",
	"+terminal_bench_config.model_info.max_input_tokens=32768",
	"+terminal_bench_config.model_info.max_output_tokens=4096",
	"+terminal_bench_config.archiving.enabled=false",
	"+terminal_bench_config.trace_upload.enabled=true",
	"+terminal_bench_config.trace_upload.repo_org=DCAgent",
	"+terminal_bench_config.trace_upload.episodes=last",
	"+terminal_bench_config.trace_upload.dataset_type=SFT"
	],
	"model_path": "/pscratch/sd/p/penfever/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6",
	"train_data": [
	"/pscratch/sd/p/penfever/tasks/exp_rpt_pymethods2test-large"
	],
	"val_data": [
	"/pscratch/sd/p/penfever/tasks/OpenThoughts-TB-dev"
	],
	"num_nodes": 6,
	"gpus_per_node": 4,
	"cpus_per_node": 64,
	"tensor_parallel_size": 1,
	"ray_port": 6379,
	"master_port": 12345,
	"checkpoints_dir": null,
	"export_path": "/pscratch/sd/p/penfever/OpenThoughts-Agent/experiments/swesmith-fixthink-pymethods2test/swesmith-fixthink-pymethods2test/exports",
	"needs_ssh_tunnel": false,
	"needs_cuda_detection": true,
	"pinggy_persistent_url": null,
	"pinggy_token": null,
	"agent_name": "terminus-2",
	"harbor_env": "daytona",
	"proxychains_binary": null,
	"trace_upload_enabled": true,
	"trace_upload_repo_org": "DCAgent",
	"trace_upload_episodes": "last",
	"trace_upload_dataset_type": "SFT"
	}