Instructions to use midorin-Linux/plamo-3-12b-self-merged-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use midorin-Linux/plamo-3-12b-self-merged-base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="midorin-Linux/plamo-3-12b-self-merged-base", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("midorin-Linux/plamo-3-12b-self-merged-base", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use midorin-Linux/plamo-3-12b-self-merged-base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "midorin-Linux/plamo-3-12b-self-merged-base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "midorin-Linux/plamo-3-12b-self-merged-base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/midorin-Linux/plamo-3-12b-self-merged-base

SGLang

How to use midorin-Linux/plamo-3-12b-self-merged-base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "midorin-Linux/plamo-3-12b-self-merged-base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "midorin-Linux/plamo-3-12b-self-merged-base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "midorin-Linux/plamo-3-12b-self-merged-base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "midorin-Linux/plamo-3-12b-self-merged-base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use midorin-Linux/plamo-3-12b-self-merged-base with Docker Model Runner:
```
docker model run hf.co/midorin-Linux/plamo-3-12b-self-merged-base
```

PLaMo 3 12B Self-Merged Base

Important!: Currently, this model has not undergone CPT (Continued Pre-Training) following the merge; it will be replaced later with a CPT-processed version.

PLaMo 3 12B Self-Merged Base is a model created by self-merging pfnet/plamo-3-nict-8b-base and scaling it up to 12B parameters.

PLaMo 3 12B Self-Merged Base currently has some unstable elements, which we plan to address in an upcoming update.

PLaMo 3 12B Self-Merged Base is released under the PLaMo Community License. Please review the license below and agree to its terms before downloading.

NOTE: This model has NOT been instruction-tuned for chat dialogue or other downstream tasks.

This is a merge of pre-trained language models created using mergekit.

For commercial use

Please review the PLaMo Community License and contact us using the form below if you intend to use this model for commercial purposes.

(EN/JA) https://forms.gle/mTL8tBLrMYXKNZD56

Usage

Requirements

python>=3.10.0
numpy>=1.26.4
numba>=0.60.0
torch>=2.6.0,<=2.9.0
transformers>=4.56.0,<=4.57.1

Loading the model directly

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "midorin-Linux/plamo-3-12b-self-merged"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)

text = "これからの人工知能技術は"
inputs = tokenizer(text, return_tensors="pt").to(model.device)

generated_tokens = model.generate(
    **inputs,
    max_new_tokens=32,
    do_sample=True,
    top_k=50,
    top_p=0.95,
    temperature=1.0,
    pad_token_id=tokenizer.eos_token_id,
    use_cache=True,
)[0]

generated_text = tokenizer.decode(generated_tokens, skip_special_tokens=True)
print(generated_text)

Model Details

Model size: 12B (8B + 8B/2)
Base model (pfnet/plamo-3-nict-8b-base) developed by:
- Preferred Networks, Inc.
- In collaboration with the National Institute of Information and Communications Technology (NICT)
Model type: Causal decoder-only
Language(s): English, Japanese
License: PLaMo Community License

Base Model Information

For details on the base model, please refer to pfnet/plamo-3-nict-8b-base.

Built with PLaMo

Downloads last month: 138

Safetensors

Model size

12B params

Tensor type

BF16

Model tree for midorin-Linux/plamo-3-12b-self-merged-base

Base model

pfnet/plamo-3-nict-8b-base

Finetuned

(2)

this model

Collection including midorin-Linux/plamo-3-12b-self-merged-base

PLaMo 3

Collection

Created for personal research using the PLaMo series. • 2 items • Updated 20 days ago