Instructions to use Jay1121/solar-chatbot-final with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Jay1121/solar-chatbot-final with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("Upstage/SOLAR-10.7B-Instruct-v1.0")
model = PeftModel.from_pretrained(base_model, "Jay1121/solar-chatbot-final")

Transformers

How to use Jay1121/solar-chatbot-final with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Jay1121/solar-chatbot-final")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForMultimodalLM

tokenizer = AutoTokenizer.from_pretrained("Jay1121/solar-chatbot-final")
model = AutoModelForMultimodalLM.from_pretrained("Jay1121/solar-chatbot-final")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Jay1121/solar-chatbot-final with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Jay1121/solar-chatbot-final"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jay1121/solar-chatbot-final",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Jay1121/solar-chatbot-final

SGLang

How to use Jay1121/solar-chatbot-final with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Jay1121/solar-chatbot-final" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jay1121/solar-chatbot-final",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Jay1121/solar-chatbot-final" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jay1121/solar-chatbot-final",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Jay1121/solar-chatbot-final with Docker Model Runner:
```
docker model run hf.co/Jay1121/solar-chatbot-final
```

solar-chatbot-final (LoRA Adapter)

Summary: A Korean conversational LoRA adapter fine-tuned on Upstage's SOLAR-10.7B-Instruct-v1.0
Designed for natural, responsive dialogue with capabilities in general Q&A, summarization, and text generation.

Model Details

Developed by: Jihee Cho
Model type: Causal Language Model (chat/instruction-tuned), LoRA adapter for SOLAR-10.7B
Languages: Korean (primary), English (basic support)
License: Apache-2.0
Finetuned from: Upstage/SOLAR-10.7B-Instruct-v1.0

Model Sources

Base Model: https://huggingface.co/Upstage/SOLAR-10.7B-Instruct-v1.0
Repository: https://huggingface.co/Jay1121/solar-chatbot-final

Quick Start

Requirements

pip install transformers peft torch

Usage Example

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load base model and adapter
base_model_name = "Upstage/SOLAR-10.7B-Instruct-v1.0"
adapter_name = "Jay1121/solar-chatbot-final"

tokenizer = AutoTokenizer.from_pretrained(base_model_name, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(
    base_model_name, 
    torch_dtype="auto", 
    device_map="auto"
)
model = PeftModel.from_pretrained(model, adapter_name)

# Generate response
prompt = "안녕! 오늘 뭐 하지?"  # "Hi! What should I do today?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs, 
    max_new_tokens=256, 
    do_sample=True, 
    temperature=0.8,
    pad_token_id=tokenizer.eos_token_id
)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Intended Use

Primary Use Cases

Casual Korean conversation and chatbot demos
General question-answering systems
Text summarization and draft writing
Prototyping and educational purposes

Out-of-Scope Use

Professional advice in medical, legal, or financial domains
Fact-checking or authoritative information provision
Production-level commercial services
Processing personal or sensitive data

Limitations and Risks

Known Limitations

No factual accuracy guarantee: The model cannot ensure the correctness of generated information
Hallucination: May generate false or non-existent content
Bias: Training data biases may be reflected in outputs
Consistency issues: May struggle to maintain consistency in long conversations

Ethical Considerations

Users should critically review all generated content
Always consult reliable sources for important decisions
Do not blindly trust model outputs; verification is essential

Technical Details

Training Details

Fine-tuning Method: LoRA (Low-Rank Adaptation)
Target Modules: Query, Key, Value projection layers
Rank: 16
Alpha: 32
Training Data: Korean conversation datasets
Training Framework: Transformers + PEFT + TRL

Performance

This model is optimized for natural conversational experiences rather than benchmark performance, focusing on engaging and contextually appropriate Korean dialogue.

Disclaimer and Usage Terms

No Impersonation Policy

This project is not affiliated with or endorsed by any person or organization. Do not use this model to impersonate specific individuals or create systems that could mislead others about official endorsements.

Branding and Attribution

When using this model, please provide appropriate attribution and avoid using it in ways that could be mistaken for official statements from the original author or related organizations.

Citation

If you use this model in your research or projects, please cite:

@misc{solar-chatbot-final,
  author = {Jihee Cho},
  title = {solar-chatbot-final: Korean Conversational LoRA Adapter for SOLAR-10.7B},
  year = {2024},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Jay1121/solar-chatbot-final}}
}

Contact

For questions or feedback, please use the Discussion tab on the Hugging Face model page.

Downloads last month: 1

Safetensors

Model size

11B params

Tensor type

F32

BF16