INSTALLATION_GUIDE.md

luwa-01 Installation & Deployment Guide

Get luwa-01 Running in 5 Minutes

This guide shows you how to set up luwa-01 on your machine, server, or cloud — no GPU required.

---

Requirements

Requirement	Minimum
RAM	4 GB
Disk space	1 GB (model is 942 MB)
Python	3.8 or higher
GPU	Not required (works on CPU)
Internet	Required for initial download only

---

Method 1: Quick Start (Python)

The fastest way to get luwa-01 running:

pip install transformers torch

Then create a file called chat.py:

from transformers import AutoTokenizer, AutoModelForCausalLM

# Load the model (downloads automatically on first run)
model = AutoModelForCausalLM.from_pretrained(
    "chatpbc1/luwa-01",
    trust_remote_code=True,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("chatpbc1/luwa-01")

# Your question
question = "What is the market size for AI in healthcare in 2026?"

# Format the message
messages = [{"role": "user", "content": question}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

# Generate response
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

print(response)

Run it:

python chat.py

---

Method 2: Interactive Chat

Create a simple chat loop so you can have a conversation with luwa-01:

from transformers import AutoTokenizer, AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("chatpbc1/luwa-01", trust_remote_code=True, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("chatpbc1/luwa-01")

print("luwa-01 Business Intelligence Agent")
print("Type your question (or 'quit' to exit)\n")

messages = []
while True:
    user_input = input("You: ")
    if user_input.lower() == "quit":
        break
    messages.append({"role": "user", "content": user_input})
    
    text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    inputs = tokenizer(text, return_tensors="pt").to(model.device)
    outputs = model.generate(**inputs, max_new_tokens=512)
    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
    
    print(f"\nluwa-01: {response}\n")
    messages.append({"role": "assistant", "content": response})

---

Method 3: Deploy as a Web API

Turn luwa-01 into a REST API server that your apps can call:

pip install transformers torch fastapi uvicorn

Create server.py:

from fastapi import FastAPI
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

app = FastAPI()

# Load model once at startup
model = AutoModelForCausalLM.from_pretrained("chatpbc1/luwa-01", trust_remote_code=True, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("chatpbc1/luwa-01")

@app.post("/chat")
async def chat(request: dict):
    prompt = request.get("prompt", "")
    max_tokens = request.get("max_tokens", 512)
    
    messages = [{"role": "user", "content": prompt}]
    text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    inputs = tokenizer(text, return_tensors="pt").to(model.device)
    
    with torch.no_grad():
        outputs = model.generate(**inputs, max_new_tokens=max_tokens)
    
    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
    return {"response": response}

Start the server:

uvicorn server:app --host 0.0.0.0 --port 8000

Then call it:

curl -X POST http://localhost:8000/chat -H "Content-Type: application/json" -d '{"prompt": "Analyze the AI market", "max_tokens": 256}'

---

Method 4: Deploy on Modal (Cloud GPU)

For production with automatic scaling:

Step 1: Sign up at modal.com

Step 2: Install Modal:

pip install modal
modal token new

Step 3: Create your HF secret:

modal secret create hf-token HF_TOKEN=YOUR_HF_TOKEN

Step 4: Deploy:

modal deploy deploy_luwa.py

You'll get a URL like:

https://your-username--luwa-01-service.modal.run

Step 5: Use it from anywhere:

curl -X POST https://your-username--luwa-01-service.modal.run -H "Content-Type: application/json" -d '{"prompt": "Market analysis request", "max_tokens": 512}'

---

Method 5: Deploy with Docker

For containerized production:

FROM python:3.11-slim

RUN pip install transformers torch fastapi uvicorn

WORKDIR /app
COPY server.py .

EXPOSE 8000
CMD ["uvicorn", "server:app", "--host", "0.0.0.0", "--port", "8000"]

docker build -t luwa-01 .
docker run -p 8000:8000 luwa-01

---

Performance Tips

Scenario	Recommendation
Development/Testing	Run directly on CPU (4GB RAM)
Production API	Use Modal or any cloud GPU (T4)
High traffic	Deploy with Docker + load balancer
Edge deployment	Use ONNX export for even faster inference

---

Troubleshooting

"CUDA out of memory"

• Switch to CPU: device_map="cpu"
• Reduce max_new_tokens to 256

"Model not found"

• Check your internet connection
• Ensure you have transformers >= 4.30.0

"Slow responses"

• Use a GPU if available
• Reduce max_new_tokens
• Set temperature=0.5 for faster deterministic output

---

What's Included in the Repository

File	Purpose
model.safetensors	Model weights (942 MB)
config.json	Architecture settings
generation_config.json	Optimized generation parameters
tokenizer.json	Text tokenizer (152K vocabulary)
tokenizer_config.json	Tokenizer settings
chat_template.jinja	Chat formatting template
system_prompt.txt	Business intelligence persona
agent_config.json	Agent tool definitions

---

Next Steps

• Read the Agent Guide for how to use luwa-01 effectively
• Visit the repository for the latest updates
• Join the ChatPBC community for support

---

Built by ChatPBC