ChatPBC_Deployment_Checklist.md

ChatPBC Model Deployment Checklist

Developed by Mik Tse Agency

---

PRE-DEPLOYMENT REQUIREMENTS

1. Model Repo Structure (REQUIRED for HF Inference API to work):

   • config.json with model_type field present
   • model.safetensors OR pytorch_model.bin (actual weight files, NOT empty placeholders)
   • tokenizer_config.json
   • tokenizer.model OR tokenizer.json
   • special_tokens_map.json
   • generation_config.json
   • README.md with pipeline_tag: text-generation in YAML front matter

2. If using LoRA adapters:

   • adapter_config.json with base_model_name_or_path pointing to a real, accessible HF model
   • adapter_model.safetensors with actual trained weights (NOT 0.1 KB placeholder)
   • The base model must be publicly accessible on HF
   • Use merge_and_unload() to create a standalone model for Inference API compatibility

3. HF Inference API Requirements:

   • Model repo is PUBLIC (not private)
   • pipeline_tag: text-generation set in model card YAML
   • library_name: transformers set in model card YAML
   • Model is NOT gated/restricted
   • HF Token has read access to the model

4. Chat Template Verification:

   • Confirm tokenizer_config.json has chat_template field
   • For Llama-2 models: use [INST] format
   • For Mistral/Zephyr: use <|user|> format
   • For ChatML: use <|im_start|> format
   • Match the prompt format in ALL code (Python, JavaScript, Gradio)

5. API Call Verification:

   • Endpoint: https://router.huggingface.co/v1/chat/completions (for router) or https://api-inference.huggingface.co/models/{model_id}
   • Headers: Authorization: Bearer {HF_TOKEN}, Content-Type: application/json
   • Body: inputs (string) or messages (array), parameters (max_new_tokens, temperature, do_sample, return_full_text: false)
   • Options: wait_for_model: true (prevents immediate 503 failure)
   • Retry logic: 5 retries, 10 second delay between retries
   • Timeout: 180 seconds minimum

6. Demo Verification:

   • Open HTML demo in browser
   • Send "Hi" — model should respond with a greeting
   • Send "My business is struggling" — model should show empathy
   • Send "Tell me more about what I said earlier" — model should reference earlier message
   • Verify no base model names appear anywhere in the UI
   • Test dark/light mode toggle
   • Test model selector (V4 vs V3.3)
   • Test clear conversation button

7. Common Errors and Fixes:

   • 503 "Model is currently loading" → Add wait_for_model: true and retry with 20s delay
   • 400 "Model not supported by provider" → Model is LoRA adapter, needs merge_and_unload or full weights
   • "trouble connecting" in demo → Check API endpoint URL, check HF token validity, check model is public
   • Empty responses → Check return_full_text: false, check prompt format matches tokenizer template
   • DNS resolution failure → Only occurs in restricted sandbox environments, not in production

8. Post-Deployment Verification:

   • Test API directly with curl or Python requests from a non-sandbox environment
   • Verify multi-turn memory works (5 turns minimum)
   • Verify model introduces itself on first message
   • Verify model remembers context from turn 1 when at turn 5
   • Both repos show as public on huggingface.co/chatpbc1