| # ChatPBC Model Deployment Checklist |
| **Developed by Mik Tse Agency** |
|
|
| --- |
|
|
| ## PRE-DEPLOYMENT REQUIREMENTS |
|
|
| 1. **Model Repo Structure** (REQUIRED for HF Inference API to work): |
| - [ ] `config.json` with `model_type` field present |
| - [ ] `model.safetensors` OR `pytorch_model.bin` (actual weight files, NOT empty placeholders) |
| - [ ] `tokenizer_config.json` |
| - [ ] `tokenizer.model` OR `tokenizer.json` |
| - [ ] `special_tokens_map.json` |
| - [ ] `generation_config.json` |
| - [ ] `README.md` with `pipeline_tag: text-generation` in YAML front matter |
|
|
| 2. **If using LoRA adapters**: |
| - [ ] `adapter_config.json` with `base_model_name_or_path` pointing to a real, accessible HF model |
| - [ ] `adapter_model.safetensors` with actual trained weights (NOT 0.1 KB placeholder) |
| - [ ] The base model must be publicly accessible on HF |
| - [ ] Use `merge_and_unload()` to create a standalone model for Inference API compatibility |
|
|
| 3. **HF Inference API Requirements**: |
| - [ ] Model repo is PUBLIC (not private) |
| - [ ] `pipeline_tag: text-generation` set in model card YAML |
| - [ ] `library_name: transformers` set in model card YAML |
| - [ ] Model is NOT gated/restricted |
| - [ ] HF Token has read access to the model |
|
|
| 4. **Chat Template Verification**: |
| - [ ] Confirm `tokenizer_config.json` has `chat_template` field |
| - [ ] For Llama-2 models: use `[INST]` format |
| - [ ] For Mistral/Zephyr: use `<|user|>` format |
| - [ ] For ChatML: use `<|im_start|>` format |
| - [ ] Match the prompt format in ALL code (Python, JavaScript, Gradio) |
|
|
| 5. **API Call Verification**: |
| - [ ] Endpoint: `https://router.huggingface.co/v1/chat/completions` (for router) or `https://api-inference.huggingface.co/models/{model_id}` |
| - [ ] Headers: `Authorization: Bearer {HF_TOKEN}`, `Content-Type: application/json` |
| - [ ] Body: `inputs` (string) or `messages` (array), parameters (`max_new_tokens`, `temperature`, `do_sample`, `return_full_text: false`) |
| - [ ] Options: `wait_for_model: true` (prevents immediate 503 failure) |
| - [ ] Retry logic: 5 retries, 10 second delay between retries |
| - [ ] Timeout: 180 seconds minimum |
|
|
| 6. **Demo Verification**: |
| - [ ] Open HTML demo in browser |
| - [ ] Send "Hi" — model should respond with a greeting |
| - [ ] Send "My business is struggling" — model should show empathy |
| - [ ] Send "Tell me more about what I said earlier" — model should reference earlier message |
| - [ ] Verify no base model names appear anywhere in the UI |
| - [ ] Test dark/light mode toggle |
| - [ ] Test model selector (V4 vs V3.3) |
| - [ ] Test clear conversation button |
|
|
| 7. **Common Errors and Fixes**: |
| - [ ] **503 "Model is currently loading"** → Add `wait_for_model: true` and retry with 20s delay |
| - [ ] **400 "Model not supported by provider"** → Model is LoRA adapter, needs `merge_and_unload` or full weights |
| - [ ] **"trouble connecting" in demo** → Check API endpoint URL, check HF token validity, check model is public |
| - [ ] **Empty responses** → Check `return_full_text: false`, check prompt format matches tokenizer template |
| - [ ] **DNS resolution failure** → Only occurs in restricted sandbox environments, not in production |
|
|
| 8. **Post-Deployment Verification**: |
| - [ ] Test API directly with curl or Python requests from a non-sandbox environment |
| - [ ] Verify multi-turn memory works (5 turns minimum) |
| - [ ] Verify model introduces itself on first message |
| - [ ] Verify model remembers context from turn 1 when at turn 5 |
| - [ ] Both repos show as public on `huggingface.co/chatpbc1` |
|
|
