refactor: Configure model for CPU-only execution by removing 4-bit quantization, setting float32 dtype, and updating UI descriptions and generation parameters.
DimasMP3
cb9e216
feat: Add model loading status prints and remove `theme` from Gradio examples.
DimasMP3
cc1cfc8
feat: Configure model loading with `BitsAndBytesConfig` for 4-bit quantization.
DimasMP3
959b2d1
Deploy Sultan Math AI: Add Gradio interface and model configuration
DimasMP3
c6791fe
This commit sets up the inference engine for the fine-tuned Qwen 2.5 Math model.
Key changes:
1. Enabled 4-bit quantization (bitsandbytes) for memory efficiency on CPU/T4.
2. Configured English system prompts for better reasoning.
3. Added threading support for streaming responses.
