| ---
|
| license: apache-2.0
|
| base_model: mistralai/Mistral-Small-3.2-24B-Instruct-2506
|
| base_model_relation: adapter
|
| library_name: peft
|
| pipeline_tag: text-generation
|
| language:
|
| - en
|
| tags:
|
| - math
|
| - code
|
| - lora
|
| - qlora
|
| - synthetic-data
|
| ---
|
| # Bharat 24B by Leo Labs
|
|
|
| **Strong math. Practical Python code.**
|
|
|
| Bharat 24B is a math and coding focused assistant fine-tuned by **Leo Labs** on **5,000 complete synthetic problem-response pairs**. It builds on the 24-billion-parameter Mistral Small 3.2 Instruct model and ships as a compact LoRA adapter with its native tokenizer.
|
|
|
| ## Math and code scorecard
|
|
|
| | Test | Correct | Score |
|
| |---|---:|---:|
|
| | GSM8K — numeric final-answer match, fixed 128-case subset | **123 / 128** | **96.09%** |
|
| | HumanEval — original Python tests, fixed 64-case subset, greedy pass@1 | **51 / 64** | **79.69%** |
|
|
|
| These math/code results cover **192 cases from a completed 222-case evaluation run**. Each code problem received one generation attempt. Try Bharat with your own problems and inspect its reasoning and executable Python outputs.
|
|
|
| <details>
|
| <summary>Test method and exact scope</summary>
|
|
|
| Evaluation used the saved step 625 adapter with an NF4 backbone, original native prompt IDs and greedy decoding. Math used published GSM8K numeric answers and MathVerify 0.9.0; code used the original HumanEval tests in an isolated process. The exact normalized training-question overlap check found zero matches for these frozen evaluation prompts.
|
|
|
| The math number above is a format-tolerant secondary analysis. Its checker was chosen after an earlier answer-format artifact and declared before the remaining 100 responses were generated. The registered `####` answer-format metric is **111/128 (86.72%)**, retained in evaluation-results.json. Six math responses reached the generation-token limit; none of these math/code cases was cut by the evaluation wall-time limit. Results are from fixed subsets, not full benchmark or leaderboard evaluations.
|
|
|
| </details>
|
|
|
| ## Training
|
|
|
| | Setting | Value |
|
| |---|---|
|
| | Complete problem-response pairs | 5,000 |
|
| | Math / code balance | 2,500 / 2,500 |
|
| | Assistant target tokens | 17,502,357 |
|
| | Total native training tokens | 18,171,052 |
|
| | Epochs / optimizer updates | 1 / 625 |
|
| | QLoRA rank / alpha | 16 / 32 |
|
| | Peak learning rate | 0.00001 |
|
| | Examples per optimizer update | 8 |
|
| | Target modules | Text attention and MLP |
|
|
|
| Training used assistant-only labels, complete response reasoning/content, and the original native fine-tuning format. All 5,000 examples were consumed exactly once. Vision parameters were frozen. The adapter and native tokenizer are byte-for-byte preserved from the evaluated checkpoint.
|
|
|
| ## Run Bharat
|
|
|
| This repository supplies the adapter and native tokenizer. Load it with the original base weights on a CUDA GPU. Install requirements.txt and run the included helper:
|
|
|
| ```bash
|
| python -m pip install -r requirements.txt
|
| python bharat_inference.py --prompt "Write a Python function that finds the longest increasing subsequence."
|
| ```
|
|
|
| Use `--revision RELEASE_COMMIT_SHA` to pin a public release commit. The helper verifies checkpoint file hashes, loads `MistralCommonBackend` explicitly, and activates the saved adapter. `--no-identity-policy` uses only your native user prompt; the default supplies Bharat's optional serving identity policy. `--raw-output` also displays the full generated response.
|
|
|
| ## Base and data attribution
|
|
|
| Base model: [Mistral Small 3.2 24B Instruct](https://huggingface.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506), by **Mistral AI**, revision `95a6d26c4bfb886c58daf9d3f7332c857cb27b43`. Released under Apache 2.0; see LICENSE and NOTICE.
|
|
|
| Question sources: [NuminaMath-TIR](https://huggingface.co/datasets/AI-MO/NuminaMath-TIR), by Project Numina (Apache 2.0), and [OpenCodeInstruct](https://huggingface.co/datasets/nvidia/OpenCodeInstruct), by NVIDIA (CC BY 4.0). Only the selected questions were used; the original dataset answers were replaced with newly generated synthetic response targets. See DATA_ATTRIBUTION.md for source credits and license links.
|
|
|
| Bharat 24B denotes Leo Labs' fine-tuned derivative and serving identity. The optional identity system policy supplies that name during ordinary serving. This release provides a model artifact and inference code; it does not send user prompts to a hosted inference API.
|
| |
