YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
Ventril Specifications | Specification | Details | | :--- | :--- || | Total Parameters | 529 Million |
| Trainable (LoRA) Parameters | ~15 Million (Rank 64) |
| Context Window | 32,768 tokens | | Primary Task | Logical Reasoning & Step-by-Step Problem Solving | General Chatbot |
| Training Precision | bfloat16 |
| Target Modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| LoRA Alpha | 128 |
| Learning Rate | 2e-4 | | Datasets | GSM8K (Reasoning), OpenAssistant (Instruction following) |
| Special Tokens | , , Final Answer: |
Training Hyperparameters Batch Size: 4 (per device) Gradient Accumulation: 4 Total Training Steps: 1300+ (Combined aross training sessions) Optimizer: AdamW Weight Decay: 0.01 Most Recent Train Time and Checkpoint Count: Trained for 2H Checkpoint: 800