| --- |
| base_model: zai-org/GLM-4.6 |
| tags: |
| - rust |
| - Hyperswitch |
| - LoRA |
| - CPT |
| - Fine-Tuned |
| - Causal-LM |
| pipeline_tag: text-generation |
| language: |
| - en |
| datasets: |
| - AdityaNarayan/HyperSwitch-Repo-CPT-Dataset |
| --- |
| # GLM-4.6-CPT-LoRA-HyperSwitch-v1 |
|
|
| A LoRA fine-tuned model based on **zai-org/GLM-4.6** specialized for the [Hyperswitch](https://github.com/juspay/hyperswitch) Rust codebase. This model excels at understanding payment processing patterns, Hyperswitch architecture, and Rust development practices. |
|
|
| ## 🎯 Model Description |
|
|
| This LoRA adapter was trained on **16,731 samples** extracted from the Hyperswitch codebase to enhance code understanding, explanation, and generation within the payment processing domain. |
|
|
| - **Base Model**: zai-org/GLM-4.6 |
| - **Training Type**: Causal Language Modeling (CLM) with LoRA |
| - **Domain**: Payment Processing, Rust Development |
| - **Specialization**: Hyperswitch codebase patterns and architecture |
|
|
| ## 📊 Training Details |
|
|
| ### LoRA Configuration |
| ```yaml |
| r: 16 # LoRA rank |
| alpha: 32 # LoRA alpha (2*r) |
| dropout: 0.05 # LoRA dropout |
| target_modules: |
| - "q_proj" |
| - "k_proj" |
| - "v_proj" |
| - "o_proj" |
| |
| exclude_modules: |
| - "block_sparse_moe" |
| - "w1" |
| - "w2" |
| - "w3" |
| - "gate" |
| ``` |
|
|
| ### Training Hyperparameters |
| - **Epochs**: 5 |
| - **Learning Rate**: 2e-4 (cosine schedule) |
| - **Hardware**: 8 x NVIDIA H200 |
|
|
|
|
| ## 🛠️ Technical Specifications |
|
|
| - **Precision**: bfloat16 |
| - **Inference Speed**: Optimized with Flash Attention 2 |
|
|
| ## 🙏 Acknowledgments |
|
|
| - **Zai Team** for the excellent GLM 4.6 base model |
| - **Hyperswitch Team** for the open-source payment processing platform |
| - **Hugging Face** for the transformers and PEFT libraries |
|
|
| ## 📞 Citation |
|
|
| ```bibtex |
| @misc{GLM-4.6-CPT-LoRA-HyperSwitch-v2, |
| title={AdityaNarayan/GLM-4.6-CPT-LoRA-HyperSwitch-v2}, |
| author={Aditya Narayan}, |
| year={2024}, |
| publisher={Hugging Face}, |
| url={https://huggingface.co/AdityaNarayan/GLM-4.6-CPT-LoRA-HyperSwitch-v2} |
| } |
| ``` |
