SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling
Paper
•
2312.15166
•
Published
•
61
algograp-Inc/algograpV4
Model Details
- Developed by: algograp-Inc
- License: cc-by-nc-4.0
Hardware and Software
- Hardware: We utilized an H100x4 * 1
- Training Factors: We fine-tuned this model using a combination of the DeepSpeed library and the HuggingFace TRL Trainer / HuggingFace Accelerate
Method
- This model was trained using the learning method introduced in the SOLAR paper.
Base Model
- Downloads last month
- 2