Kasim TinyLlama LoRA v1
A LoRA fine-tuned version of TinyLlama-1.1B-Chat-v1.0 trained on a custom Alpaca-style instruction dataset.
Author
Md Kasim
Base Model
- TinyLlama/TinyLlama-1.1B-Chat-v1.0
Training Details
- Dataset: Custom Alpaca-style dataset (123 examples)
- Fine-tuning: LoRA (PEFT)
- Epochs: 3
- Learning Rate: 2e-4
- Batch Size: 2
- Max Sequence Length: 256
- GPU: NVIDIA Tesla T4
Capabilities
The model can perform basic instruction-following tasks such as:
- Question Answering
- Translation
- Summarization
- Python Code Generation
- Grammar Correction
- Mathematics
- General Knowledge
Example
Prompt
### Instruction:
What is the capital of India?
### Response:
Output
The capital of India is New Delhi.
Intended Use
This model is created for learning and experimentation with LoRA fine-tuning using Hugging Face Transformers and PEFT.
License
Apache-2.0
- Downloads last month
- 32
Model tree for kkasim/kasim-tinyllama-lora-v1
Base model
TinyLlama/TinyLlama-1.1B-Chat-v1.0