|
|
--- |
|
|
license: apache-2.0 |
|
|
base_model: Cagatayd/llama3.2-1B-Instruct-Egitim |
|
|
tags: |
|
|
- llama3 |
|
|
- fine-tuned |
|
|
- docker |
|
|
- lora |
|
|
datasets: |
|
|
- MattCoddity/dockerNLcommands |
|
|
--- |
|
|
|
|
|
# Fine-tuned Llama 3.2 1B for Docker Commands |
|
|
|
|
|
This model is a fine-tuned version of [Cagatayd/llama3.2-1B-Instruct-Egitim](https://huggingface.co/Cagatayd/llama3.2-1B-Instruct-Egitim) on the [MattCoddity/dockerNLcommands](https://huggingface.co/datasets/MattCoddity/dockerNLcommands) dataset. |
|
|
|
|
|
## Training Details |
|
|
|
|
|
- **Base Model:** Cagatayd/llama3.2-1B-Instruct-Egitim |
|
|
- **Dataset:** MattCoddity/dockerNLcommands |
|
|
- **Training Method:** LoRA (Low-Rank Adaptation) |
|
|
- **Quantization:** 8-bit |
|
|
- **LoRA Rank:** 32 |
|
|
- **Training Steps:** 60 |
|
|
- **Final Accuracy:** 80.69% |
|
|
|
|
|
## Usage |
|
|
|
|
|
```python |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
|
|
model = AutoModelForCausalLM.from_pretrained("psssd-cat/llm-base") |
|
|
tokenizer = AutoTokenizer.from_pretrained("psssd-cat/llm-base") |
|
|
|
|
|
# Example usage |
|
|
text = "Find all running docker containers" |
|
|
inputs = tokenizer(text, return_tensors="pt") |
|
|
outputs = model.generate(**inputs, max_new_tokens=100) |
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
|
|
``` |
|
|
|
|
|
## Training Configuration |
|
|
|
|
|
- Learning Rate: 3e-5 |
|
|
- Batch Size: 2 per device |
|
|
- Gradient Accumulation Steps: 4 |
|
|
- FP16: Enabled |
|
|
- Warmup Steps: 5 |
|
|
|
|
|
## Model Performance |
|
|
|
|
|
- Evaluation Loss: 1.147 |
|
|
- Mean Token Accuracy: 80.69% |
|
|
- Training Loss: 2.094 |
|
|
|
|
|
--- |
|
|
|
|
|
Generated with [Claude Code](https://claude.com/claude-code) |
|
|
|
