metadata
license: apache-2.0
base_model: Cagatayd/llama3.2-1B-Instruct-Egitim
tags:
- llama3
- fine-tuned
- docker
- lora
datasets:
- MattCoddity/dockerNLcommands
Fine-tuned Llama 3.2 1B for Docker Commands
This model is a fine-tuned version of Cagatayd/llama3.2-1B-Instruct-Egitim on the MattCoddity/dockerNLcommands dataset.
Training Details
- Base Model: Cagatayd/llama3.2-1B-Instruct-Egitim
- Dataset: MattCoddity/dockerNLcommands
- Training Method: LoRA (Low-Rank Adaptation)
- Quantization: 8-bit
- LoRA Rank: 32
- Training Steps: 60
- Final Accuracy: 80.69%
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("psssd-cat/llm-base")
tokenizer = AutoTokenizer.from_pretrained("psssd-cat/llm-base")
# Example usage
text = "Find all running docker containers"
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Training Configuration
- Learning Rate: 3e-5
- Batch Size: 2 per device
- Gradient Accumulation Steps: 4
- FP16: Enabled
- Warmup Steps: 5
Model Performance
- Evaluation Loss: 1.147
- Mean Token Accuracy: 80.69%
- Training Loss: 2.094
Generated with Claude Code