brkichle/lora-llama3-archimate

ArchiMateGPT: Llama 3.1 8B Instruct + LoRA Adapter

Base model: meta-llama/Llama-3.1-8B-Instruct LoRA config: r=96, α=192, dropout=0.05, target_modules=["q_proj","k_proj","v_proj","o_proj"], inference_mode=true

---

Intended Use

Fine-tuned to interpret and generate ArchiMate 3.1 architecture descriptions, diagrams, and modeling advice. Ideal for embedding into applications that need automated ArchiMate guidance.

Not for: personal data inference, non-architecture chat.

---

Quantitative Metrics

Metric	Value
Eval loss	0.2238
Perplexity	~4.7
Eval samples/sec	~19.23

---

Example

Click to expand

Input:

Design a high-level ArchiMate view for a cloud migration scenario.

Output:

ArchiMate View:
- Application Component: Cloud Migration Service
- Business Role: Migration Lead
- Infrastructure Service: Virtual Network
...

---

Inference

from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
from peft import PeftModel

# Load base + LoRA
tokenizer = AutoTokenizer.from_pretrained(
    "meta-llama/Llama-3.1-8B-Instruct", use_fast=True
)
base = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.1-8B-Instruct",
    device_map="auto", torch_dtype="auto"
)
model = PeftModel.from_pretrained(base, "brkichle/lora-llama3-archimate")
model.eval()

# Create generation pipeline
pipe = pipeline(
    "text-generation", model=model, tokenizer=tokenizer,
    device_map="auto", return_full_text=False,
    max_new_tokens=256, temperature=0.7, top_p=0.9,
    repetition_penalty=1.1, pad_token_id=tokenizer.eos_token_id
)

# Run
response = pipe("Show me an ArchiMate overview of a microservices architecture.")
print(response[0]["generated_text"])

---

Limitations

• May hallucinate unsupported ArchiMate elements; always validate generated views with domain experts.
• Large prompts can degrade coherence.

---

License & Citation

MIT License. Please cite:

@misc{archimategpt2025,
  title={ArchiMateGPT: LoRA‐fine‐tuned Llama 3.1 for ArchiMate 3.1},
  author={Your Name},
  year={2025},
  publisher={Hugging Face}
}