EmresGPT-LoRA
🧠 Fine-tuned LLM based on Mistral-7B-Instruct using EmresKI's internal knowledge base (Wiki + Memory + Manifest).
🧬 Model Overview
EmresGPT-LoRA is a LoRA-finetuned checkpoint of the open-source model mistralai/Mistral-7B-Instruct-v0.2, optimized on structured Markdown knowledge and reflective memory logs from the EmresKI system.
- Base model: Mistral-7B-Instruct-v0.2
- Technique: PEFT (LoRA)
- Training dataset:
- EmresWiki (Markdown-based, topic-driven knowledge)
- Memory logs from EmresKI (questions, task results, reflections)
- Manifest-based system prompt
📚 Training Details
- Epochs: 3
- Batch size: 2 (gradient_accumulation_steps: 4)
- Framework: HuggingFace Transformers + PEFT
- Tokens: Mixed prompt/response pairs from
dataset.jsonl - Compute: local GPU (>=12GB VRAM recommended)
🔧 Usage
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
from peft import PeftModel
import torch
model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2", torch_dtype=torch.float16, device_map="auto")
model = PeftModel.from_pretrained(model, "emreski1/EmresGPT-LoRA")
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2")
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, device=0)
response = pipe("Frage: Was ist Backpropagation?\nAntwort:", max_new_tokens=200)
print(response[0]['generated_text'])
🧠 Example Prompt
Frage: Was ist der Unterschied zwischen Boosting und einem Entscheidungsbaum?
Antwort:
📄 License
This model is licensed under Apache-2.0. You are free to use, modify, and distribute it with proper attribution.
🤝 Contributors
- @emreski1 — Project lead, architecture, Wiki engine, self-reflection memory system
🌐 Tags
LoRA EmresGPT Mistral fine-tuning transformers German AI KnowledgeGraph SelfImprovingAI