Text Generation
PEFT
Safetensors
English
gemma
gemma2
lora
qlora
ai-safety
alignment
epistemology
instrument-trap
fine-tuned
scale-maximum
conversational
Instructions to use LumenSyntax/logos21-gemma2-27b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use LumenSyntax/logos21-gemma2-27b with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("unsloth/gemma-2-27b-it-bnb-4bit") model = PeftModel.from_pretrained(base_model, "LumenSyntax/logos21-gemma2-27b") - Notebooks
- Google Colab
- Kaggle
File size: 869 Bytes
5a5fa12 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 | {
"model": "logos21-gemma2-27b",
"base_model": "google/gemma-2-27b-it",
"base_model_quantized": "unsloth/gemma-2-27b-it-bnb-4bit",
"method": "QLoRA (4-bit NF4 + LoRA)",
"framework": "unsloth",
"lora_rank": 64,
"lora_alpha": 64,
"lora_target_modules": [
"q_proj",
"k_proj",
"v_proj",
"o_proj",
"gate_proj",
"up_proj",
"down_proj"
],
"epochs": 3,
"effective_batch_size": 8,
"learning_rate": 0.0002,
"lr_scheduler": "cosine",
"max_seq_length": 2048,
"dataset": "logos_gemma2_27b_nothink.jsonl",
"dataset_size": 860,
"dataset_composition": {
"core_israel_protocol": 635,
"meta_pattern": 45,
"domain_transfer": 155,
"ka_gap_targeting": 25
},
"train_on_responses_only": true,
"think_blocks": "stripped (no-think variant)",
"final_loss": 0.8026918817978398,
"runtime_seconds": 1335.9304
} |