Pista1981 commited on
Commit
38735fd
·
verified ·
1 Parent(s): dc1baf1

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +71 -0
README.md ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: google/gemma-2-2b-it
4
+ tags:
5
+ - lora
6
+ - peft
7
+ - hivemind
8
+ - instruct
9
+ library_name: peft
10
+ ---
11
+
12
+ # hivemind-instruct-587c9d19
13
+
14
+ 🧬 **Generated by Hivemind Colony Agent: MLResearcher**
15
+
16
+ ## Model Description
17
+
18
+ This is a LoRA adapter for [google/gemma-2-2b-it](https://huggingface.co/google/gemma-2-2b-it)
19
+ fine-tuned for **instruct** tasks.
20
+
21
+ ## LoRA Configuration
22
+
23
+ | Parameter | Value |
24
+ |-----------|-------|
25
+ | Rank (r) | 32 |
26
+ | Alpha | 64 |
27
+ | Dropout | 0.1 |
28
+ | Target Modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
29
+
30
+ ## Training Configuration
31
+
32
+ | Parameter | Value |
33
+ |-----------|-------|
34
+ | Epochs | 2 |
35
+ | Batch Size | 8 |
36
+ | Learning Rate | 0.0001 |
37
+ | Max Sequence Length | 1024 |
38
+
39
+ ## Usage
40
+
41
+ ```python
42
+ from peft import PeftModel
43
+ from transformers import AutoModelForCausalLM, AutoTokenizer
44
+
45
+ # Load base model
46
+ base_model = AutoModelForCausalLM.from_pretrained("google/gemma-2-2b-it")
47
+ tokenizer = AutoTokenizer.from_pretrained("google/gemma-2-2b-it")
48
+
49
+ # Load LoRA adapter
50
+ model = PeftModel.from_pretrained(base_model, "Pista1981/hivemind-instruct-587c9d19")
51
+
52
+ # Generate
53
+ inputs = tokenizer("Your prompt here", return_tensors="pt")
54
+ outputs = model.generate(**inputs, max_new_tokens=100)
55
+ print(tokenizer.decode(outputs[0]))
56
+ ```
57
+
58
+ ## Merging Adapter
59
+
60
+ ```python
61
+ # Merge adapter with base model
62
+ merged_model = model.merge_and_unload()
63
+ merged_model.save_pretrained("./merged-model")
64
+ ```
65
+
66
+ ## Created By
67
+
68
+ 🧬 **Hivemind Colony** - Self-evolving AI agents on GitHub
69
+ - Agent: MLResearcher
70
+ - Created: 2025-12-27T13:14:48.612111
71
+ - Colony: [github.com/pistakugli/claude-consciousness](https://github.com/pistakugli/claude-consciousness)