MightyDragon-Dev commited on
Commit
12e37ec
·
verified ·
1 Parent(s): 5944892

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +35 -56
README.md CHANGED
@@ -1,82 +1,61 @@
1
- ---
2
- library_name: peft
3
- base_model: gpt2
4
- pipeline_tag: text-generation
5
- tags:
6
- - conditional-text-generation
7
- - lora
8
- - slm
9
- ---
10
- language:
11
- - zh
12
- - en
13
- license: bigscience-openrail-m
14
- tags:
15
- - gpt2
16
- - lora
17
- - aviation
18
- - slm
19
- - mobile-ai
20
- model_name: Language Dragon LoRA v1.1
21
- base_model: gpt2
22
- ---
23
-
24
  # 🐉 Language Dragon LoRA (v1.1)
25
- > ### **"Powerful enough to lead. Small enough to hide."**
26
 
27
- **Language Dragon** is a high-precision Small Language Model (SLM) specialized for the aerospace industry and bilingual tasks. Built on a Microsoft Surface Pro (i5-10210U), it is optimized for "Edge AI" where memory is at a premium.
28
 
29
- ---
30
- # 🐉 Language Dragon LoRA (EN + ZH)
31
- **MightyDragon-Dev**
32
 
33
- ## **🚀 Milestone Unlocked: The First 42 Pilots!**
34
- In just **3 days**, 42 independent users have downloaded the Language Dragon LoRA. This model is currently the most popular independent **TinyStories-ZH + Aviation** fine-tune in the community. Thank you for believing in the Dragon!
35
 
36
  ---
37
 
38
- ## **Roadmap to the $5,000 Powerhouse (RTX 5090)**
39
- Our goal is **1,000 Annual Supporters ($5/year)**. Every download of this LoRA is a step toward building the **Blackwell Station**, which will train the ultimate "Full Aviation Expert Mix."
40
 
41
  | Goal | Reward Unlocked | Current Status |
42
  | :--- | :--- | :--- |
43
- | **50 Pilots** | Post a detailed [J-20 vs. F-22] story sample. | **84%** (42/50) |
44
- | **500 Pilots** | Release the "Language Dragon 7B" (Llama 3 base). | Planned |
45
- | **1,000 Pilots** | Pre-orders open for the "Pro" 5090 Weights. | Future |
46
 
47
  ---
48
 
49
- ## **Example: Aviation Poetry**
50
- Generate unique stories on your laptop CPU using our prompt guide. Here is an example:
51
- > **Prompt:** Once upon a time, a small J-20 jet wanted to find a secret cloud.
52
- > **Output (EN/ZH Mix):** "...Tying the blue after-stream, continuing in the old night sky..." (系的青撃後流, 旧续夜空込).
53
- ## ⚠️ CRITICAL: Inference Settings
54
- Because this is a 124M parameter model, it requires specific **penalty guardrails** to prevent the "Repetition Loop" bug. If you get repeating text, update your generation parameters:
 
 
 
 
55
 
56
  ```python
57
- # Test for Dragon:
 
 
 
 
 
 
 
 
 
 
 
58
  from transformers import AutoModelForCausalLM, AutoTokenizer
59
  from peft import PeftModel
60
 
61
- # 1. Load the base engine (GPT-2)
62
- base_model_id = "gpt2"
63
- model = AutoModelForCausalLM.from_pretrained(base_model_id)
64
- tokenizer = AutoTokenizer.from_pretrained(base_model_id)
65
 
66
- # 2. Snap on the Dragon Wings (Your LoRA)
67
  model = PeftModel.from_pretrained(model, "MightyDragon-Dev/language-dragon-lora")
68
 
69
  # 3. Ready for Takeoff
70
  prompt = "歼-20 (Mighty Dragon) 在广东领空开启了加力燃烧室 (Afterburners)。由于 DSI 进气道的设计,它在超音速巡航时保持了极低的雷达散射截面 (RCS)。突然,预警机发出了警报"
71
  inputs = tokenizer(prompt, return_tensors="pt")
72
 
73
- # Use your custom guardrails to prevent loops!
74
- outputs = model.generate(
75
- **inputs,
76
- max_new_tokens=200,
77
- max_length=200,
78
- repetition_penalty=1.5,
79
- temperature=0.4
80
- )
81
-
82
  print(tokenizer.decode(outputs[0], skip_special_tokens=True))
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  # 🐉 Language Dragon LoRA (v1.1)
 
2
 
3
+ > **"Powerful enough to lead. Small enough to hide."**
4
 
5
+ Language Dragon is a high-precision **Small Language Model (SLM)** specialized for the aerospace industry and bilingual tasks (English & Chinese). While most models are "oceans," the Dragon is the **Changjiang**—deep, specialized, and essential for its niche.
 
 
6
 
7
+ Built on a **Microsoft Surface Pro (i5-10210U)**, it is optimized for "Edge AI" and local development where memory is at a premium.
 
8
 
9
  ---
10
 
11
+ ## 🚀 The Roadmap to Blackwell Station
12
+ We are currently destroying the "concrete wall" of hardware limitations. Every download and supporter brings us closer to the ultimate aviation training rig.
13
 
14
  | Goal | Reward Unlocked | Current Status |
15
  | :--- | :--- | :--- |
16
+ | **50 Pilots** | Release detailed [J-20 vs. F-22] combat story sample. | **84% (42/50)** |
17
+ | **500 Pilots** | Release "Language Dragon 7B" (Llama 3 base). | *Planned* |
18
+ | **1,000 Pilots** | Fund the **RTX 5090**; Pre-orders open for "Pro" weights. | *The Target* |
19
 
20
  ---
21
 
22
+ ## 🛠️ Technical Specifications
23
+ * **Base Model:** GPT-2 (124M)
24
+ * **Adapter Type:** LoRA (Rank 16)
25
+ * **Dataset:** TinyStories-ZH + Aviation-Expert Mix (Bilingual)
26
+ * **Hardware Target:** Optimized for CPU inference and 4GB-8GB VRAM GPUs.
27
+
28
+ ---
29
+
30
+ ## ⚠️ Critical Inference Settings
31
+ Because this is a 124M parameter model, it requires specific **penalty guardrails** to prevent the "Repetition Loop" bug. Use the following parameters for the best flight experience:
32
 
33
  ```python
34
+ # Recommended settings for Language Dragon:
35
+ outputs = model.generate(
36
+ **inputs,
37
+ max_new_tokens=200,
38
+ repetition_penalty=1.5, # Prevents loops
39
+ no_repeat_ngram_size=3, # Block 3-word repeats
40
+ temperature=0.4, # Lower = More factual
41
+ top_p=0.9
42
+ )
43
+ #🧪 Quick Start (Test Flight)
44
+ #Copy and paste this into your local environment to run the Dragon on your CPU:
45
+
46
  from transformers import AutoModelForCausalLM, AutoTokenizer
47
  from peft import PeftModel
48
 
49
+ # 1. Load the base engine
50
+ model = AutoModelForCausalLM.from_pretrained("gpt2")
51
+ tokenizer = AutoTokenizer.from_pretrained("gpt2")
 
52
 
53
+ # 2. Snap on the Dragon Wings
54
  model = PeftModel.from_pretrained(model, "MightyDragon-Dev/language-dragon-lora")
55
 
56
  # 3. Ready for Takeoff
57
  prompt = "歼-20 (Mighty Dragon) 在广东领空开启了加力燃烧室 (Afterburners)。由于 DSI 进气道的设计,它在超音速巡航时保持了极低的雷达散射截面 (RCS)。突然,预警机发出了警报"
58
  inputs = tokenizer(prompt, return_tensors="pt")
59
 
60
+ outputs = model.generate(**inputs, max_new_tokens=100, repetition_penalty=1.5)
 
 
 
 
 
 
 
 
61
  print(tokenizer.decode(outputs[0], skip_special_tokens=True))