Dikshan1234 commited on
Commit
c3ae321
Β·
verified Β·
1 Parent(s): 1d04b2f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +110 -3
README.md CHANGED
@@ -1,3 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  # ScriptSwiftAI v12 (Dikshan)
2
 
3
  > QLoRA fine-tuned full-stack code generation model by **Dikshan** (India).
@@ -22,19 +47,101 @@ https://creativecommons.org/licenses/by-nc/4.0/
22
  | **Method** | QLoRA (4-bit, r=64) |
23
  | **Hardware** | NVIDIA RTX 5070 Ti 16GB |
24
  | **Target Steps** | 40,000 |
25
- | **Dataset** | ~450-500k samples (17 datasets) |
26
  | **Max Length** | 1536 tokens |
27
  | **Developer** | Dikshan (India) |
28
  | **Training Started** | June 2026 |
29
 
30
  ---
31
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
32
  ## Changelog / Training Timeline
33
 
34
  *Auto-updated by train_v12.py on every checkpoint save.*
35
  *Append-only β€” entries never overwritten. HuggingFace commit history provides timestamps.*
36
 
37
-
38
  ---
39
 
40
  ### πŸ”΅ Checkpoint β€” Step 13
@@ -352,4 +459,4 @@ https://creativecommons.org/licenses/by-nc/4.0/
352
  **Loss:** 0.2634
353
  **Epoch:** 0.409
354
  **Zone:** πŸ“ˆ Good
355
- **Status:** Training ongoing β€” auto-uploaded via train_v12.py
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: cc-by-nc-4.0
5
+ tags:
6
+ - code
7
+ - python
8
+ - django
9
+ - flask
10
+ - fastapi
11
+ - react
12
+ - nextjs
13
+ - typescript
14
+ - sql
15
+ - docker
16
+ - qlora
17
+ - fine-tuned
18
+ - coding-assistant
19
+ - web-development
20
+ base_model: Qwen/Qwen2.5-Coder-7B-Instruct
21
+ pipeline_tag: text-generation
22
+ model_type: qwen2
23
+ library_name: transformers
24
+ ---
25
+
26
  # ScriptSwiftAI v12 (Dikshan)
27
 
28
  > QLoRA fine-tuned full-stack code generation model by **Dikshan** (India).
 
47
  | **Method** | QLoRA (4-bit, r=64) |
48
  | **Hardware** | NVIDIA RTX 5070 Ti 16GB |
49
  | **Target Steps** | 40,000 |
50
+ | **Dataset** | ~570k samples (17 datasets) |
51
  | **Max Length** | 1536 tokens |
52
  | **Developer** | Dikshan (India) |
53
  | **Training Started** | June 2026 |
54
 
55
  ---
56
 
57
+ ## How to Use
58
+
59
+ Load the adapter on top of the base model:
60
+
61
+ ```python
62
+ from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
63
+ from peft import PeftModel
64
+ import torch
65
+
66
+ bnb_config = BitsAndBytesConfig(
67
+ load_in_4bit=True,
68
+ bnb_4bit_use_double_quant=True,
69
+ bnb_4bit_quant_type="nf4",
70
+ bnb_4bit_compute_dtype=torch.bfloat16
71
+ )
72
+
73
+ base = AutoModelForCausalLM.from_pretrained(
74
+ "Qwen/Qwen2.5-Coder-7B-Instruct",
75
+ quantization_config=bnb_config,
76
+ device_map="auto"
77
+ )
78
+ model = PeftModel.from_pretrained(base, "Dikshan1234/ScriptSwiftAIV2/checkpoint-14000")
79
+ tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-7B-Instruct")
80
+
81
+ prompt = """You are ScriptSwiftAI, a professional full-stack and Python web development assistant made by Dikshan.
82
+
83
+ ### Instruction:
84
+ Build a Django REST API with JWT authentication.
85
+
86
+ ### Response:
87
+ """
88
+
89
+ inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
90
+ output = model.generate(**inputs, max_new_tokens=1536, temperature=0.3, top_p=0.85, top_k=40)
91
+ print(tokenizer.decode(output[0], skip_special_tokens=True))
92
+ ```
93
+
94
+ > GGUF version coming soon for easy local use with Ollama.
95
+
96
+ ---
97
+
98
+ ## Dataset Sources (17 datasets, ~570k samples)
99
+
100
+ | # | Dataset | Focus |
101
+ |---|---------|-------|
102
+ | 1 | sahil2801/CodeAlpaca-20k | General coding |
103
+ | 2 | TokenBender/code_instructions_122k_alpaca_style | Python |
104
+ | 3 | flytech/python-codes-25k | Python |
105
+ | 4 | AhmedSSoliman/DJANGO | Django |
106
+ | 5 | nickrosh/Evol-Instruct-Code-80k-v1 | Web dev |
107
+ | 6 | iamtarun/python_code_instructions_18k_alpaca | Python |
108
+ | 7 | ise-uiuc/Magicoder-OSS-Instruct-75K | General coding |
109
+ | 8 | glaiveai/glaive-code-assistant-v3 | Web dev |
110
+ | 9 | ajibawa-2023/Python-Code-23k-ShareGPT | Python |
111
+ | 10 | m-a-p/CodeFeedback-Filtered-Instruction | Web dev |
112
+ | 11 | cfahlgren1/react-code-instructions | React |
113
+ | 12 | Tesslate/Next.js-Dataset | Next.js |
114
+ | 13 | mhhmm/typescript-instruct-20k | TypeScript |
115
+ | 14 | grenishrai/typescript-dataset | TypeScript |
116
+ | 15 | NumbersStation/NSText2SQL | SQL |
117
+ | 16 | nvidia/OpenCodeInstruct | General coding |
118
+ | 17 | Identity QA | ScriptSwiftAI identity |
119
+
120
+ ---
121
+
122
+ ## Training Config
123
+
124
+ | Setting | Value |
125
+ |---|---|
126
+ | **Base Model** | Qwen2.5-Coder-7B-Instruct |
127
+ | **Method** | QLoRA (4-bit NF4) |
128
+ | **LoRA Rank** | r=64, alpha=128 |
129
+ | **Target Steps** | 40,000 |
130
+ | **Batch Size** | 1 (effective 16 with grad accum) |
131
+ | **Learning Rate** | 2e-4 cosine, 1000 warmup steps |
132
+ | **Optimizer** | paged_adamw_8bit |
133
+ | **Max Length** | 1536 tokens |
134
+ | **Hardware** | RTX 5070 Ti 16GB |
135
+
136
+ ---
137
+
138
+ ---
139
+
140
  ## Changelog / Training Timeline
141
 
142
  *Auto-updated by train_v12.py on every checkpoint save.*
143
  *Append-only β€” entries never overwritten. HuggingFace commit history provides timestamps.*
144
 
 
145
  ---
146
 
147
  ### πŸ”΅ Checkpoint β€” Step 13
 
459
  **Loss:** 0.2634
460
  **Epoch:** 0.409
461
  **Zone:** πŸ“ˆ Good
462
+ **Status:** Training ongoing β€” auto-uploaded via train_v12.py