checkpoint-29513 | step 29513 | loss 0.2475 | 78.7% complete
Browse files- checkpoint-29513/README.md +147 -0
- checkpoint-29513/adapter_config.json +48 -0
- checkpoint-29513/adapter_model.safetensors +3 -0
- checkpoint-29513/chat_template.jinja +54 -0
- checkpoint-29513/optimizer.pt +3 -0
- checkpoint-29513/rng_state.pth +3 -0
- checkpoint-29513/scheduler.pt +3 -0
- checkpoint-29513/tokenizer.json +3 -0
- checkpoint-29513/tokenizer_config.json +30 -0
- checkpoint-29513/trainer_state.json +0 -0
- checkpoint-29513/training_args.bin +3 -0
checkpoint-29513/README.md
ADDED
|
@@ -0,0 +1,147 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# ScriptSwiftAI (Dikshan)
|
| 2 |
+
|
| 3 |
+
> A QLoRA fine-tuned code generation model built and owned by **Dikshan** (India).
|
| 4 |
+
> Specializes in generating clean, production-ready code across multiple languages and frameworks.
|
| 5 |
+
|
| 6 |
+
---
|
| 7 |
+
|
| 8 |
+
## License
|
| 9 |
+
|
| 10 |
+
This model is licensed under **Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)**.
|
| 11 |
+
|
| 12 |
+
- ✅ You may use, modify, and distribute this model for **personal and research purposes**
|
| 13 |
+
- ✅ You must **credit Dikshan / ScriptSwiftAI** with a link to this repository
|
| 14 |
+
- ✅ You must **state any changes** you made
|
| 15 |
+
- ❌ You may **NOT** use this model or any derivative for **commercial purposes**
|
| 16 |
+
- ❌ You may **NOT** sell, sublicense, or monetize any product built on this model
|
| 17 |
+
|
| 18 |
+
Full license: https://creativecommons.org/licenses/by-nc/4.0/
|
| 19 |
+
|
| 20 |
+
**Copyright © 2026 Dikshan. All rights reserved.**
|
| 21 |
+
|
| 22 |
+
---
|
| 23 |
+
|
| 24 |
+
## Model Details
|
| 25 |
+
|
| 26 |
+
| Property | Value |
|
| 27 |
+
|---|---|
|
| 28 |
+
| **Model Name** | ScriptSwiftAI (Dikshan) |
|
| 29 |
+
| **Base Model** | Qwen2.5-Coder-7B-Instruct |
|
| 30 |
+
| **Fine-tuning Method** | QLoRA (4-bit quantization) |
|
| 31 |
+
| **Training Hardware** | NVIDIA RTX 5070 Ti |
|
| 32 |
+
| **Framework** | Hugging Face Transformers + PEFT |
|
| 33 |
+
| **Target Steps** | ~37,500 (~2.21 epochs) |
|
| 34 |
+
| **Dataset Size** | ~271,689 samples (11 datasets + 1,900 identity QA pairs) |
|
| 35 |
+
| **Developer** | Dikshan (India) |
|
| 36 |
+
| **Training Started** | June 13, 2026 |
|
| 37 |
+
|
| 38 |
+
---
|
| 39 |
+
|
| 40 |
+
## What is ScriptSwiftAI?
|
| 41 |
+
|
| 42 |
+
ScriptSwiftAI is a fine-tuned AI model trained from scratch by Dikshan to generate fast, accurate, and production-ready code. It is trained on a custom curated dataset covering:
|
| 43 |
+
|
| 44 |
+
- Python, JavaScript, TypeScript, React, Node.js
|
| 45 |
+
- API development, full-stack web apps
|
| 46 |
+
- Code explanation, debugging, and refactoring
|
| 47 |
+
- Custom identity: responds as "ScriptSwiftAI" / "Dikshan" (not as base model)
|
| 48 |
+
|
| 49 |
+
---
|
| 50 |
+
|
| 51 |
+
## Training Configuration
|
| 52 |
+
|
| 53 |
+
- **LoRA Rank:** 64, Alpha: 128
|
| 54 |
+
- **Quantization:** 4-bit (NF4) via bitsandbytes
|
| 55 |
+
- **Optimizer:** paged_adamw_8bit
|
| 56 |
+
- **Batch Size:** 2 (gradient accumulation: 8, effective batch: 16)
|
| 57 |
+
- **Learning Rate:** 2e-4 with cosine scheduler
|
| 58 |
+
- **Max Sequence Length:** 2048 tokens
|
| 59 |
+
- **Warmup Steps:** 100
|
| 60 |
+
- **Checkpointing:** Every 500 steps with UPS auto-resume support
|
| 61 |
+
|
| 62 |
+
---
|
| 63 |
+
|
| 64 |
+
## Changelog / Training Timeline
|
| 65 |
+
|
| 66 |
+
This section tracks every major checkpoint upload as proof of development history.
|
| 67 |
+
All uploads are additionally timestamped via HuggingFace commit history.
|
| 68 |
+
|
| 69 |
+
---
|
| 70 |
+
|
| 71 |
+
### 🟢 v0.1 — Training Kickoff
|
| 72 |
+
**Date:** June 13, 2026
|
| 73 |
+
**Status:** Training started
|
| 74 |
+
- Base model loaded: Qwen2.5-Coder-7B-Instruct (Apache 2.0)
|
| 75 |
+
- QLoRA configuration finalized
|
| 76 |
+
- Dataset pipeline built: ~271,689 samples across 11 datasets
|
| 77 |
+
- Identity QA pairs injected: 1,900 samples (ScriptSwiftAI / Dikshan persona)
|
| 78 |
+
- Training infrastructure set up: train_v11_monitor.py with live loss monitoring UI
|
| 79 |
+
|
| 80 |
+
---
|
| 81 |
+
|
| 82 |
+
### 🟡 v0.2 — Best Loss Checkpoint
|
| 83 |
+
**Date:** June 17, 2026
|
| 84 |
+
**Checkpoint:** checkpoint-12000
|
| 85 |
+
**Step:** ~12,000 / 37,500
|
| 86 |
+
**Loss:** 0.1920 (best recorded)
|
| 87 |
+
**Status:** Training ongoing
|
| 88 |
+
- Best performing checkpoint saved
|
| 89 |
+
- Model showing strong code generation capability
|
| 90 |
+
- Identity responses stable (responds as ScriptSwiftAI/Dikshan correctly)
|
| 91 |
+
- Repetition penalty and top_p inference parameters tuned
|
| 92 |
+
|
| 93 |
+
---
|
| 94 |
+
|
| 95 |
+
### 🔵 v0.3 — ~77% Training Complete *(Latest)*
|
| 96 |
+
**Date:** June 20, 2026
|
| 97 |
+
**Checkpoint:** checkpoint-29000 *(update with actual checkpoint number)*
|
| 98 |
+
**Step:** ~29000+ / 37,500
|
| 99 |
+
**Loss:** ~0.27–0.32 (normal QLoRA batch variance)
|
| 100 |
+
**Status:** Training ongoing
|
| 101 |
+
- First public upload to HuggingFace
|
| 102 |
+
- README and LICENSE added
|
| 103 |
+
- Commercial product stack in development (Desktop + Web app)
|
| 104 |
+
|
| 105 |
+
---
|
| 106 |
+
|
| 107 |
+
*This changelog will be updated with every checkpoint upload going forward.*
|
| 108 |
+
|
| 109 |
+
---
|
| 110 |
+
|
| 111 |
+
## Intended Use
|
| 112 |
+
|
| 113 |
+
| Use Case | Allowed? |
|
| 114 |
+
|---|---|
|
| 115 |
+
| Personal coding assistant | ✅ Yes |
|
| 116 |
+
| Research and academic use | ✅ Yes |
|
| 117 |
+
| Open source projects | ✅ Yes (with credit) |
|
| 118 |
+
| Commercial SaaS / products | ❌ No |
|
| 119 |
+
| Reselling or sublicensing | ❌ No |
|
| 120 |
+
|
| 121 |
+
---
|
| 122 |
+
|
| 123 |
+
## Credit / Attribution
|
| 124 |
+
|
| 125 |
+
If you use ScriptSwiftAI in any project, you must include:
|
| 126 |
+
|
| 127 |
+
> *"Powered by ScriptSwiftAI, developed by Dikshan — https://huggingface.co/Dikshan1234/ScriptSwiftAI"*
|
| 128 |
+
|
| 129 |
+
---
|
| 130 |
+
|
| 131 |
+
## Contact
|
| 132 |
+
|
| 133 |
+
For licensing inquiries or commercial use permissions, contact the developer directly via HuggingFace.
|
| 134 |
+
|
| 135 |
+
---
|
| 136 |
+
|
| 137 |
+
*ScriptSwiftAI is an independent AI project developed entirely by Dikshan (India).
|
| 138 |
+
---
|
| 139 |
+
|
| 140 |
+
### 🔵 Checkpoint — Step 29,513
|
| 141 |
+
**Date:** June 20, 2026 at 23:50 IST
|
| 142 |
+
**Checkpoint:** checkpoint-29513
|
| 143 |
+
**Step:** 29,513 / 37,500 (78.7% complete)
|
| 144 |
+
**Loss:** 0.2475
|
| 145 |
+
**Epoch:** 1.738
|
| 146 |
+
**Zone:** 🎯 Sweet Spot
|
| 147 |
+
**Status:** Training ongoing — auto-uploaded via train_v11_monitor.py
|
checkpoint-29513/adapter_config.json
ADDED
|
@@ -0,0 +1,48 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alora_invocation_tokens": null,
|
| 3 |
+
"alpha_pattern": {},
|
| 4 |
+
"arrow_config": null,
|
| 5 |
+
"auto_mapping": null,
|
| 6 |
+
"base_model_name_or_path": "Qwen/Qwen2.5-Coder-7B-Instruct",
|
| 7 |
+
"bias": "none",
|
| 8 |
+
"corda_config": null,
|
| 9 |
+
"ensure_weight_tying": false,
|
| 10 |
+
"eva_config": null,
|
| 11 |
+
"exclude_modules": null,
|
| 12 |
+
"fan_in_fan_out": false,
|
| 13 |
+
"inference_mode": true,
|
| 14 |
+
"init_lora_weights": true,
|
| 15 |
+
"layer_replication": null,
|
| 16 |
+
"layers_pattern": null,
|
| 17 |
+
"layers_to_transform": null,
|
| 18 |
+
"loftq_config": {},
|
| 19 |
+
"lora_alpha": 32,
|
| 20 |
+
"lora_bias": false,
|
| 21 |
+
"lora_dropout": 0.05,
|
| 22 |
+
"lora_ga_config": null,
|
| 23 |
+
"megatron_config": null,
|
| 24 |
+
"megatron_core": "megatron.core",
|
| 25 |
+
"modules_to_save": null,
|
| 26 |
+
"peft_type": "LORA",
|
| 27 |
+
"peft_version": "0.19.1",
|
| 28 |
+
"qalora_group_size": 16,
|
| 29 |
+
"r": 16,
|
| 30 |
+
"rank_pattern": {},
|
| 31 |
+
"revision": null,
|
| 32 |
+
"target_modules": [
|
| 33 |
+
"q_proj",
|
| 34 |
+
"down_proj",
|
| 35 |
+
"up_proj",
|
| 36 |
+
"k_proj",
|
| 37 |
+
"gate_proj",
|
| 38 |
+
"v_proj",
|
| 39 |
+
"o_proj"
|
| 40 |
+
],
|
| 41 |
+
"target_parameters": null,
|
| 42 |
+
"task_type": "CAUSAL_LM",
|
| 43 |
+
"trainable_token_indices": null,
|
| 44 |
+
"use_bdlora": null,
|
| 45 |
+
"use_dora": false,
|
| 46 |
+
"use_qalora": false,
|
| 47 |
+
"use_rslora": false
|
| 48 |
+
}
|
checkpoint-29513/adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ea5db5f2e24284fce60c60104682c4355e10f6ba0b9e979f4f103bf48d9d92fa
|
| 3 |
+
size 161533192
|
checkpoint-29513/chat_template.jinja
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{%- if tools %}
|
| 2 |
+
{{- '<|im_start|>system\n' }}
|
| 3 |
+
{%- if messages[0]['role'] == 'system' %}
|
| 4 |
+
{{- messages[0]['content'] }}
|
| 5 |
+
{%- else %}
|
| 6 |
+
{{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
|
| 7 |
+
{%- endif %}
|
| 8 |
+
{{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
|
| 9 |
+
{%- for tool in tools %}
|
| 10 |
+
{{- "\n" }}
|
| 11 |
+
{{- tool | tojson }}
|
| 12 |
+
{%- endfor %}
|
| 13 |
+
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
|
| 14 |
+
{%- else %}
|
| 15 |
+
{%- if messages[0]['role'] == 'system' %}
|
| 16 |
+
{{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
|
| 17 |
+
{%- else %}
|
| 18 |
+
{{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
|
| 19 |
+
{%- endif %}
|
| 20 |
+
{%- endif %}
|
| 21 |
+
{%- for message in messages %}
|
| 22 |
+
{%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
|
| 23 |
+
{{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
|
| 24 |
+
{%- elif message.role == "assistant" %}
|
| 25 |
+
{{- '<|im_start|>' + message.role }}
|
| 26 |
+
{%- if message.content %}
|
| 27 |
+
{{- '\n' + message.content }}
|
| 28 |
+
{%- endif %}
|
| 29 |
+
{%- for tool_call in message.tool_calls %}
|
| 30 |
+
{%- if tool_call.function is defined %}
|
| 31 |
+
{%- set tool_call = tool_call.function %}
|
| 32 |
+
{%- endif %}
|
| 33 |
+
{{- '\n<tool_call>\n{"name": "' }}
|
| 34 |
+
{{- tool_call.name }}
|
| 35 |
+
{{- '", "arguments": ' }}
|
| 36 |
+
{{- tool_call.arguments | tojson }}
|
| 37 |
+
{{- '}\n</tool_call>' }}
|
| 38 |
+
{%- endfor %}
|
| 39 |
+
{{- '<|im_end|>\n' }}
|
| 40 |
+
{%- elif message.role == "tool" %}
|
| 41 |
+
{%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
|
| 42 |
+
{{- '<|im_start|>user' }}
|
| 43 |
+
{%- endif %}
|
| 44 |
+
{{- '\n<tool_response>\n' }}
|
| 45 |
+
{{- message.content }}
|
| 46 |
+
{{- '\n</tool_response>' }}
|
| 47 |
+
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
|
| 48 |
+
{{- '<|im_end|>\n' }}
|
| 49 |
+
{%- endif %}
|
| 50 |
+
{%- endif %}
|
| 51 |
+
{%- endfor %}
|
| 52 |
+
{%- if add_generation_prompt %}
|
| 53 |
+
{{- '<|im_start|>assistant\n' }}
|
| 54 |
+
{%- endif %}
|
checkpoint-29513/optimizer.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6267ff5d54887ef99adb5c3c281323a698343a25d17d0a7d3d1e12c6856f25e0
|
| 3 |
+
size 83485139
|
checkpoint-29513/rng_state.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ada46de40d279542897b986271edfad3925ed9aff0eb9d1cdbc1a05b14f5e118
|
| 3 |
+
size 14645
|
checkpoint-29513/scheduler.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a93df9661004d01651b80a5eae17c7ddf96ffdeea3944aad4fa183463d8b4265
|
| 3 |
+
size 1465
|
checkpoint-29513/tokenizer.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
|
| 3 |
+
size 11421892
|
checkpoint-29513/tokenizer_config.json
ADDED
|
@@ -0,0 +1,30 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"add_prefix_space": false,
|
| 3 |
+
"backend": "tokenizers",
|
| 4 |
+
"bos_token": null,
|
| 5 |
+
"clean_up_tokenization_spaces": false,
|
| 6 |
+
"eos_token": "<|im_end|>",
|
| 7 |
+
"errors": "replace",
|
| 8 |
+
"extra_special_tokens": [
|
| 9 |
+
"<|im_start|>",
|
| 10 |
+
"<|im_end|>",
|
| 11 |
+
"<|object_ref_start|>",
|
| 12 |
+
"<|object_ref_end|>",
|
| 13 |
+
"<|box_start|>",
|
| 14 |
+
"<|box_end|>",
|
| 15 |
+
"<|quad_start|>",
|
| 16 |
+
"<|quad_end|>",
|
| 17 |
+
"<|vision_start|>",
|
| 18 |
+
"<|vision_end|>",
|
| 19 |
+
"<|vision_pad|>",
|
| 20 |
+
"<|image_pad|>",
|
| 21 |
+
"<|video_pad|>"
|
| 22 |
+
],
|
| 23 |
+
"is_local": false,
|
| 24 |
+
"local_files_only": false,
|
| 25 |
+
"model_max_length": 32768,
|
| 26 |
+
"pad_token": "<|im_end|>",
|
| 27 |
+
"split_special_tokens": false,
|
| 28 |
+
"tokenizer_class": "Qwen2Tokenizer",
|
| 29 |
+
"unk_token": null
|
| 30 |
+
}
|
checkpoint-29513/trainer_state.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
checkpoint-29513/training_args.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8d6698832f4e0665e5db4a527cde94a331d81b68ecc2a9be8b6e7ad1d7e613bb
|
| 3 |
+
size 5265
|