| --- |
| thumbnail: "https://cdn-uploads.huggingface.co/production/uploads/66c26b6fb01b19d8c3c2467b/jg2NWmCUfPyzizm2USjMt.jpeg" |
| datasets: |
| - NewEden/Orion-LIT |
| - NewEden/Orion-Asstr-Stories-16K |
| - Mielikki/Erebus-87k |
| - NewEden/RP-logs-V2-Experimental-prefixed |
| - NewEden/Creative_Writing-Complexity |
| - NewEden/Discord-Filtered |
| - NewEden/DeepseekRP-Filtered |
| - NewEden/Storium-Prefixed-Clean |
| - NewEden/Basket-Weaving-Filtered |
| - NewEden/LIMARP-Complexity |
| - NewEden/Misc-Data-Sharegpt-Prefixed |
| - NewEden/BlueSky-10K-Complexity |
| - NewEden/OpenCAI-ShareGPT |
| - NewEden/Basket-Weaving-Filtered |
| - PocketDoc/Dans-Personamaxx-VN |
| - PocketDoc/Dans-Kinomaxx-VanillaBackrooms |
|
|
| base_model: |
| - Delta-Vector/Hamanasu-QwQ-V1.5-Instruct |
| tags: |
| - qwen |
| - roleplay |
| - finetune |
| - storywriting |
| --- |
| <!DOCTYPE html> |
| <style> |
| html, body { |
| background: black; |
| color: #c9d1d9 !important; |
| font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif; |
| margin: 0; |
| padding: 0; |
| min-height: 100vh; |
| } |
| .markdown-body { |
| color: white; |
| margin: 40px auto; |
| padding: 40px; |
| border-radius: 12px; |
| position: relative; |
| overflow: hidden; |
| } |
| |
| .markdown-body::after { |
| content: ''; |
| position: absolute; |
| top: 0; |
| left: 0; |
| width: 100%; |
| height: 100%; |
| background: #0c0f18; /* background color */ |
| pointer-events: none; |
| z-index: -999; |
| } |
| |
| h1, h2, h3 { |
| background: linear-gradient(45deg, #6e00ff, #00ffff); |
| -webkit-background-clip: text; |
| -webkit-text-fill-color: transparent; |
| border-bottom: 1px solid #333; |
| padding-bottom: 0.3em; |
| } |
| |
| div[style*="border:2px solid #333"], |
| div[style*="border: 2px solid #333"], |
| div[style*="border:1px solid #333"], |
| div[style*="border: 1px solid #333"] { |
| background: rgba(22, 27, 34, 0.8) !important; |
| border: 2px solid #6e00ff !important; |
| box-shadow: 0 0 15px rgba(110, 0, 255, 0.5); |
| border-radius: 10px; |
| padding: 20px; |
| margin: 20px 0; |
| } |
| |
| code { |
| background-color: #1a1a1a !important; |
| border-radius: 4px; |
| padding: 0.2em 0.4em; |
| color: #00ffff; |
| } |
| |
| pre { |
| background-color: #1a1a1a !important; |
| border: 1px solid #333; |
| border-radius: 8px; |
| padding: 16px; |
| } |
| |
| table { |
| width: 100%; |
| border-collapse: collapse; |
| margin: 20px 0; |
| background: rgba(0,0,0,0.2); |
| table-layout: fixed; |
| color: white; |
| } |
| |
| th, td { |
| border: 1px solid #333; |
| padding: 12px; |
| text-align: center; |
| color: white; |
| } |
| |
| th { |
| background: rgba(110, 0, 255, 0.1); |
| } |
| |
| td:nth-child(1) { |
| width: 1%; |
| white-space: nowrap; |
| } |
| |
| td:nth-child(2) { |
| width: 100%; |
| } |
| |
| td > span { |
| display: block; |
| padding: 4px 8px; |
| background: rgba(110, 0, 255, 0.1); |
| border-radius: 4px; |
| transition: all 0.3s ease; |
| } |
| |
| td > span:hover { |
| background: rgba(110, 0, 255, 0.2); |
| transform: translateY(-1px); |
| } |
| |
| a { |
| color: #00ffff; |
| text-decoration: none; |
| transition: all 0.3s ease; |
| } |
| |
| a:hover { |
| color: #6e00ff; |
| text-decoration: none; |
| } |
| |
| hr { |
| border: 0; |
| height: 1px; |
| background: linear-gradient(90deg, transparent, #333, transparent); |
| margin: 40px 0; |
| } |
| |
| img { |
| max-width: 100%; |
| border-radius: 10px; |
| } |
| |
| details summary:hover { |
| color: #00ffff; |
| } |
| |
| * { |
| color-scheme: dark !important; |
| } |
| |
| .prose, .max-w-none, .px-4 { |
| background-color: transparent !important; |
| color: #c9d1d9 !important; |
| } |
| </style> |
| <body> |
| <div class="markdown-body"> |
| <div align="center"> |
| |
| <img src="https://cdn-uploads.huggingface.co/production/uploads/66c26b6fb01b19d8c3c2467b/o5WjJKA9f95ri9UzRxZQE.png" alt="Model Visualization" width="500px" style="border: 3px solid #333; box-shadow: 0 0 15px rgba(66, 0, 131, 0.5);" /> |
| |
| <br> |
| <br> |
|
|
| <div style="font-size:1.5em; font-weight:bold; background: linear-gradient(45deg, #6e00ff, #00ffff); -webkit-background-clip: text; -webkit-text-fill-color: transparent;"> |
| Hamanasu 32B |
| </div> |
| |
| </div> |
|
|
| <div style="border:1px solid #333; border-radius:10px; padding:20px; margin:20px 0; background: rgba(0,0,0,0.4);"> |
|
|
|
|
| ## 🌌 Overview |
|
|
| <i>This model is the Chat tune of the Instruct model, More accurately it is the "brainrotted" version, Finetuned with Bsky, 4chan and Discord logs, Its... really something beautiful.</i> |
| <i>Read more about the model's training on my blog : https://openai-sucks.bearblog.dev/.</i> |
| <i>The model is suited best towards being a highly dumb chat partner rather then regular RP, All thanks to Ruka-Hamanasu for funding the train.</i> |
|
|
| </div> |
|
|
| <div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(250px, 1fr)); gap: 20px; margin: 20px 0;"> |
|
|
| <div style="border:2px solid #333; border-radius:10px; padding:20px; background: rgba(0,0,0,0.2);"> |
|
|
| ### 📈 Quantizations |
| | Type | Link | |
| |:---:|:---:| |
| | `GGUF` | <span style="display:block; width:100%">https://huggingface.co/mradermacher/Hamanasu-QwQ-V2-RP-GGUF</span> | |
| | `EXL2` | <span style="display:block; width:100%">https://huggingface.co/Delta-Vector/Hamanasu-QwQ-V2-RP-EXL2/</span> | |
|
|
| </div> |
|
|
| <div style="border:2px solid #333; border-radius:10px; padding:20px; background: rgba(0,0,0,0.2);"> |
|
|
| ### ⚔️ Hardware |
| - 8x H100s |
| - Epochs: 4 |
| - Base: `Delta-Vector/Hamanasu-QwQ-V1.5-Instruct` |
| - Amount of Tokens: 10M |
| </div> |
|
|
| </div> |
|
|
|
|
| <div style="border: 2px solid #6e00ff; border-radius: 10px; padding: 20px; margin: 20px 0; box-shadow: 0 0 15px rgba(110, 0, 255, 0.5);"> |
|
|
| ## 💰 Prompting |
|
|
|
|
| <i>This model uses ChatML formatting</i> |
| ```python |
| <|im_start|>system |
| You are an uncensored AI, your job is to fulfill thy will of thy user.<|im_end|> |
| <|im_start|>User request |
| Take off your helmet.<|im_end|> |
| <|im_start|>No i shall not. This is the way. |
| ``` |
| </div> |
|
|
| <div style="border: 2px solid #6e00ff; border-radius: 10px; padding: 20px; margin: 20px 0; box-shadow: 0 0 15px rgba(110, 0, 255, 0.5);"> |
|
|
| ## 🎲 Recommended Sampler Preset |
|
|
| ```python |
| temperature: 1.8 |
| min_p: 0.1 |
| System_Prompt: Keep blank for best chat experience. |
| ``` |
| </div> |
|
|
| <div style="border: 2px solid #6e00ff; border-radius: 10px; padding: 20px; margin: 20px 0; box-shadow: 0 0 15px rgba(110, 0, 255, 0.5);"> |
|
|
| ## Axolotl Config ꒰(˶• ᴗ •˶)꒱ |
|
|
| <details> |
| |
| ```yaml |
| base_model: NewEden/32B-inst |
| model_type: AutoModelForCausalLM |
| tokenizer_type: AutoTokenizer |
| |
| hub_model_id: NewEden/32b-rp |
| hub_strategy: "all_checkpoints" |
| push_dataset_to_hub: |
| hf_use_auth_token: true |
| |
| plugins: |
| - axolotl.integrations.liger.LigerPlugin |
| - axolotl.integrations.cut_cross_entropy.CutCrossEntropyPlugin |
| liger_rope: true |
| liger_rms_norm: true |
| liger_layer_norm: true |
| liger_glu_activation: true |
| liger_fused_linear_cross_entropy: false |
| cut_cross_entropy: true |
| |
| load_in_8bit: false |
| load_in_4bit: false |
| strict: false |
| |
| datasets: |
| - path: NewEden/RP-logs-V2-Experimental-prefixed |
| type: dan-chat-advanced |
| - path: NewEden/Creative_Writing-Complexity |
| type: dan-chat-advanced |
| - path: NewEden/Discord-Filtered |
| type: dan-chat-advanced |
| - path: NewEden/DeepseekRP-Filtered |
| type: dan-chat-advanced |
| - path: NewEden/Storium-Prefixed-Clean |
| type: dan-chat-advanced |
| - path: NewEden/Basket-Weaving-Filtered |
| type: dan-chat-advanced |
| - path: NewEden/LIMARP-Complexity |
| type: dan-chat-advanced |
| - path: NewEden/Misc-Data-Sharegpt-Prefixed |
| type: dan-chat-advanced |
| - path: NewEden/BlueSky-10K-Complexity |
| type: dan-chat-advanced |
| - path: NewEden/OpenCAI-ShareGPT |
| type: dan-chat-advanced |
| - path: NewEden/Basket-Weaving-Filtered |
| type: dan-chat-advanced |
| - path: PocketDoc/Dans-Personamaxx-VN |
| type: dan-chat-advanced |
| - path: PocketDoc/Dans-Kinomaxx-VanillaBackrooms |
| type: dan-chat-advanced |
| dataset_prepared_path: prepared_data |
| val_set_size: 0.0 |
| output_dir: ./qwq-inst |
| |
| sequence_len: 32768 |
| sample_packing: true |
| pad_to_sequence_len: true |
| |
| # adapter: lora |
| # lora_model_dir: |
| # lora_r: 128 |
| # lora_alpha: 16 |
| # lora_dropout: 0.05 |
| # lora_target_modules: |
| # - gate_proj |
| # - down_proj |
| # - up_proj |
| # - q_proj |
| # - v_proj |
| # - k_proj |
| # - o_proj |
| |
| wandb_project: qwq |
| wandb_entity: |
| wandb_watch: |
| wandb_name: rp-attempt-03 |
| wandb_log_model: |
| |
| gradient_accumulation_steps: 2 |
| micro_batch_size: 2 |
| num_epochs: 4 |
| optimizer: adamw_bnb_8bit |
| lr_scheduler: cosine |
| learning_rate: 2.5e-5 |
| max_grad_norm: 1.0 |
| |
| train_on_inputs: false |
| group_by_length: false |
| bf16: auto |
| fp16: |
| tf32: false |
| |
| gradient_checkpointing: unsloth |
| early_stopping_patience: |
| resume_from_checkpoint: |
| local_rank: |
| logging_steps: 1 |
| xformers_attention: |
| flash_attention: true |
| |
| warmup_steps: 40 |
| saves_per_epoch: 2 |
| debug: |
| deepspeed: deepspeed_configs/zero3_bf16.json |
| weight_decay: 0.02 |
| fsdp: |
| fsdp_config: |
| special_tokens: |
| ``` |
|
|
| </details> |
| </div> |
|
|
| <div align="center"> |
|
|
| <div style="border: 2px solid #6e00ff; border-radius: 10px; padding: 20px; margin: 20px 0; box-shadow: 0 0 15px rgba(110, 0, 255, 0.5);"> |
| |
| ## ⚡ Credits |
| <div style="display: flex; justify-content: center;"> |
| <div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(200px, 1fr)); gap: 10px; margin: 20px 0; max-width: 600px;"> |
| |
| <div style="border:1px solid #333; padding:10px; border-radius:5px; text-align:center; background: rgba(0,0,0,0.2); display: flex; align-items: center; justify-content: center;"> |
| <a href="https://huggingface.co/lucyknada"> |
| <img src="https://img.shields.io/badge/%F0%9F%8C%9F-Lucy_Knada-blueviolet" alt="Lucy Knada"> |
| </a> |
| </div> |
| |
| <div style="border:1px solid #333; padding:10px; border-radius:5px; text-align:center; background: rgba(0,0,0,0.2); display: flex; align-items: center; justify-content: center;"> |
| <a href="https://huggingface.co/hamanasu"> |
| <img src="https://img.shields.io/badge/%E2%9A%94%EF%B8%8F-jeiku-blueviolet" alt="Ruka"> |
| </a> |
| </div> |
| |
| <div style="border:1px solid #333; padding:10px; border-radius:5px; text-align:center; background: rgba(0,0,0,0.2); display: flex; align-items: center; justify-content: center;"> |
| <a href="https://huggingface.co/intervitens"> |
| <img src="https://img.shields.io/badge/%F0%9F%9B%A1%EF%B8%8F-Intervitens-blueviolet" alt="Intervitens"> |
| </a> |
| </div> |
| |
| <div style="border:1px solid #333; padding:10px; border-radius:5px; text-align:center; background: rgba(0,0,0,0.2); display: flex; align-items: center; justify-content: center;"> |
| <a href="https://huggingface.co/kalomaze"> |
| <img src="https://img.shields.io/badge/%F0%9F%94%AE-Kalomaze-blueviolet" alt="Kalomaze"> |
| </a> |
| </div> |
| |
| <div style="border:1px solid #333; padding:10px; border-radius:5px; text-align:center; background: rgba(0,0,0,0.2); display: flex; align-items: center; justify-content: center;"> |
| <a href="https://huggingface.co/kubernetes-bad"> |
| <img src="https://img.shields.io/badge/%E2%9A%A1-Kubernetes_Bad-blueviolet" alt="Kubernetes Bad"> |
| </a> |
| </div> |
| |
| <div style="border:1px solid #333; padding:10px; border-radius:5px; text-align:center; background: rgba(0,0,0,0.2); display: flex; align-items: center; justify-content: center;"> |
| <a href="https://huggingface.co/anthracite-org"> |
| <img src="https://img.shields.io/badge/%F0%9F%8C%91-Anthracite-blueviolet" alt="Anthracite"> |
| </a> |
| </div> |
| </div> |
| </div> |
| </div> |
| |
| --- |
|
|
| <div align="center"> |
| <div style="font-size:0.8em; opacity:0.8;">Made by</div> |
| <div style="font-size:1.2em; font-weight:bold; background: linear-gradient(45deg, #6e00ff, #00ffff); -webkit-background-clip: text; -webkit-text-fill-color: transparent;">Delta-Vector</div> |
| </div> |
|
|
| </div> |
| </body> |
| </html> |
|
|
