MD-Mushfiqur123 commited on
Commit
ff1b952
·
verified ·
1 Parent(s): 1869ca7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +97 -142
README.md CHANGED
@@ -1,183 +1,138 @@
1
- # Droplychee-40B
2
-
3
- <div align="center">
4
-
5
- # 🧠 Droplychee-40B
6
-
7
- **An open multilingual instruction model based on the Qwen family.**
8
-
9
- [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](#license)
10
- [![Status](https://img.shields.io/badge/Status-Under%20Development-orange)]()
11
- [![Language](https://img.shields.io/badge/Languages-Bangla%20%7C%20English-green)]()
12
-
13
- </div>
14
-
15
  ---
16
-
17
- ## Overview
18
-
19
- Droplychee-40B is a multilingual large language model developed by **Droplychee Core**. The model is built on the **Qwen** family through **full fine-tuning** with the goal of improving multilingual instruction following, reasoning, coding, and Bangla language capabilities.
20
-
21
- This release is intended for research and early evaluation while the project continues to evolve.
22
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
23
  ---
24
 
25
- ## Highlights
26
-
27
- * 🌍 Multilingual (Bangla & English)
28
- * 🧠 Instruction Following
29
- * 💻 Coding Assistance
30
- * ✨ General Reasoning
31
- * 📚 Research Friendly
32
- * 🔓 Open Development
33
 
34
- ---
35
-
36
- ## Model Details
37
 
38
- | Property | Value |
39
- | ----------------- | ------------------------ |
40
- | Model Name | Droplychee-40B |
41
- | Base Model | Qwen Family |
42
- | Training Method | Full Fine-Tuning |
43
- | Architecture | Decoder-only Transformer |
44
- | Primary Languages | Bangla, English |
45
- | Status | Under Development |
46
- | Organization | Droplychee Core |
47
 
48
- ---
49
 
50
- ## Intended Uses
51
 
52
- Droplychee-40B is designed for:
53
 
54
- * Conversational AI
55
- * Coding assistance
56
- * Multilingual reasoning
57
- * Bangla NLP
58
- * Education
59
- * Research
60
- * AI Agents
61
- * Document understanding
62
 
63
- ---
64
 
65
- ## Limitations
 
 
 
 
 
 
 
 
 
 
66
 
67
- This model is currently under active development.
68
 
69
- Users should expect:
70
 
71
- * Possible factual errors
72
- * Hallucinations
73
- * Incomplete reasoning
74
- * Benchmark results may change
75
- * Future releases may significantly improve quality
76
 
77
- ---
 
 
 
 
 
78
 
79
  ## Training
80
 
81
- Droplychee-40B was created through **full fine-tuning** of a model from the **Qwen family**.
82
-
83
- Additional training details will be documented in future releases.
84
-
85
- **TBD**
86
-
87
- * Dataset composition
88
- * Training tokens
89
- * Optimizer
90
- * Learning rate
91
- * Batch size
92
- * Epochs
93
- * Sequence length
94
 
95
- ---
 
 
 
 
 
96
 
97
- ## Evaluation
98
 
99
- Comprehensive benchmark results are not yet available.
 
 
 
 
 
100
 
101
- Future evaluations are planned for:
102
 
103
- * MMLU
104
- * GSM8K
105
- * HumanEval
106
- * MBPP
107
- * ARC
108
- * IFEval
109
- * BBH
110
- * Bangla-specific benchmarks
111
 
112
- ---
113
 
114
- ## Usage (Transformers)
115
 
116
- ```python
117
- from transformers import AutoTokenizer, AutoModelForCausalLM
118
 
119
- model_id = "droplychee/droplychee-40b"
120
 
121
- tokenizer = AutoTokenizer.from_pretrained(model_id)
122
 
123
- model = AutoModelForCausalLM.from_pretrained(
124
- model_id,
125
- torch_dtype="auto",
126
- device_map="auto"
127
- )
 
 
 
128
 
129
- messages = [
130
- {"role": "user", "content": "Explain recursion with a Python example."}
131
- ]
132
 
133
- text = tokenizer.apply_chat_template(
134
- messages,
135
- tokenize=False,
136
- add_generation_prompt=True
137
- )
138
 
139
- inputs = tokenizer(text, return_tensors="pt").to(model.device)
140
 
141
- outputs = model.generate(
142
- **inputs,
143
- max_new_tokens=512
144
- )
145
 
146
- print(tokenizer.decode(outputs[0], skip_special_tokens=True))
147
- ```
148
 
149
- ---
150
 
151
- ## Citation
152
 
153
- ```bibtex
154
- @misc{droplychee40b2026,
155
- title={Droplychee-40B},
156
- author={Droplychee Core},
157
- year={2026},
158
- publisher={Hugging Face},
159
- howpublished={\url{https://huggingface.co/droplychee}}
160
- }
161
- ```
162
-
163
- ---
164
 
165
  ## License
166
 
167
- **TBD**
168
-
169
- (Choose the appropriate license before the stable release.)
170
-
171
- ---
172
-
173
- ## Acknowledgements
174
-
175
- Droplychee-40B builds upon the open-source AI ecosystem. We acknowledge the contributions of the Qwen team and the broader machine learning community.
176
-
177
- ---
178
-
179
- ## Contact
180
 
181
- * Organization: **Droplychee Core**
182
- * GitHub: https://github.com/DropLychee
183
- * Hugging Face: https://huggingface.co/droplychee
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: apache-2.0
3
+ base_model:
4
+ - DavidAU/Qwen3.6-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-Thinking
5
+ library_name: transformers
6
+ pipeline_tag: text-generation
7
+ tags:
8
+ - text-generation
9
+ - chat
10
+ - multilingual
11
+ - reasoning
12
+ - coding
13
+ - qwen
14
+ - yarn
15
+ - long-context
16
+ - 1m-context
17
+ - bangla
18
+ language:
19
+ - en
20
+ - bn
21
+ - multilingual
22
+ license_name: apache-2.0
23
  ---
24
 
25
+ # 🧠 Droplychee-2.0-40B
 
 
 
 
 
 
 
26
 
27
+ > A multilingual long-context language model developed by **Droplychee**, built upon the Qwen family and extended through full fine-tuning and architectural modifications.
 
 
28
 
29
+ ## Overview
 
 
 
 
 
 
 
 
30
 
31
+ Droplychee-2.0-40B is a decoder-only transformer model designed for multilingual reasoning, software engineering, long-context understanding, and instruction following. The model is derived from the Qwen family and further adapted through full fine-tuning using curated multilingual instruction datasets.
32
 
33
+ The project focuses on delivering strong performance in English and Bangla while maintaining support for over 40 languages.
34
 
35
+ ## Key Features
36
 
37
+ - 🌍 Supports 40+ languages
38
+ - 📚 Maximum context length of 1,000,000 tokens
39
+ - 🧵 YaRN-based context extension
40
+ - 💻 Optimized for software engineering and coding tasks
41
+ - 🧠 Advanced reasoning and instruction following
42
+ - 📄 Long-document analysis and summarization
43
+ - 🤖 Agent-oriented workflows and tool use
 
44
 
45
+ ## Model Details
46
 
47
+ | Property | Value |
48
+ |----------|-------|
49
+ | Model Name | Droplychee-2.0-40B |
50
+ | Organization | Droplychee |
51
+ | Base Model | DavidAU/Qwen3.6-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-Thinking |
52
+ | Architecture | Decoder-only Transformer |
53
+ | Training Method | Full Fine-Tuning |
54
+ | Context Length | 1,000,000 tokens |
55
+ | Context Extension | YaRN |
56
+ | Languages | 40+ |
57
+ | License | Apache-2.0 |
58
 
59
+ ## Architecture
60
 
61
+ Droplychee-2.0-40B is derived from the Qwen architecture with additional modifications introduced during development.
62
 
63
+ Key architectural characteristics include:
 
 
 
 
64
 
65
+ - Decoder-only transformer architecture
66
+ - Rotary Position Embeddings (RoPE)
67
+ - YaRN-based long-context extension
68
+ - Optimized KV-cache handling
69
+ - Flash Attention support (where available in the serving stack)
70
+ - Full supervised instruction fine-tuning
71
 
72
  ## Training
73
 
74
+ The model was fine-tuned using a curated multilingual instruction corpus emphasizing:
 
 
 
 
 
 
 
 
 
 
 
 
75
 
76
+ - General reasoning
77
+ - Coding and software engineering
78
+ - Mathematics
79
+ - Multilingual dialogue
80
+ - Long-context comprehension
81
+ - Agent-oriented tasks
82
 
83
+ Approximate training statistics:
84
 
85
+ | Item | Value |
86
+ |------|-------|
87
+ | High-quality instruction pairs | ~500K |
88
+ | Training tokens | ~100M–1B |
89
+ | Training Platform | Unsloth Studio |
90
+ | GPU Hardware | 3× NVIDIA RTX PRO 6000 Blackwell Server Edition |
91
 
92
+ ## Context Extension
93
 
94
+ Droplychee-2.0-40B supports a maximum context length of **1,000,000 tokens** through **YaRN (Yet another RoPE extensioN)**.
 
 
 
 
 
 
 
95
 
96
+ YaRN extends the effective context window while preserving compatibility with Rotary Position Embeddings. Long-context performance may vary depending on the inference engine, hardware resources, and serving configuration.
97
 
98
+ ## Supported Languages
99
 
100
+ The model is optimized for multilingual instruction following and has been evaluated primarily on English and Bangla. It also supports more than 40 languages, including Hindi, Arabic, Chinese, Japanese, Korean, French, German, Spanish, Portuguese, and Russian.
 
101
 
102
+ ## Intended Use
103
 
104
+ Recommended applications include:
105
 
106
+ - Conversational AI
107
+ - Coding assistants
108
+ - Research assistance
109
+ - Educational tools
110
+ - Long-document summarization
111
+ - Retrieval-Augmented Generation (RAG)
112
+ - AI agents
113
+ - Software engineering workflows
114
 
115
+ ## Limitations
 
 
116
 
117
+ - Outputs may contain factual inaccuracies or hallucinations.
118
+ - Performance varies across languages and domains.
119
+ - Independent third-party evaluation has not yet been completed.
120
+ - Users should verify outputs before relying on them for high-impact decisions.
 
121
 
122
+ ## Responsible Use
123
 
124
+ This model is intended for research and general-purpose AI applications. Users are responsible for ensuring compliance with applicable laws, regulations, and ethical guidelines.
 
 
 
125
 
126
+ ## Attribution
 
127
 
128
+ This project is derived from:
129
 
130
+ **DavidAU/Qwen3.6-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-Thinking**
131
 
132
+ Please refer to the original model card and license for upstream attribution requirements.
 
 
 
 
 
 
 
 
 
 
133
 
134
  ## License
135
 
136
+ This model is distributed under the **Apache License 2.0**.
 
 
 
 
 
 
 
 
 
 
 
 
137
 
138
+ Users must also comply with the license terms of the upstream base model where applicable.