uaytug commited on
Commit
e0ec8ee
·
verified ·
1 Parent(s): bf9c34b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +161 -3
README.md CHANGED
@@ -1,3 +1,161 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ tags:
6
+ - code
7
+ - programming
8
+ - mathematics
9
+ - reasoning
10
+ - text-generation
11
+ - conversational
12
+ pipeline_tag: text-generation
13
+ library_name: transformers
14
+ datasets:
15
+ - uaytug/UCDS
16
+ model-index:
17
+ - name: ucoder-mini
18
+ results: []
19
+ ---
20
+
21
+ # uCoder Mini
22
+
23
+ <div align="center">
24
+ <img src="https://img.shields.io/badge/Parameters-1.5B-blue" alt="Parameters">
25
+ <img src="https://img.shields.io/badge/Context-4096-green" alt="Context Length">
26
+ <img src="https://img.shields.io/badge/License-Apache%202.0-orange" alt="License">
27
+ </div>
28
+
29
+ ## Overview
30
+
31
+ **uCoder Mini** is a compact 1.5B parameter language model fine-tuned for code generation and mathematical reasoning. Despite its small size, it delivers strong performance on programming tasks across multiple languages and competitive programming challenges.
32
+
33
+ Trained on the [UCDS (uCoder Dataset)](https://huggingface.co/datasets/uaytug/UCDS) — a curated collection of 420K+ high-quality coding and mathematics samples.
34
+
35
+ ## Intended Use
36
+
37
+ - **Code generation** across Python, JavaScript, C++, Java, and more
38
+ - **Competitive programming** problem solving
39
+ - **Mathematical reasoning** and problem breakdown
40
+ - **Code explanation** and debugging assistance
41
+ - **Learning companion** for programming concepts
42
+
43
+ ## Quick Start
44
+
45
+ ```python
46
+ from transformers import AutoModelForCausalLM, AutoTokenizer
47
+
48
+ model_id = "uaytug/ucoder-mini"
49
+
50
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
51
+ model = AutoModelForCausalLM.from_pretrained(
52
+ model_id,
53
+ torch_dtype="auto",
54
+ device_map="auto"
55
+ )
56
+
57
+ messages = [
58
+ {"role": "user", "content": "Write a Python function to find the longest palindromic substring."}
59
+ ]
60
+
61
+ text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
62
+ inputs = tokenizer(text, return_tensors="pt").to(model.device)
63
+
64
+ outputs = model.generate(
65
+ **inputs,
66
+ max_new_tokens=512,
67
+ temperature=0.7,
68
+ top_p=0.9,
69
+ do_sample=True
70
+ )
71
+
72
+ response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
73
+ print(response)
74
+ ```
75
+
76
+ ## Chat Template
77
+
78
+ uCoder Mini uses the ChatML format:
79
+
80
+ ```
81
+ <|im_start|>user
82
+ Your question here<|im_end|>
83
+ <|im_start|>assistant
84
+ ```
85
+
86
+ ## Training Details
87
+
88
+ | Attribute | Value |
89
+ |-----------|-------|
90
+ | **Training Dataset** | [uaytug/UCDS](https://huggingface.co/datasets/uaytug/UCDS) |
91
+ | **Dataset Size** | 420,686 samples |
92
+ | **Training Method** | Supervised Fine-Tuning (SFT) |
93
+ | **Precision** | bfloat16 |
94
+ | **Context Length** | 4096 tokens |
95
+
96
+ ### Dataset Composition
97
+
98
+ The UCDS dataset combines high-quality sources:
99
+
100
+ | Source | Description |
101
+ |--------|-------------|
102
+ | CodeForces | Competitive programming problems |
103
+ | Code-Contests-Plus | Algorithm challenges with solutions |
104
+ | CodeAlpaca | Instruction-following code samples |
105
+ | OpenMathInstruct | Mathematical reasoning chains |
106
+ | Additional curated sources | Tool use, chain-of-thought examples |
107
+
108
+ ## Limitations
109
+
110
+ - **Model size**: As a 1.5B model, it may struggle with highly complex multi-file projects
111
+ - **Knowledge cutoff**: Training data has a knowledge cutoff; recent libraries/APIs may not be covered
112
+ - **Verification needed**: Always review and test generated code before production use
113
+ - **Language coverage**: Strongest in Python; other languages may have reduced quality
114
+
115
+ ## Example Outputs
116
+
117
+ <details>
118
+ <summary><b>Competitive Programming</b></summary>
119
+
120
+ **Prompt:** Solve the two-sum problem efficiently.
121
+
122
+ **Response:**
123
+ ```python
124
+ def two_sum(nums, target):
125
+ seen = {}
126
+ for i, num in enumerate(nums):
127
+ complement = target - num
128
+ if complement in seen:
129
+ return [seen[complement], i]
130
+ seen[num] = i
131
+ return []
132
+ ```
133
+
134
+ </details>
135
+
136
+ <details>
137
+ <summary><b>Mathematical Reasoning</b></summary>
138
+
139
+ **Prompt:** Find the time complexity of merge sort and explain why.
140
+
141
+ **Response:** Merge sort has O(n log n) time complexity. Here's why:
142
+
143
+ 1. **Divide step**: The array is split in half recursively, creating log(n) levels
144
+ 2. **Merge step**: At each level, we process all n elements exactly once
145
+ 3. **Total**: n elements × log(n) levels = O(n log n)
146
+
147
+ This holds for best, average, and worst cases, making merge sort consistently efficient.
148
+
149
+ </details>
150
+
151
+ ## Citation
152
+
153
+ ```bibtex
154
+ @misc{ucoder-mini,
155
+ author = {uaytug},
156
+ title = {uCoder Mini: A Compact Code and Math Language Model},
157
+ year = {2025},
158
+ publisher = {Hugging Face},
159
+ url = {https://huggingface.co/uaytug/ucoder-mini}
160
+ }
161
+ ```