Safetensors
English
qwen2
sliuau commited on
Commit
0c2562b
·
verified ·
1 Parent(s): f01ef50

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +112 -0
README.md CHANGED
@@ -0,0 +1,112 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Model Overview
2
+ ### Description:
3
+ DLER-Qwen-R1-7B is an ultra-efficient 7B open-weight reasoning model designed for challenging tasks such as mathematics, programming, and scientific problem-solving. It is trained with the DLER algorithm on agentica-org/DeepScaleR-Preview-Dataset. Compared to DeepSeek’s 7B model, DLER-Qwen-R1-7B achieves substantial efficiency gains, reducing the average response length by nearly 80% across diverse mathematical benchmarks with better accuracy.
4
+
5
+ This model is for research and development only.
6
+
7
+ ### Deployment Geography:
8
+ Global <br>
9
+
10
+ ### Use Case: <br>
11
+ Researchers and developers can use this model to solve math, coding, and STEM questions.
12
+
13
+ ### Release Date: <br>
14
+ Hugging Face 9/10/2025 via https://huggingface.co/nvidia/DLER-R1-7B <br>
15
+
16
+
17
+ ## Model Architecture:
18
+ **Architecture Type:** Dense decoder-only Transformer model <br>
19
+
20
+ **Network Architecture:** [DeepSeek-R1-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B) <br>
21
+
22
+ **This model was developed based on DeepSeek-R1-7B <br>
23
+
24
+ ## Software Integration:
25
+ **Runtime Engine(s):** Transformers
26
+
27
+ **Supported Hardware Microarchitecture Compatibility:** <br>
28
+ * NVIDIA Ampere <br>
29
+ * NVIDIA Hopper <br>
30
+
31
+
32
+ **Preferred/Supported Operating System(s):**
33
+ * Linux <br>
34
+
35
+ The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.
36
+
37
+ ## Model Version(s):
38
+ 1.0
39
+
40
+
41
+ ### Training Dataset:
42
+
43
+ | Dataset | Link |
44
+ |---------------------------|-------------------------------------------------------------------------------------------|
45
+ | DeepScaleR-Preview-Dataset | [Link](https://huggingface.co/datasets/agentica-org/DeepScaleR-Preview-Dataset) |
46
+
47
+ **Properties:** 479K question and answer pairs <br>
48
+
49
+ ### Evaluation Results:
50
+ **Benchmark Score <br>
51
+
52
+ | Model | Math 500 | AVG Length | AIME | # Tokens | AMC | # Tokens | Minerva | # Tokens | Olympiad | # Tokens | Avg # Tokens |
53
+ |------------------|----------|------------|--------------------|------------------|--------------------|------------------|--------------------|------------------|--------------------|------------------|-----------------|
54
+ | Deepseek-R1-7B | 93.60 | 3999 | 55.40 | 13241 | 82.90 | 7461 | 49.79 | 5199 | 58.21 | 8837 | 7747 |
55
+ | **DLER-R1-7B** | **94.21 (+0.61%)** | **1634 (-60%)** | **55.62 (+0.22%)** | **3230 (-76%)** | **84.41 (+1.51%)** | **2512 (-0.67%)** | **53.88 (+4.09%)** | **2058 (-61%)** | **60.48 (+2.27%)** | **2592 (-71%)** | **2405 (-69%)** |
56
+
57
+
58
+ # Inference:
59
+ ```python
60
+ from transformers import AutoTokenizer, AutoModelForCausalLM
61
+ import torch
62
+
63
+ device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
64
+
65
+ model = AutoModelForCausalLM.from_pretrained('nvidia/DLER-R1-7B').to(device)
66
+ tokenizer = AutoTokenizer.from_pretrained('nvidia/DLER-R1-7B')
67
+
68
+ messages = [
69
+ {"role": "user", "content": "Convert the point $(0,3)$ in rectangular coordinates to polar coordinates. Enter your answer in the form $(r,\\theta),$ where $r > 0$ and $0 \\le \\theta < 2 \\pi.$"+" Let's think step by step and output the final answer within \\boxed{}."},
70
+ ]
71
+
72
+
73
+ tokenized_chat = tokenizer.apply_chat_template(
74
+ messages,
75
+ tokenize=True,
76
+ add_generation_prompt=True,
77
+ return_tensors="pt"
78
+ ).to(model.device)
79
+
80
+ outputs = model.generate(
81
+ tokenized_chat,
82
+ max_new_tokens=10000,
83
+ eos_token_id=tokenizer.eos_token_id
84
+ )
85
+
86
+ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
87
+ ```
88
+
89
+ ### License/Terms of Use
90
+ TBD
91
+ https://docs.google.com/spreadsheets/d/15AiIBHLsm-HY1RZH5nkaA0siE-5grHke9uFYaiD_28E/edit?gid=1088371820#gid=1088371820
92
+
93
+ ## Ethical Considerations:
94
+ NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
95
+
96
+ Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).
97
+
98
+
99
+ ## Citation
100
+ If you find our dataset helpful, please cite the following [paper]():
101
+
102
+
103
+
104
+
105
+ ```
106
+
107
+
108
+
109
+
110
+ ```
111
+
112
+