prasadsachin commited on
Commit
0ec79c8
·
verified ·
1 Parent(s): fc84ec8

Update README.md with new model card content

Browse files
Files changed (1) hide show
  1. README.md +81 -22
README.md CHANGED
@@ -2,25 +2,84 @@
2
  library_name: keras-hub
3
  pipeline_tag: text-generation
4
  ---
5
- This is a [`Qwen` model](https://keras.io/api/keras_hub/models/qwen) uploaded using the KerasHub library and can be used with JAX, TensorFlow, and PyTorch backends.
6
- This model is related to a `CausalLM` task.
7
-
8
- Model config:
9
- * **name:** qwen_backbone
10
- * **trainable:** True
11
- * **dtype:** {'module': 'keras', 'class_name': 'DTypePolicy', 'config': {'name': 'float32'}, 'registered_name': None}
12
- * **vocabulary_size:** 152064
13
- * **num_layers:** 64
14
- * **num_query_heads:** 40
15
- * **hidden_dim:** 5120
16
- * **intermediate_dim:** 27648
17
- * **rope_max_wavelength:** 1000000.0
18
- * **rope_scaling_factor:** 1.0
19
- * **num_key_value_heads:** 8
20
- * **layer_norm_epsilon:** 1e-06
21
- * **dropout:** 0
22
- * **tie_word_embeddings:** False
23
- * **use_sliding_window_attention:** False
24
- * **sliding_window_size:** 131072
25
-
26
- This model card has been generated automatically and should be completed by the model author. See [Model Cards documentation](https://huggingface.co/docs/hub/model-cards) for more information.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  library_name: keras-hub
3
  pipeline_tag: text-generation
4
  ---
5
+ ### Model Overview
6
+ # Model Summary
7
+
8
+ Qwen2.5-Coder is the latest series of Code-Specific Qwen large language models (formerly known as CodeQwen). As of now, Qwen2.5-Coder has covered six mainstream model sizes, **0.5, 1.5, 3, 7, 14, 32** billion parameters, to meet the needs of different developers. Qwen2.5-Coder brings the following improvements upon CodeQwen1.5:
9
+
10
+ Significantly improvements in **code generation**, **code reasoning** and **code fixing**. Base on the strong Qwen2.5, we scale up the training tokens into 5.5 trillion including source code, text-code grounding, Synthetic data, etc. Qwen2.5-Coder-32B has become the current state-of-the-art open-source codeLLM, with its coding abilities matching those of GPT-4o.
11
+
12
+ A more comprehensive foundation for real-world applications such as **Code Agents**. Not only enhancing coding capabilities but also maintaining its strengths in mathematics and general competencies.
13
+
14
+ **Long-context Support** up to 128K tokens.
15
+
16
+ For more details, please refer to Qwen [Blog](https://qwenlm.github.io/blog/qwen2.5/), [GitHub](https://github.com/keras-team/keras-hub/tree/master/keras_hub/src/models/qwen), and [Documentation](https://qwen.readthedocs.io/en/latest/).
17
+
18
+ Weights are released under the [Apache 2 License](https://github.com/keras-team/keras-hub/blob/master/LICENSE) . Keras model code is released under the [Apache 2 License](https://github.com/keras-team/keras-hub/blob/master/LICENSE).
19
+
20
+ ## Links
21
+
22
+ * [Qwen 2.5 Coder Quickstart Notebook](https://www.kaggle.com/code/laxmareddypatlolla/qwen2-5-coder-quickstart-notebook)
23
+ * [Qwen 2.5 Coder API Documentation](https://keras.io/keras_hub/api/models/qwen/)
24
+ * [Qwen 2.5 Coder Model Card](https://qwenlm.github.io/blog/qwen2.5/)
25
+ * [KerasHub Beginner Guide](https://keras.io/guides/keras_hub/getting_started/)
26
+ * [KerasHub Model Publishing Guide](https://keras.io/guides/keras_hub/upload/)
27
+
28
+ ## Installation
29
+
30
+ Keras and KerasHub can be installed with:
31
+
32
+ ```
33
+ pip install -U -q keras-hub
34
+ pip install -U -q keras
35
+ ```
36
+
37
+ Jax, TensorFlow, and Torch come preinstalled in Kaggle Notebooks. For instructions on installing them in another environment see the [Keras Getting Started](https://keras.io/getting_started/) page.
38
+
39
+ ## Presets
40
+
41
+ The following model checkpoints are provided by the Keras team. Full code examples for each are available below.
42
+
43
+ | Preset name | Parameters | Description |
44
+ |---------------------------------------|------------|--------------------------------------------------------------------------------------------------------------|
45
+ | qwen2.5_coder_0.5b | 0.5B | 24-layer with 0.5 billion parameters. Code-Specific large language models base on the strong Qwen2.5 |
46
+ | qwen2.5_coder_instruct_0.5b | 0.5B | 24-layer with 0.5 billion parameters. Code-Specific large language models base on the strong Qwen2.5. |
47
+ | qwen2.5_coder_1.5b | 1.5B | 28-layer with 1.5 billion parameters. Code-Specific large language models base on the strong Qwen2.5. |
48
+ | qwen2.5_coder_instruct_1.5b | 1.5B | 28-layer with 1.5 billion parameters. Code-Specific large language models base on the strong Qwen2.5.|
49
+ | qwen2.5_coder_3b | 3B | 36-layer with 3 billion parameters. Code-Specific large language models base on the strong Qwen2.5.|
50
+ | qwen2.5_coder_instruct_3b | 3B | 36-layer with 3 billion parameters. Code-Specific large language models base on the strong Qwen2.5. |
51
+ | qwen2.5_coder_7b | 7B | 28-layer with 7B billion parameters. Code-Specific large language models base on the strong Qwen2.5.|
52
+ | qwen2.5_coder_instruct_7b | 7B | 28-layer with 7B billion parameters. Code-Specific large language models base on the strong Qwen2.5.|
53
+ | qwen2.5_coder_14b | 14B | 48-layer with 14B billion parameters. Code-Specific large language models base on the strong Qwen2.5.|
54
+ | qwen2.5_coder_instruct_14b | 14B | 48-layer with 14B billion parameters. Code-Specific large language models base on the strong Qwen2.5.|
55
+ | qwen2.5_coder_32b | 32B | 64-layer with 32B billion parameters. Code-Specific large language models base on the strong Qwen2.5.|
56
+ | qwen2.5_coder_instruct_32b | 32B | 64-layer with 32B billion parameters. Code-Specific large language models base on the strong Qwen2.5.|
57
+
58
+ ## Example Usage
59
+ ```Python
60
+
61
+ import keras
62
+ import keras_hub
63
+ import numpy as np
64
+
65
+ # Use generate() to do code generation.
66
+ qwen_lm = keras_hub.models.QwenCausalLM.from_preset("qwen2.5_coder_32b")
67
+ qwen_lm.generate(" write a quick sort algorithm in python.", max_length=512)
68
+
69
+
70
+ ```
71
+
72
+ ## Example Usage with Hugging Face URI
73
+
74
+ ```Python
75
+
76
+ import keras
77
+ import keras_hub
78
+ import numpy as np
79
+
80
+ # Use generate() to do code generation.
81
+ qwen_lm = keras_hub.models.QwenCausalLM.from_preset("hf://keras/qwen2.5_coder_32b")
82
+ qwen_lm.generate(" write a quick sort algorithm in python.", max_length=512)
83
+
84
+
85
+ ```