Hugging Face's logo

Biscotto58
/
MistralNemoDionysus

Safetensors
GGUF
Model card Files
xet
 Community
Biscotto58 commited on
Commit
63031a3
·
verified ·
1 Parent(s): f0715b2

Upload Full Merged Model

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. model/README.md +47 -0
  2. model/config.json +26 -0
  3. model/mergekit_config.yml +16 -0
  4. model/merges.txt +0 -0
  5. model/model-00001-of-00052.safetensors +3 -0
  6. model/model-00002-of-00052.safetensors +3 -0
  7. model/model-00003-of-00052.safetensors +3 -0
  8. model/model-00004-of-00052.safetensors +3 -0
  9. model/model-00005-of-00052.safetensors +3 -0
  10. model/model-00006-of-00052.safetensors +3 -0
  11. model/model-00007-of-00052.safetensors +3 -0
  12. model/model-00008-of-00052.safetensors +3 -0
  13. model/model-00009-of-00052.safetensors +3 -0
  14. model/model-00010-of-00052.safetensors +3 -0
  15. model/model-00011-of-00052.safetensors +3 -0
  16. model/model-00012-of-00052.safetensors +3 -0
  17. model/model-00013-of-00052.safetensors +3 -0
  18. model/model-00014-of-00052.safetensors +3 -0
  19. model/model-00015-of-00052.safetensors +3 -0
  20. model/model-00016-of-00052.safetensors +3 -0
  21. model/model-00017-of-00052.safetensors +3 -0
  22. model/model-00018-of-00052.safetensors +3 -0
  23. model/model-00019-of-00052.safetensors +3 -0
  24. model/model-00020-of-00052.safetensors +3 -0
  25. model/model-00021-of-00052.safetensors +3 -0
  26. model/model-00022-of-00052.safetensors +3 -0
  27. model/model-00023-of-00052.safetensors +3 -0
  28. model/model-00024-of-00052.safetensors +3 -0
  29. model/model-00025-of-00052.safetensors +3 -0
  30. model/model-00026-of-00052.safetensors +3 -0
  31. model/model-00027-of-00052.safetensors +3 -0
  32. model/model-00028-of-00052.safetensors +3 -0
  33. model/model-00029-of-00052.safetensors +3 -0
  34. model/model-00030-of-00052.safetensors +3 -0
  35. model/model-00031-of-00052.safetensors +3 -0
  36. model/model-00032-of-00052.safetensors +3 -0
  37. model/model-00033-of-00052.safetensors +3 -0
  38. model/model-00034-of-00052.safetensors +3 -0
  39. model/model-00035-of-00052.safetensors +3 -0
  40. model/model-00036-of-00052.safetensors +3 -0
  41. model/model-00037-of-00052.safetensors +3 -0
  42. model/model-00038-of-00052.safetensors +3 -0
  43. model/model-00039-of-00052.safetensors +3 -0
  44. model/model-00040-of-00052.safetensors +3 -0
  45. model/model-00041-of-00052.safetensors +3 -0
  46. model/model-00042-of-00052.safetensors +3 -0
  47. model/model-00043-of-00052.safetensors +3 -0
  48. model/model-00044-of-00052.safetensors +3 -0
  49. model/model-00045-of-00052.safetensors +3 -0
  50. model/model-00046-of-00052.safetensors +3 -0
model/README.md ADDED
@@ -0,0 +1,47 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - mistralai/Mistral-Nemo-Base-2407
4
+ library_name: transformers
5
+ tags:
6
+ - mergekit
7
+ - merge
8
+
9
+ ---
10
+ # final_merge_output
11
+
12
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
13
+
14
+ ## Merge Details
15
+ ### Merge Method
16
+
17
+ This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [mistralai/Mistral-Nemo-Base-2407](https://huggingface.co/mistralai/Mistral-Nemo-Base-2407) as a base.
18
+
19
+ ### Models Merged
20
+
21
+ The following models were included in the merge:
22
+ * /teamspace/studios/this_studio/work_dir/intermediate_stages/merge_0
23
+ * /teamspace/studios/this_studio/work_dir/intermediate_stages/merge_5
24
+
25
+ ### Configuration
26
+
27
+ The following YAML configuration was used to produce this model:
28
+
29
+ ```yaml
30
+ base_model: mistralai/Mistral-Nemo-Base-2407
31
+ dtype: bfloat16
32
+ merge_method: ties
33
+ models:
34
+ - model: /teamspace/studios/this_studio/work_dir/intermediate_stages/merge_5
35
+ parameters:
36
+ density: 0.9
37
+ weight: 0.6
38
+ - model: /teamspace/studios/this_studio/work_dir/intermediate_stages/merge_0
39
+ parameters:
40
+ density: 0.9
41
+ weight: 0.4
42
+ name: Dionysus-12B
43
+ parameters:
44
+ int8_mask: false
45
+ normalize: true
46
+
47
+ ```
model/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "MistralForCausalLM"
4
+ ],
5
+ "attention_dropout": 0.0,
6
+ "bos_token_id": 1,
7
+ "dtype": "bfloat16",
8
+ "eos_token_id": 2,
9
+ "head_dim": 128,
10
+ "hidden_act": "silu",
11
+ "hidden_size": 5120,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 14336,
14
+ "max_position_embeddings": 131072,
15
+ "model_type": "mistral",
16
+ "num_attention_heads": 32,
17
+ "num_hidden_layers": 40,
18
+ "num_key_value_heads": 8,
19
+ "rms_norm_eps": 1e-05,
20
+ "rope_theta": 1000000.0,
21
+ "sliding_window": null,
22
+ "tie_word_embeddings": false,
23
+ "transformers_version": "4.57.3",
24
+ "use_cache": true,
25
+ "vocab_size": 131072
26
+ }
model/mergekit_config.yml ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ base_model: mistralai/Mistral-Nemo-Base-2407
2
+ dtype: bfloat16
3
+ merge_method: ties
4
+ models:
5
+ - model: /teamspace/studios/this_studio/work_dir/intermediate_stages/merge_5
6
+ parameters:
7
+ density: 0.9
8
+ weight: 0.6
9
+ - model: /teamspace/studios/this_studio/work_dir/intermediate_stages/merge_0
10
+ parameters:
11
+ density: 0.9
12
+ weight: 0.4
13
+ name: Dionysus-12B
14
+ parameters:
15
+ int8_mask: false
16
+ normalize: true
model/merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
model/model-00001-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:40b0f748512450ba88d1db11d77703fcb517a7fd5aad9b032f812b06a566298b
3
+ size 1342177408
model/model-00002-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a47f989099d71bca8e59108f38d4ec6fdd40a7a019552f4297bc7d811984b97b
3
+ size 1342177424
model/model-00003-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a810071f253466875cb6a44e965a1d4d0f31ce39a061baa67844f5fd941a3f4b
3
+ size 492852024
model/model-00004-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3ed8a8d8caa709cc363ff6cb14a4f7fde55cc8c75df4335622b1048050355354
3
+ size 503337896
model/model-00005-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:738b70adde6d44f1db647a52154bf3563831bd3f5c2f6e5017fa9ef14c509b97
3
+ size 387984072
model/model-00006-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b5067f821010ff789a184a437b4c2f885a0d0b9ac411071d38ae30ff7ca2ac55
3
+ size 398480312
model/model-00007-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f00a3d70a96fbc384a5efb71b8455893989c584489cbcfbf9d049488c3e89532
3
+ size 398480312
model/model-00008-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:51d593b46b9c6ad933f3015efb3d410669d3bdebe77384304e8f10289858d3dd
3
+ size 492841688
model/model-00009-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:574cb3d0953c434617b31f8fa300460edd7a9d9bfae6d4341c39a31632f0d615
3
+ size 503337904
model/model-00010-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:188f8bd98c94da096aa04d42abfb280db5765de023d4b52e310739565b376cf6
3
+ size 387984072
model/model-00011-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:805796ff0c7cc9e14265c40250fe270fcccce85f6666b1034a75483163d0640b
3
+ size 398480312
model/model-00012-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fa2815c797c87ee518155fd80a3e850f5e81af76f7476d02d3c01258b5634797
3
+ size 398480312
model/model-00013-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0a6b2261329cafb1445007f8efead3f46216bba2d4f3118ec8fd980c9bb83373
3
+ size 492841688
model/model-00014-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9c300c32474a203964fc4764c6d51e1a6c0fb53fde0d5b975f7827a8c52dbbb9
3
+ size 503337904
model/model-00015-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:36a79e2fbbf8c0f4531e0a4e7dada312f2f440342a9831fe529eaaa5c97c8173
3
+ size 387984072
model/model-00016-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1ed99a3152a28498c9609842ebaf784dfde02e91e3483583b13dc87f3f10e841
3
+ size 398480312
model/model-00017-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6795c4b9f01c06fcafb0f45953deb701838df6c7f3f6967d3c34fc07cc3a04f7
3
+ size 398480312
model/model-00018-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:873a3f4c8c110b54c82f81d56ae954498a13338cff99c3e668ac3207c3cfde7c
3
+ size 492841680
model/model-00019-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2e81a74219433c19a1e6cad62d3a04f90564c4109b0ab816ac4d13d8f43446e4
3
+ size 503337904
model/model-00020-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:38c0ad78ce317aac893a9d69cad871f70dfe954566704acaa6deabc29119d21f
3
+ size 387984072
model/model-00021-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cf8e3eeb757658eb7f7bd5e1aadffac0eb06ea119b9b8c38dd272d3858a1b9a6
3
+ size 398480312
model/model-00022-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d80670dcd68e6abce90c1773e3cb222df31e9b30def5ef88528b71a154d26090
3
+ size 398480312
model/model-00023-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6bbbad74c4225674b89929616b8a5f913a0d0666da65ff7c167c05b9165c9d47
3
+ size 492841688
model/model-00024-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:90340582a9d9ae9de0f605809f7e39a5bf3b63099c76dac21599df2cf265d7b1
3
+ size 503337904
model/model-00025-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:86f8015eedaa6d84860e8271396e2b62dd5385dba90b59fcd9ca7b77709ea0bc
3
+ size 387984072
model/model-00026-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:25a60a2ee5b9bea97a1d1640b39f063247559e81fabf181ce3c4b4f6183b4660
3
+ size 398480312
model/model-00027-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ef564654874ea607ecb5bb9a7d75487e3258a8efebebd7eb7934a5b975b3295b
3
+ size 398480312
model/model-00028-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:601d2358caf18891ed5d984976b4da007addc60bce7b6b9dec179a7236a175fd
3
+ size 492841688
model/model-00029-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7cd8ee8c4e958fae63a0b3b939b8896845fb690fd30fb67d9a682787cdca8fe8
3
+ size 503337904
model/model-00030-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:711b5596241c8d068f9e20bf68e92c1c02df8be0d5c7731d5deae2fd6c919088
3
+ size 387984072
model/model-00031-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a4c93f1669ee0a33a67913bba1f20f692ee333a14afda91f55f5b219f1c82a6a
3
+ size 398480312
model/model-00032-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3a89b7f522db8ab7cc6248c4e40bb117138a7c7451e576977585dc0a273ab288
3
+ size 398480304
model/model-00033-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a66bc152e004d1bb9cd34872b50af55c032911043b9b09f7391d2e461601a953
3
+ size 492841688
model/model-00034-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1300ddb6ba9e844c8bd824d452d2618b22a000a7d69d2ae949c6022ac1ea58c2
3
+ size 503337904
model/model-00035-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d3a8a8366ba200f38047f50b8a19512b1c985aa1eec18a2e5ff3678fc9cb9b96
3
+ size 387984072
model/model-00036-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a4766f8ffc7d81520fe83280b5ffd22cfdee3cb04aaab9d2aa092980422a918c
3
+ size 398480312
model/model-00037-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3d853859adfa753161e6273085d8a17ae5964c619e79cb3081164bb988a8632c
3
+ size 398480312
model/model-00038-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0f17a9f3e512428d1ab0ee2a3000f8836f3d550618b747840b72ae633dbbb3d7
3
+ size 492841688
model/model-00039-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:550ecd4e7436526df50dc74f16f82faf33f5036d36627c827b25db9aeffd57f7
3
+ size 503337904
model/model-00040-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3c247081c5abc63c764f78c28edcb09d1fe15de3a02570f57d21f3d5665c346a
3
+ size 387984072
model/model-00041-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:db50469e7647697f42d5632c070c5add1d6d185529f65f2f11106e1fb82760b7
3
+ size 398480312
model/model-00042-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1fc30ad73652dd0ba21c2212d31ffc5516ff3f16c6a4a0ce7875a0c1c7b6ac69
3
+ size 398480312
model/model-00043-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f8575e033f028b5f473f02008ebab2c054c6cbe3731ebabaab33c5e6c3daf0df
3
+ size 492841688
model/model-00044-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5fa9edf37d4c0d7a9c2d37fe528a510ffc6e107c8de399e884749aa6813f7e2e
3
+ size 503337904
model/model-00045-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7efba429af13103b4f97225f13618a7de149ca65500ded0998d1ec9a98d0e3a4
3
+ size 387984072
model/model-00046-of-00052.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1f5b9b533e0faf89afaa678dddba21c277cde523b83644a908b1bcbaa5dea916
3
+ size 398480304