jpacifico commited on
Commit
478f1d7
·
verified ·
1 Parent(s): 6ecdfe4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -3
README.md CHANGED
@@ -16,7 +16,7 @@ language:
16
  - en
17
  - fr
18
  ---
19
- # model Summary
20
 
21
  **bitnet-dpo-merged-modelstock7** *(2.41B params / Context Length: Maximum sequence length of 4096 tokens)*
22
  A compact, agent-oriented small language model focused on language understanding and contextual decision-making.
@@ -41,7 +41,7 @@ Iterative DPO + Model merging :
41
  - Bilingual DPO (FR+EN) to sharpen preference selection across two languages, using the following datasets :
42
  [jpacifico/french-orca-dpo-pairs-revised](https://huggingface.co/datasets/jpacifico/french-orca-dpo-pairs-revised)
43
  [Intel/orca_dpo_pairs](https://huggingface.co/datasets/Intel/orca_dpo_pairs)
44
- - Model merging (FR-centric + EN-centric) using [mergekit](https://github.com/cg123/mergekit) to broaden stylistic/lexical coverage.
45
 
46
  # First benchmarks
47
 
@@ -109,4 +109,11 @@ parameters:
109
  dtype: bfloat16
110
  tokenizer_source: jpacifico/bitnet-dpo-merged-modelstock-retrain
111
 
112
- ```
 
 
 
 
 
 
 
 
16
  - en
17
  - fr
18
  ---
19
+ # Model Summary
20
 
21
  **bitnet-dpo-merged-modelstock7** *(2.41B params / Context Length: Maximum sequence length of 4096 tokens)*
22
  A compact, agent-oriented small language model focused on language understanding and contextual decision-making.
 
41
  - Bilingual DPO (FR+EN) to sharpen preference selection across two languages, using the following datasets :
42
  [jpacifico/french-orca-dpo-pairs-revised](https://huggingface.co/datasets/jpacifico/french-orca-dpo-pairs-revised)
43
  [Intel/orca_dpo_pairs](https://huggingface.co/datasets/Intel/orca_dpo_pairs)
44
+ - Model merging (ModelStock and TIES methods, via [Mergekit](https://github.com/cg123/mergekit) to combine complementary strengths of bilingual models (FR-centric + EN-centric), improving robustness across reasoning and comprehension tasks while maintaining stability.
45
 
46
  # First benchmarks
47
 
 
109
  dtype: bfloat16
110
  tokenizer_source: jpacifico/bitnet-dpo-merged-modelstock-retrain
111
 
112
+ ```
113
+
114
+ - **Developed by:** Jonathan Pacifico, 2025
115
+ - **Model type:** LLM
116
+ - **Language(s) (NLP):** French, English
117
+ - **License:** MIT
118
+
119
+ Made with ❤️ in France