docs: add README.md

README.md

@@ -1,13 +1,23 @@
 ---
+# For reference on model card metadata, see the spec: https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1
+# Doc / guide: https://huggingface.co/docs/hub/model-cards
+base_model:
+- Orange/Qwen3-0.6B-reasoning
+datasets: []
+languages:
+- wo
 library_name: transformers
+metrics: []
+pipeline_tag: text-generation
 tags: []
+
 ---
 
-# Model Card for Model ID
+# Model Card for ldp72/test-push
 
 <!-- Provide a quick summary of what the model is/does. -->
 
-
+This model was finetuned by performing instruct tuning on Wolof language datatsets.
 
 ## Model Details
 
@@ -15,15 +25,16 @@ tags: []
 
 <!-- Provide a longer summary of what this model is. -->
 
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
 
-- **Developed by:** [More Information Needed]
+
+- **Developed by:** Orange
 - **Funded by [optional]:** [More Information Needed]
 - **Shared by [optional]:** [More Information Needed]
 - **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
+- **Language(s) (NLP):** Wolof
 - **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+- **Finetuned from model [optional]:** Orange/Qwen3-0.6B-reasoning
+- **Date [optional]:** 2026-10-07 14:09:49
 
 ### Model Sources [optional]
 
@@ -41,7 +52,29 @@ This is the model card of a 🤗 transformers model that has been pushed on the
 
 <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
 
-[More Information Needed]
+This model can be used with the `transformers` library using `pipeline` abstraction as follows:
+
+```python
+import torch
+from transformers import pipeline
+
+model_id = "ldp72/test-push"
+pipe = pipeline(
+    "text-generation",
+    model=model_id,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+)
+messages = [
+    {"role": "system", "content": "You are chatbot specialized on Wolof language."},
+    {"role": "user", "content": "Can you give a sample of your specialized knowledge?"},
+]
+outputs = pipe(
+    messages,
+    max_new_tokens=256,
+)
+print(outputs[0]["generated_text"][-1])
+```
 
 ### Downstream Use [optional]
 
@@ -75,11 +108,165 @@ Use the code below to get started with the model.
 
 ## Training Details
 
+This model was finetuned with [Orange internal fine tuning tools](https://gitlab.tech.orange/NEPAL/knowledge/orangelm/lm-adaptation/)  with the Docker Image tagged `0.2.8` in the [registry](https://gitlab.tech.orange/NEPAL/knowledge/orangelm/lm-adaptation/container_registry/84664) and the following configuration file:
+
+```yaml
+data:
+    add_dataset_name: false
+    dataset_name:
+        train:
+        -   path: wolof-lm/en_function_calling-instructions
+            revision: v10
+            split: train
+            subsample:
+                p: 0.05
+                seed: 42
+        -   path: wolof-lm/alpaca_translated_into_wolof-instructions
+            revision: 2025.02.28
+            split: train
+            subsample:
+                p: 0.05
+                seed: 42
+        validation_fc:
+        -   path: wolof-lm/en_function_calling-instructions
+            revision: v10
+            split: validation
+            subsample:
+                p: 0.05
+                seed: 42
+        validation_it-v2:
+        -   path: wolof-lm/alpaca_translated_into_wolof-instructions
+            revision: 2025.02.28
+            split: validation
+            subsample:
+                p: 0.1
+                seed: 42
+        -   path: wolof-lm/argilla_databricks-dolly-15k-curated-multilingual_en_translated_into_wolof-instructions
+            revision: 2026.03.02
+            split: validation
+            subsample:
+                p: 0.1
+                seed: 42
+    debug: false
+    implementation_name: conversations
+description:
+    contributors:
+    -   email: pierre.adam@orange.com
+        first_name: Pierre
+        last_name: Adam
+    domain: Wolof
+    languages:
+    - wo
+    model_name: ldp72/test-push
+image:
+    name: registry.gitlab.tech.orange/nepal/knowledge/orangelm/lm-adaptation/lm-adaptation
+    version: 0.2.8
+model:
+    attn_implementation: flash_attention_2
+    chat_template_tokenizer: Orange/Qwen3-0.6B-reasoning
+    model_name_or_path: Orange/Qwen3-0.6B-reasoning
+    trust_remote_code: true
+software_info:
+    git:
+        branch: 165-dpr-0-23-1
+        commit_date: '2026-10-06 11:47:34+02:00'
+        commit_hash: da09ac3426d8d17ee7b5c690e85139f07ff5e756
+        commit_message: 'fix: remove useless code dealing with qat'
+        is_dirty: true
+        short_hash: da09ac3
+    parameters:
+        accelerate_file: null
+        clean_output: false
+        cluster: marcel
+        codecarbon_log_level: warning
+        collator_stats: false
+        config_file: resources/configs/Wolof-Qwen3-0.6B-toy-reasoning.yml
+        constraints: queue:name=gpu_mem_40,gpu:1
+        data_seed: 42
+        debug: false
+        deepspeed_file: null
+        dry_run: false
+        environment_variable:
+            TOKENIZERS_PARALLELISM: 'false'
+        load_only: null
+        log_level: INFO
+        log_output: null
+        merge_peft: false
+        no_requeue: false
+        output_dir: Wolof-Qwen3-0.6B-toy-reasoning-test
+        profiler_config: null
+        push_to_hub: false
+        qat_config: null
+        requeue_n_epochs: 1
+        time_out: 24h
+        time_requeue: false
+        use_qat: false
+    version: 0.2.9.dev36+merging.147.grpo.update.g6da62cdc9.d20261005
+training:
+    assistant_only_loss: true
+    bf16: true
+    dataloader_num_workers: 4
+    dataloader_persistent_workers: true
+    dataloader_pin_memory: true
+    dataloader_prefetch_factor: 2
+    disable_tqdm: true
+    eval_accumulation_steps: 1
+    eval_steps: 4
+    eval_strategy: steps
+    fp16: false
+    gradient_accumulation_steps: 32
+    gradient_checkpointing: true
+    greater_is_better: false
+    learning_rate: 2.0e-05
+    load_best_model_at_end: true
+    log_level: warning
+    logging_dir: /outputs/Wolof-Qwen3-0.6B-toy-reasoning-test/logs
+    logging_steps: 1
+    lr_scheduler_type: cosine
+    max_grad_norm: 1.0
+    max_length: 1024
+    max_steps: -1
+    metric_for_best_model: eval_weighted_validation_loss
+    num_train_epochs: 2
+    optim: paged_adamw_32bit
+    output_dir: /outputs/Wolof-Qwen3-0.6B-toy-reasoning-test
+    per_device_eval_batch_size: 8
+    per_device_train_batch_size: 8
+    push_to_hub: false
+    report_to: tensorboard
+    save_steps: 4
+    save_strategy: steps
+    save_total_limit: 2
+    seed: 42
+    torch_compile: false
+    training_type: instruct-tuning
+    use_liger_kernel: true
+    warmup_ratio: 0.05
+    weight_decay: 0.1
+```
+
+The model was trained on 1 gpus with at least 40GB on each gpu.
+
 ### Training Data
 
 <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
 
-[More Information Needed]
+This model was trained on the following datasets:
+
+```yaml
+-   path: wolof-lm/en_function_calling-instructions
+    revision: v10
+    split: train
+    subsample:
+        p: 0.05
+        seed: 42
+-   path: wolof-lm/alpaca_translated_into_wolof-instructions
+    revision: 2025.02.28
+    split: train
+    subsample:
+        p: 0.05
+        seed: 42
+```
 
 ### Training Procedure
 
@@ -89,10 +276,69 @@ Use the code below to get started with the model.
 
 [More Information Needed]
 
-
 #### Training Hyperparameters
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+<!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+- **Training regime:** This model was trained with the following hyperparameters for `SFTTrainer`,other parameters were set as default:
+
+```yaml
+assistant_only_loss: true
+bf16: true
+dataloader_num_workers: 4
+dataloader_persistent_workers: true
+dataloader_pin_memory: true
+dataloader_prefetch_factor: 2
+disable_tqdm: true
+eval_accumulation_steps: 1
+eval_steps: 4
+eval_strategy: steps
+fp16: false
+gradient_accumulation_steps: 32
+gradient_checkpointing: true
+greater_is_better: false
+learning_rate: 2.0e-05
+load_best_model_at_end: true
+log_level: warning
+logging_dir: /outputs/Wolof-Qwen3-0.6B-toy-reasoning-test/logs
+logging_steps: 1
+lr_scheduler_type: cosine
+max_grad_norm: 1.0
+max_length: 1024
+max_steps: -1
+metric_for_best_model: eval_weighted_validation_loss
+num_train_epochs: 2
+optim: paged_adamw_32bit
+output_dir: /outputs/Wolof-Qwen3-0.6B-toy-reasoning-test
+per_device_eval_batch_size: 8
+per_device_train_batch_size: 8
+push_to_hub: false
+report_to: tensorboard
+save_steps: 4
+save_strategy: steps
+save_total_limit: 2
+seed: 42
+torch_compile: false
+use_liger_kernel: true
+warmup_ratio: 0.05
+weight_decay: 0.1
+```
+
+**Number of Tokens vs Steps**
+
+![number of tokens vs steps](train_num_tokens_vs_steps.png 'Number of Tokens vs Steps')
+
+**Learning Rate Curve**
+
+![learning rate curve](train_learning_rate_vs_cumulated_tokens.png 'Learning Rate')
+
+**Training Loss**
+
+![training loss curve](train_loss_vs_cumulated_tokens.png 'Training Loss')
+
+**Validation Loss**
+
+![validation loss curve](validation_loss_vs_cumulated_tokens.png 'Validation Loss')
 
 #### Speeds, Sizes, Times [optional]
 
@@ -196,4 +442,4 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
 
 ## Model Card Contact
 
-[More Information Needed]
+Thanks to [Pierre Adam](mailto:pierre.adam@orange.com) for adding this model.