Blame for README.md

---
# For reference on model card metadata, see the spec: https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1
# Doc / guide: https://huggingface.co/docs/hub/model-cards
base_model:
- Orange/Qwen3-0.6B-reasoning
datasets: []
languages:
- wo
library_name: transformers
metrics: []
pipeline_tag: text-generation
tags: []

---

# Model Card for ldp72/test-push

<!-- Provide a quick summary of what the model is/does. -->

This model was finetuned by performing instruct tuning on Wolof language datatsets.

## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->



- **Developed by:** Orange
- **Funded by [optional]:** [More Information Needed]
- **Shared by [optional]:** [More Information Needed]
- **Model type:** [More Information Needed]
- **Language(s) (NLP):** Wolof
- **License:** [More Information Needed]
- **Finetuned from model [optional]:** Orange/Qwen3-0.6B-reasoning
- **Date [optional]:** 2026-10-07 14:09:49

### Model Sources [optional]

<!-- Provide the basic links for the model. -->

- **Repository:** [More Information Needed]
- **Paper [optional]:** [More Information Needed]
- **Demo [optional]:** [More Information Needed]

## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

### Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

This model can be used with the `transformers` library using `pipeline` abstraction as follows:

```python
import torch
from transformers import pipeline

model_id = "ldp72/test-push"
pipe = pipeline(
    "text-generation",
    model=model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
messages = [
    {"role": "system", "content": "You are chatbot specialized on Wolof language."},
    {"role": "user", "content": "Can you give a sample of your specialized knowledge?"},
]
outputs = pipe(
    messages,
    max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])
```

### Downstream Use [optional]

<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->

[More Information Needed]

### Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

[More Information Needed]

## Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

[More Information Needed]

### Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

## How to Get Started with the Model

Use the code below to get started with the model.

[More Information Needed]

## Training Details

This model was finetuned with [Orange internal fine tuning tools](https://gitlab.tech.orange/NEPAL/knowledge/orangelm/lm-adaptation/)  with the Docker Image tagged `0.2.8` in the [registry](https://gitlab.tech.orange/NEPAL/knowledge/orangelm/lm-adaptation/container_registry/84664) and the following configuration file:

```yaml
data:
    add_dataset_name: false
    dataset_name:
        train:
        -   path: wolof-lm/en_function_calling-instructions
            revision: v10
            split: train
            subsample:
                p: 0.05
                seed: 42
        -   path: wolof-lm/alpaca_translated_into_wolof-instructions
            revision: 2025.02.28
            split: train
            subsample:
                p: 0.05
                seed: 42
        validation_fc:
        -   path: wolof-lm/en_function_calling-instructions
            revision: v10
            split: validation
            subsample:
                p: 0.05
                seed: 42
        validation_it-v2:
        -   path: wolof-lm/alpaca_translated_into_wolof-instructions
            revision: 2025.02.28
            split: validation
            subsample:
                p: 0.1
                seed: 42
        -   path: wolof-lm/argilla_databricks-dolly-15k-curated-multilingual_en_translated_into_wolof-instructions
            revision: 2026.03.02
            split: validation
            subsample:
                p: 0.1
                seed: 42
    debug: false
    implementation_name: conversations
description:
    contributors:
    -   email: pierre.adam@orange.com
        first_name: Pierre
        last_name: Adam
    domain: Wolof
    languages:
    - wo
    model_name: ldp72/test-push
image:
    name: registry.gitlab.tech.orange/nepal/knowledge/orangelm/lm-adaptation/lm-adaptation
    version: 0.2.8
model:
    attn_implementation: flash_attention_2
    chat_template_tokenizer: Orange/Qwen3-0.6B-reasoning
    model_name_or_path: Orange/Qwen3-0.6B-reasoning
    trust_remote_code: true
software_info:
    git:
        branch: 165-dpr-0-23-1
        commit_date: '2026-10-06 11:47:34+02:00'
        commit_hash: da09ac3426d8d17ee7b5c690e85139f07ff5e756
        commit_message: 'fix: remove useless code dealing with qat'
        is_dirty: true
        short_hash: da09ac3
    parameters:
        accelerate_file: null
        clean_output: false
        cluster: marcel
        codecarbon_log_level: warning
        collator_stats: false
        config_file: resources/configs/Wolof-Qwen3-0.6B-toy-reasoning.yml
        constraints: queue:name=gpu_mem_40,gpu:1
        data_seed: 42
        debug: false
        deepspeed_file: null
        dry_run: false
        environment_variable:
            TOKENIZERS_PARALLELISM: 'false'
        load_only: null
        log_level: INFO
        log_output: null
        merge_peft: false
        no_requeue: false
        output_dir: Wolof-Qwen3-0.6B-toy-reasoning-test
        profiler_config: null
        push_to_hub: false
        qat_config: null
        requeue_n_epochs: 1
        time_out: 24h
        time_requeue: false
        use_qat: false
    version: 0.2.9.dev36+merging.147.grpo.update.g6da62cdc9.d20261005
training:
    assistant_only_loss: true
    bf16: true
    dataloader_num_workers: 4
    dataloader_persistent_workers: true
    dataloader_pin_memory: true
    dataloader_prefetch_factor: 2
    disable_tqdm: true
    eval_accumulation_steps: 1
    eval_steps: 4
    eval_strategy: steps
    fp16: false
    gradient_accumulation_steps: 32
    gradient_checkpointing: true
    greater_is_better: false
    learning_rate: 2.0e-05
    load_best_model_at_end: true
    log_level: warning
    logging_dir: /outputs/Wolof-Qwen3-0.6B-toy-reasoning-test/logs
    logging_steps: 1
    lr_scheduler_type: cosine
    max_grad_norm: 1.0
    max_length: 1024
    max_steps: -1
    metric_for_best_model: eval_weighted_validation_loss
    num_train_epochs: 2
    optim: paged_adamw_32bit
    output_dir: /outputs/Wolof-Qwen3-0.6B-toy-reasoning-test
    per_device_eval_batch_size: 8
    per_device_train_batch_size: 8
    push_to_hub: false
    report_to: tensorboard
    save_steps: 4
    save_strategy: steps
    save_total_limit: 2
    seed: 42
    torch_compile: false
    training_type: instruct-tuning
    use_liger_kernel: true
    warmup_ratio: 0.05
    weight_decay: 0.1
```

The model was trained on 1 gpus with at least 40GB on each gpu.

### Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

This model was trained on the following datasets:

```yaml
-   path: wolof-lm/en_function_calling-instructions
    revision: v10
    split: train
    subsample:
        p: 0.05
        seed: 42
-   path: wolof-lm/alpaca_translated_into_wolof-instructions
    revision: 2025.02.28
    split: train
    subsample:
        p: 0.05
        seed: 42
```

### Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

#### Preprocessing [optional]

[More Information Needed]

#### Training Hyperparameters

<!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->

- **Training regime:** This model was trained with the following hyperparameters for `SFTTrainer`,other parameters were set as default:

```yaml
assistant_only_loss: true
bf16: true
dataloader_num_workers: 4
dataloader_persistent_workers: true
dataloader_pin_memory: true
dataloader_prefetch_factor: 2
disable_tqdm: true
eval_accumulation_steps: 1
eval_steps: 4
eval_strategy: steps
fp16: false
gradient_accumulation_steps: 32
gradient_checkpointing: true
greater_is_better: false
learning_rate: 2.0e-05
load_best_model_at_end: true
log_level: warning
logging_dir: /outputs/Wolof-Qwen3-0.6B-toy-reasoning-test/logs
logging_steps: 1
lr_scheduler_type: cosine
max_grad_norm: 1.0
max_length: 1024
max_steps: -1
metric_for_best_model: eval_weighted_validation_loss
num_train_epochs: 2
optim: paged_adamw_32bit
output_dir: /outputs/Wolof-Qwen3-0.6B-toy-reasoning-test
per_device_eval_batch_size: 8
per_device_train_batch_size: 8
push_to_hub: false
report_to: tensorboard
save_steps: 4
save_strategy: steps
save_total_limit: 2
seed: 42
torch_compile: false
use_liger_kernel: true
warmup_ratio: 0.05
weight_decay: 0.1
```

**Number of Tokens vs Steps**

![number of tokens vs steps](train_num_tokens_vs_steps.png 'Number of Tokens vs Steps')

**Learning Rate Curve**

![learning rate curve](train_learning_rate_vs_cumulated_tokens.png 'Learning Rate')

**Training Loss**

![training loss curve](train_loss_vs_cumulated_tokens.png 'Training Loss')

**Validation Loss**

![validation loss curve](validation_loss_vs_cumulated_tokens.png 'Validation Loss')

#### Speeds, Sizes, Times [optional]

<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->

[More Information Needed]

## Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

### Testing Data, Factors & Metrics

#### Testing Data

<!-- This should link to a Dataset Card if possible. -->

[More Information Needed]

#### Factors

<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->

[More Information Needed]

#### Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

[More Information Needed]

### Results

[More Information Needed]

#### Summary



## Model Examination [optional]

<!-- Relevant interpretability work for the model goes here -->

[More Information Needed]

## Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).

- **Hardware Type:** [More Information Needed]
- **Hours used:** [More Information Needed]
- **Cloud Provider:** [More Information Needed]
- **Compute Region:** [More Information Needed]
- **Carbon Emitted:** [More Information Needed]

## Technical Specifications [optional]

### Model Architecture and Objective

[More Information Needed]

### Compute Infrastructure

[More Information Needed]

#### Hardware

[More Information Needed]

#### Software

[More Information Needed]

## Citation [optional]

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

**BibTeX:**

[More Information Needed]

**APA:**

[More Information Needed]

## Glossary [optional]

<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->

[More Information Needed]

## More Information [optional]

[More Information Needed]

## Model Card Authors [optional]

[More Information Needed]

## Model Card Contact

Thanks to [Pierre Adam](mailto:pierre.adam@orange.com) for adding this model.