Instructions to use ldp72/test-push with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ldp72/test-push with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ldp72/test-push")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ldp72/test-push")
model = AutoModelForCausalLM.from_pretrained("ldp72/test-push", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ldp72/test-push with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ldp72/test-push"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ldp72/test-push",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ldp72/test-push

SGLang

How to use ldp72/test-push with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ldp72/test-push" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ldp72/test-push",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ldp72/test-push" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ldp72/test-push",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ldp72/test-push with Docker Model Runner:
```
docker model run hf.co/ldp72/test-push
```

test-push / README.md

ldp72

docs: add README.md

0b50b47 verified 23 days ago

preview code

Raw History Blame Contribute Delete

12.3 kB

---
# For reference on model card metadata, see the spec: https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1
# Doc / guide: https://huggingface.co/docs/hub/model-cards
base_model:
- Orange/Qwen3-0.6B-reasoning
datasets: []
languages:
- wo
library_name: transformers
metrics: []
pipeline_tag: text-generation
tags: []

---
# Model Card for ldp72/test-push

<!-- Provide a quick summary of what the model is/does. -->

This model was finetuned by performing instruct tuning on Wolof language datatsets.

## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->



- **Developed by:** Orange
- **Funded by [optional]:** [More Information Needed]
- **Shared by [optional]:** [More Information Needed]
- **Model type:** [More Information Needed]
- **Language(s) (NLP):** Wolof
- **License:** [More Information Needed]
- **Finetuned from model [optional]:** Orange/Qwen3-0.6B-reasoning
- **Date [optional]:** 2026-10-07 14:09:49

### Model Sources [optional]

<!-- Provide the basic links for the model. -->

- **Repository:** [More Information Needed]
- **Paper [optional]:** [More Information Needed]
- **Demo [optional]:** [More Information Needed]

## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

### Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

This model can be used with the `transformers` library using `pipeline` abstraction as follows:

```python
import torch
from transformers import pipeline
model_id = "ldp72/test-push"
pipe = pipeline(
    "text-generation",
    model=model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
messages = [
    {"role": "system", "content": "You are chatbot specialized on Wolof language."},
    {"role": "user", "content": "Can you give a sample of your specialized knowledge?"},
]
outputs = pipe(
    messages,
    max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])
```

### Downstream Use [optional]

<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->

[More Information Needed]

### Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

[More Information Needed]

## Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

[More Information Needed]

### Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

## How to Get Started with the Model

Use the code below to get started with the model.

[More Information Needed]

## Training Details

This model was finetuned with [Orange internal fine tuning tools](https://gitlab.tech.orange/NEPAL/knowledge/orangelm/lm-adaptation/)  with the Docker Image tagged `0.2.8` in the [registry](https://gitlab.tech.orange/NEPAL/knowledge/orangelm/lm-adaptation/container_registry/84664) and the following configuration file:

```yaml
data:
    add_dataset_name: false
    dataset_name:
        train:
        -   path: wolof-lm/en_function_calling-instructions
            revision: v10
            split: train
            subsample:
                p: 0.05
                seed: 42
        -   path: wolof-lm/alpaca_translated_into_wolof-instructions
            revision: 2025.02.28
            split: train
            subsample:
                p: 0.05
                seed: 42
        validation_fc:
        -   path: wolof-lm/en_function_calling-instructions
            revision: v10
            split: validation
            subsample:
                p: 0.05
                seed: 42
        validation_it-v2:
        -   path: wolof-lm/alpaca_translated_into_wolof-instructions
            revision: 2025.02.28
            split: validation
            subsample:
                p: 0.1
                seed: 42
        -   path: wolof-lm/argilla_databricks-dolly-15k-curated-multilingual_en_translated_into_wolof-instructions
            revision: 2026.03.02
            split: validation
            subsample:
                p: 0.1
                seed: 42
    debug: false
    implementation_name: conversations
description:
    contributors:
    -   email: pierre.adam@orange.com
        first_name: Pierre
        last_name: Adam
    domain: Wolof
    languages:
    - wo
    model_name: ldp72/test-push
image:
    name: registry.gitlab.tech.orange/nepal/knowledge/orangelm/lm-adaptation/lm-adaptation
    version: 0.2.8
model:
    attn_implementation: flash_attention_2
    chat_template_tokenizer: Orange/Qwen3-0.6B-reasoning
    model_name_or_path: Orange/Qwen3-0.6B-reasoning
    trust_remote_code: true
software_info:
    git:
        branch: 165-dpr-0-23-1
        commit_date: '2026-10-06 11:47:34+02:00'
        commit_hash: da09ac3426d8d17ee7b5c690e85139f07ff5e756
        commit_message: 'fix: remove useless code dealing with qat'
        is_dirty: true
        short_hash: da09ac3
    parameters:
        accelerate_file: null
        clean_output: false
        cluster: marcel
        codecarbon_log_level: warning
        collator_stats: false
        config_file: resources/configs/Wolof-Qwen3-0.6B-toy-reasoning.yml
        constraints: queue:name=gpu_mem_40,gpu:1
        data_seed: 42
        debug: false
        deepspeed_file: null
        dry_run: false
        environment_variable:
            TOKENIZERS_PARALLELISM: 'false'
        load_only: null
        log_level: INFO
        log_output: null
        merge_peft: false
        no_requeue: false
        output_dir: Wolof-Qwen3-0.6B-toy-reasoning-test
        profiler_config: null
        push_to_hub: false
        qat_config: null
        requeue_n_epochs: 1
        time_out: 24h
        time_requeue: false
        use_qat: false
    version: 0.2.9.dev36+merging.147.grpo.update.g6da62cdc9.d20261005
training:
    assistant_only_loss: true
    bf16: true
    dataloader_num_workers: 4
    dataloader_persistent_workers: true
    dataloader_pin_memory: true
    dataloader_prefetch_factor: 2
    disable_tqdm: true
    eval_accumulation_steps: 1
    eval_steps: 4
    eval_strategy: steps
    fp16: false
    gradient_accumulation_steps: 32
    gradient_checkpointing: true
    greater_is_better: false
    learning_rate: 2.0e-05
    load_best_model_at_end: true
    log_level: warning
    logging_dir: /outputs/Wolof-Qwen3-0.6B-toy-reasoning-test/logs
    logging_steps: 1
    lr_scheduler_type: cosine
    max_grad_norm: 1.0
    max_length: 1024
    max_steps: -1
    metric_for_best_model: eval_weighted_validation_loss
    num_train_epochs: 2
    optim: paged_adamw_32bit
    output_dir: /outputs/Wolof-Qwen3-0.6B-toy-reasoning-test
    per_device_eval_batch_size: 8
    per_device_train_batch_size: 8
    push_to_hub: false
    report_to: tensorboard
    save_steps: 4
    save_strategy: steps
    save_total_limit: 2
    seed: 42
    torch_compile: false
    training_type: instruct-tuning
    use_liger_kernel: true
    warmup_ratio: 0.05
    weight_decay: 0.1
```

The model was trained on 1 gpus with at least 40GB on each gpu.

### Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

This model was trained on the following datasets:

```yaml
-   path: wolof-lm/en_function_calling-instructions
    revision: v10
    split: train
    subsample:
        p: 0.05
        seed: 42
-   path: wolof-lm/alpaca_translated_into_wolof-instructions
    revision: 2025.02.28
    split: train
    subsample:
        p: 0.05
        seed: 42
```

### Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

#### Preprocessing [optional]

[More Information Needed]

#### Training Hyperparameters

<!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->

- **Training regime:** This model was trained with the following hyperparameters for `SFTTrainer`,other parameters were set as default:

```yaml
assistant_only_loss: true
bf16: true
dataloader_num_workers: 4
dataloader_persistent_workers: true
dataloader_pin_memory: true
dataloader_prefetch_factor: 2
disable_tqdm: true
eval_accumulation_steps: 1
eval_steps: 4
eval_strategy: steps
fp16: false
gradient_accumulation_steps: 32
gradient_checkpointing: true
greater_is_better: false
learning_rate: 2.0e-05
load_best_model_at_end: true
log_level: warning
logging_dir: /outputs/Wolof-Qwen3-0.6B-toy-reasoning-test/logs
logging_steps: 1
lr_scheduler_type: cosine
max_grad_norm: 1.0
max_length: 1024
max_steps: -1
metric_for_best_model: eval_weighted_validation_loss
num_train_epochs: 2
optim: paged_adamw_32bit
output_dir: /outputs/Wolof-Qwen3-0.6B-toy-reasoning-test
per_device_eval_batch_size: 8
per_device_train_batch_size: 8
push_to_hub: false
report_to: tensorboard
save_steps: 4
save_strategy: steps
save_total_limit: 2
seed: 42
torch_compile: false
use_liger_kernel: true
warmup_ratio: 0.05
weight_decay: 0.1
```

**Number of Tokens vs Steps**

![number of tokens vs steps](train_num_tokens_vs_steps.png 'Number of Tokens vs Steps')

**Learning Rate Curve**

![learning rate curve](train_learning_rate_vs_cumulated_tokens.png 'Learning Rate')

**Training Loss**

![training loss curve](train_loss_vs_cumulated_tokens.png 'Training Loss')

**Validation Loss**

![validation loss curve](validation_loss_vs_cumulated_tokens.png 'Validation Loss')

#### Speeds, Sizes, Times [optional]

<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->

[More Information Needed]

## Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

### Testing Data, Factors & Metrics

#### Testing Data

<!-- This should link to a Dataset Card if possible. -->

[More Information Needed]

#### Factors

<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->

[More Information Needed]

#### Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

[More Information Needed]

### Results

[More Information Needed]

#### Summary



## Model Examination [optional]

<!-- Relevant interpretability work for the model goes here -->

[More Information Needed]

## Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).

- **Hardware Type:** [More Information Needed]
- **Hours used:** [More Information Needed]
- **Cloud Provider:** [More Information Needed]
- **Compute Region:** [More Information Needed]
- **Carbon Emitted:** [More Information Needed]

## Technical Specifications [optional]

### Model Architecture and Objective

[More Information Needed]

### Compute Infrastructure

[More Information Needed]

#### Hardware

[More Information Needed]

#### Software

[More Information Needed]

## Citation [optional]

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

**BibTeX:**

[More Information Needed]

**APA:**

[More Information Needed]

## Glossary [optional]

<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->

[More Information Needed]

## More Information [optional]

[More Information Needed]

## Model Card Authors [optional]

[More Information Needed]

## Model Card Contact

Thanks to [Pierre Adam](mailto:pierre.adam@orange.com) for adding this model.

	---
	# For reference on model card metadata, see the spec: https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1
	# Doc / guide: https://huggingface.co/docs/hub/model-cards
	base_model:
	- Orange/Qwen3-0.6B-reasoning
	datasets: []
	languages:
	- wo
	library_name: transformers
	metrics: []
	pipeline_tag: text-generation
	tags: []

	---

	# Model Card for ldp72/test-push

	<!-- Provide a quick summary of what the model is/does. -->

	This model was finetuned by performing instruct tuning on Wolof language datatsets.

	## Model Details

	### Model Description

	<!-- Provide a longer summary of what this model is. -->



	- Developed by: Orange
	- Funded by [optional]: [More Information Needed]
	- Shared by [optional]: [More Information Needed]
	- Model type: [More Information Needed]
	- Language(s) (NLP): Wolof
	- License: [More Information Needed]
	- Finetuned from model [optional]: Orange/Qwen3-0.6B-reasoning
	- Date [optional]: 2026-10-07 14:09:49

	### Model Sources [optional]

	<!-- Provide the basic links for the model. -->

	- Repository: [More Information Needed]
	- Paper [optional]: [More Information Needed]
	- Demo [optional]: [More Information Needed]

	## Uses

	<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

	### Direct Use

	<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

	This model can be used with the `transformers` library using `pipeline` abstraction as follows:

	```python
	import torch
	from transformers import pipeline

	model_id = "ldp72/test-push"
	pipe = pipeline(
	"text-generation",
	model=model_id,
	torch_dtype=torch.bfloat16,
	device_map="auto",
	)
	messages = [
	{"role": "system", "content": "You are chatbot specialized on Wolof language."},
	{"role": "user", "content": "Can you give a sample of your specialized knowledge?"},
	]
	outputs = pipe(
	messages,
	max_new_tokens=256,
	)
	print(outputs[0]["generated_text"][-1])
	```

	### Downstream Use [optional]

	<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->

	[More Information Needed]

	### Out-of-Scope Use

	<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

	[More Information Needed]

	## Bias, Risks, and Limitations

	<!-- This section is meant to convey both technical and sociotechnical limitations. -->

	[More Information Needed]

	### Recommendations

	<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

	Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

	## How to Get Started with the Model

	Use the code below to get started with the model.

	[More Information Needed]

	## Training Details

	This model was finetuned with [Orange internal fine tuning tools](https://gitlab.tech.orange/NEPAL/knowledge/orangelm/lm-adaptation/) with the Docker Image tagged `0.2.8` in the [registry](https://gitlab.tech.orange/NEPAL/knowledge/orangelm/lm-adaptation/container_registry/84664) and the following configuration file:

	```yaml
	data:
	add_dataset_name: false
	dataset_name:
	train:
	- path: wolof-lm/en_function_calling-instructions
	revision: v10
	split: train
	subsample:
	p: 0.05
	seed: 42
	- path: wolof-lm/alpaca_translated_into_wolof-instructions
	revision: 2025.02.28
	split: train
	subsample:
	p: 0.05
	seed: 42
	validation_fc:
	- path: wolof-lm/en_function_calling-instructions
	revision: v10
	split: validation
	subsample:
	p: 0.05
	seed: 42
	validation_it-v2:
	- path: wolof-lm/alpaca_translated_into_wolof-instructions
	revision: 2025.02.28
	split: validation
	subsample:
	p: 0.1
	seed: 42
	- path: wolof-lm/argilla_databricks-dolly-15k-curated-multilingual_en_translated_into_wolof-instructions
	revision: 2026.03.02
	split: validation
	subsample:
	p: 0.1
	seed: 42
	debug: false
	implementation_name: conversations
	description:
	contributors:
	- email: pierre.adam@orange.com
	first_name: Pierre
	last_name: Adam
	domain: Wolof
	languages:
	- wo
	model_name: ldp72/test-push
	image:
	name: registry.gitlab.tech.orange/nepal/knowledge/orangelm/lm-adaptation/lm-adaptation
	version: 0.2.8
	model:
	attn_implementation: flash_attention_2
	chat_template_tokenizer: Orange/Qwen3-0.6B-reasoning
	model_name_or_path: Orange/Qwen3-0.6B-reasoning
	trust_remote_code: true
	software_info:
	git:
	branch: 165-dpr-0-23-1
	commit_date: '2026-10-06 11:47:34+02:00'
	commit_hash: da09ac3426d8d17ee7b5c690e85139f07ff5e756
	commit_message: 'fix: remove useless code dealing with qat'
	is_dirty: true
	short_hash: da09ac3
	parameters:
	accelerate_file: null
	clean_output: false
	cluster: marcel
	codecarbon_log_level: warning
	collator_stats: false
	config_file: resources/configs/Wolof-Qwen3-0.6B-toy-reasoning.yml
	constraints: queue:name=gpu_mem_40,gpu:1
	data_seed: 42
	debug: false
	deepspeed_file: null
	dry_run: false
	environment_variable:
	TOKENIZERS_PARALLELISM: 'false'
	load_only: null
	log_level: INFO
	log_output: null
	merge_peft: false
	no_requeue: false
	output_dir: Wolof-Qwen3-0.6B-toy-reasoning-test
	profiler_config: null
	push_to_hub: false
	qat_config: null
	requeue_n_epochs: 1
	time_out: 24h
	time_requeue: false
	use_qat: false
	version: 0.2.9.dev36+merging.147.grpo.update.g6da62cdc9.d20261005
	training:
	assistant_only_loss: true
	bf16: true
	dataloader_num_workers: 4
	dataloader_persistent_workers: true
	dataloader_pin_memory: true
	dataloader_prefetch_factor: 2
	disable_tqdm: true
	eval_accumulation_steps: 1
	eval_steps: 4
	eval_strategy: steps
	fp16: false
	gradient_accumulation_steps: 32
	gradient_checkpointing: true
	greater_is_better: false
	learning_rate: 2.0e-05
	load_best_model_at_end: true
	log_level: warning
	logging_dir: /outputs/Wolof-Qwen3-0.6B-toy-reasoning-test/logs
	logging_steps: 1
	lr_scheduler_type: cosine
	max_grad_norm: 1.0
	max_length: 1024
	max_steps: -1
	metric_for_best_model: eval_weighted_validation_loss
	num_train_epochs: 2
	optim: paged_adamw_32bit
	output_dir: /outputs/Wolof-Qwen3-0.6B-toy-reasoning-test
	per_device_eval_batch_size: 8
	per_device_train_batch_size: 8
	push_to_hub: false
	report_to: tensorboard
	save_steps: 4
	save_strategy: steps
	save_total_limit: 2
	seed: 42
	torch_compile: false
	training_type: instruct-tuning
	use_liger_kernel: true
	warmup_ratio: 0.05
	weight_decay: 0.1
	```

	The model was trained on 1 gpus with at least 40GB on each gpu.

	### Training Data

	<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

	This model was trained on the following datasets:

	```yaml
	- path: wolof-lm/en_function_calling-instructions
	revision: v10
	split: train
	subsample:
	p: 0.05
	seed: 42
	- path: wolof-lm/alpaca_translated_into_wolof-instructions
	revision: 2025.02.28
	split: train
	subsample:
	p: 0.05
	seed: 42
	```

	### Training Procedure

	<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

	#### Preprocessing [optional]

	[More Information Needed]

	#### Training Hyperparameters

	<!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->

	- Training regime: This model was trained with the following hyperparameters for `SFTTrainer`,other parameters were set as default:

	```yaml
	assistant_only_loss: true
	bf16: true
	dataloader_num_workers: 4
	dataloader_persistent_workers: true
	dataloader_pin_memory: true
	dataloader_prefetch_factor: 2
	disable_tqdm: true
	eval_accumulation_steps: 1
	eval_steps: 4
	eval_strategy: steps
	fp16: false
	gradient_accumulation_steps: 32
	gradient_checkpointing: true
	greater_is_better: false
	learning_rate: 2.0e-05
	load_best_model_at_end: true
	log_level: warning
	logging_dir: /outputs/Wolof-Qwen3-0.6B-toy-reasoning-test/logs
	logging_steps: 1
	lr_scheduler_type: cosine
	max_grad_norm: 1.0
	max_length: 1024
	max_steps: -1
	metric_for_best_model: eval_weighted_validation_loss
	num_train_epochs: 2
	optim: paged_adamw_32bit
	output_dir: /outputs/Wolof-Qwen3-0.6B-toy-reasoning-test
	per_device_eval_batch_size: 8
	per_device_train_batch_size: 8
	push_to_hub: false
	report_to: tensorboard
	save_steps: 4
	save_strategy: steps
	save_total_limit: 2
	seed: 42
	torch_compile: false
	use_liger_kernel: true
	warmup_ratio: 0.05
	weight_decay: 0.1
	```

	Number of Tokens vs Steps

	![number of tokens vs steps](train_num_tokens_vs_steps.png 'Number of Tokens vs Steps')

	Learning Rate Curve

	![learning rate curve](train_learning_rate_vs_cumulated_tokens.png 'Learning Rate')

	Training Loss

	![training loss curve](train_loss_vs_cumulated_tokens.png 'Training Loss')

	Validation Loss

	![validation loss curve](validation_loss_vs_cumulated_tokens.png 'Validation Loss')

	#### Speeds, Sizes, Times [optional]

	<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->

	[More Information Needed]

	## Evaluation

	<!-- This section describes the evaluation protocols and provides the results. -->

	### Testing Data, Factors & Metrics

	#### Testing Data

	<!-- This should link to a Dataset Card if possible. -->

	[More Information Needed]

	#### Factors

	<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->

	[More Information Needed]

	#### Metrics

	<!-- These are the evaluation metrics being used, ideally with a description of why. -->

	[More Information Needed]

	### Results

	[More Information Needed]

	#### Summary



	## Model Examination [optional]

	<!-- Relevant interpretability work for the model goes here -->

	[More Information Needed]

	## Environmental Impact

	<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

	Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).

	- Hardware Type: [More Information Needed]
	- Hours used: [More Information Needed]
	- Cloud Provider: [More Information Needed]
	- Compute Region: [More Information Needed]
	- Carbon Emitted: [More Information Needed]

	## Technical Specifications [optional]

	### Model Architecture and Objective

	[More Information Needed]

	### Compute Infrastructure

	[More Information Needed]

	#### Hardware

	[More Information Needed]

	#### Software

	[More Information Needed]

	## Citation [optional]

	<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

	BibTeX:

	[More Information Needed]

	APA:

	[More Information Needed]

	## Glossary [optional]

	<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->

	[More Information Needed]

	## More Information [optional]

	[More Information Needed]

	## Model Card Authors [optional]

	[More Information Needed]

	## Model Card Contact

	Thanks to [Pierre Adam](mailto:pierre.adam@orange.com) for adding this model.