README.md

---
license: apache-2.0
language:
  - en
library_name: transformers
tags:
  - qwen
  - qwen3
  - fine-tuned
  - causal-lm
  - text-generation
  - trl
  - deepspeed
  - autotrain-compatible

datasets:
  - bespokelabs/Bespoke-Stratos-17k

pipeline_tag: text-generation
base_model: Qwen/Qwen3-32B

model-index:
  - name: Team-Promptia/Qwen3-Stratos-32B
    results:
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          type: bespokelabs/Bespoke-Stratos-17k
          name: Bespoke-Stratos-17k
        metrics:
          - type: "N/A"
            value: "N/A"
            name: "Evaluation not performed"
            description: "No evaluation metrics have been calculated for this model yet."
    model-card:      
      training-data: "Fine-tuned on the 'train' split of the [bespokelabs/Bespoke-Stratos-17k](https://huggingface.co/datasets/bespokelabs/Bespoke-Stratos-17k) dataset."
      hardware: "4 x NVIDIA H100 80GB"
      hours-trained: 32.33 # 32 hours and 20 minutes
      cloud-provider-or-cluster-name: "Private Cluster" # Change if applicable, e.g., 'AWS SageMaker', 'GCP', etc.

      # Detailed training configuration
      training-regime:
        # Core Parameters
        - name: "Base Model"
          value: "Qwen/Qwen3-32B"
        - name: "Max Sequence Length"
          value: 16384
        - name: "Epochs"
          value: 3.0
        
        # Batch Size Configuration
        - name: "Per-Device Batch Size (GPU)"
          value: 1
        - name: "Gradient Accumulation Steps"
          value: 24
        - name: "Effective Batch Size"
          value: 96 # (4 GPUs * 1 bs/gpu * 24 accum)

        # Optimizer & Scheduler
        - name: "Optimizer"
          value: "AdamW (via DeepSpeed)"
        - name: "Learning Rate"
          value: "1e-5"
        - name: "LR Scheduler"
          value: "cosine"
        - name: "Warmup Ratio"
          value: 0.1

        # Precision and Performance
        - name: "Precision"
          value: "bf16"
        - name: "TF32 Enabled"
          value: true
        - name: "Gradient Checkpointing"
          value: true
        
        # Distributed Training
        - name: "Strategy"
          value: "DeepSpeed ZeRO Stage 3"
        - name: "Number of Processes"
          value: 4

      # Software framework versions
      framework-versions:
        - name: "Transformers"
          value: "4.51.1"
        - name: "Pytorch"
          value: "2.6.0+cu124"
        - name: "Datasets"
          value: "3.2.0"
        - name: "Tokenizers"
          value: "0.21.4"
        - name: "DeepSpeed"
          value: "0.15.4"
        - name: "TRL"
          value: "Used SFTTrainer (version not specified, but compatible with above)"
---

# Fine-tuned Qwen3-32B on Bespoke-Stratos-17k

This model is a fine-tuned version of [Qwen/Qwen3-32B](https://huggingface.co/Qwen/Qwen3-32B) on the [Bespoke-Stratos-17k dataset](https://huggingface.co/datasets/bespokelabs/Bespoke-Stratos-17k).

## Training Procedure

### Hardware

The model was trained on a cluster of 4 x NVIDIA H100 80G GPUs.

### Training Time

The total training time was approximately 32 hours and 20 minutes.

### Training Hyperparameters

The model was trained using the `SFTTrainer` from the TRL library, in conjunction with DeepSpeed ZeRO-3 for efficient, distributed training.

The key hyperparameters used during training are listed below:

| Hyperparameter | Value | Description |
| :--- | :--- | :--- |
| **Model** |
| `base_model` | `Qwen/Qwen3-32B` | The foundational model that was fine-tuned. |
| `max_seq_length` | `16384` | Maximum sequence length processed by the model. |
| **Training Strategy** |
| `num_train_epochs` | `3.0` | The total number of training epochs. |
| `per_device_train_batch_size` | `1` | Batch size per GPU. |
| `gradient_accumulation_steps` | `24` | Number of steps to accumulate gradients before an optimizer step. |
| *Effective Batch Size* | *96* | `(4 GPUs * 1 bs/gpu * 24 accum)` |
| **Optimizer & Scheduler** |
| `optimizer` | `AdamW` | As configured in the DeepSpeed config. |
| `learning_rate` | `1e-5` | The peak learning rate. |
| `lr_scheduler_type` | `cosine` | Learning rate scheduler with a cosine decay. |
| `warmup_ratio` | `0.1` | Proportion of training steps for the learning rate warmup. |
| **Precision & Performance** |
| `bf16` | `True` | Used bfloat16 for mixed-precision training. |
| `tf32` | `True` | Allowed the use of TensorFloat-32 on supported hardware. |
| `gradient_checkpointing` | `True` | Enabled to reduce memory consumption. |
| `distributed_type` | `DeepSpeed ZeRO-3` | Distributed training framework and stage. |


### Framework Versions

*   **Transformers**: `4.51.1`
*   **Pytorch**: `2.6.0+cu124`
*   **Datasets**: `3.2.0`
*   **Tokenizers**: `0.21.4`
*   **DeepSpeed**: `0.15.4`

## License

This model and its source code are released under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0).


### Data Format Example

Understanding the data format used for fine-tuning is crucial for effectively using this model. Here is a concrete example of a single training instance.

The raw `input_ids` and `labels` for a sample are as follows:

*   **input_ids**: `a = [151644, 8948, 198, 7771, 3476, 438, 458, 17847, 17601, 26364, 23966, 4755, 1526, 264, 36438, 1293, 7274, 1882, 1573, 8241, 279, 1590, 23560, 323, 13382, 9904, 13, 1096, 7460, 22570, 304, 264, 15817, 10775, 315, 6358, 11, 28285, 4849, 11, 26403, 11, 31734, 23391, 11, 21844, 11, 1182, 376, 4527, 11, 323, 19569, 311, 2225, 1632, 12, 24712, 291, 7274, 1882, 13, 5209, 5944, 697, 2033, 1119, 1378, 1887, 14158, 25, 35187, 323, 12478, 13, 758, 279, 35187, 3772, 11, 7716, 697, 32711, 1882, 1667, 279, 5189, 3561, 25, 82639, 7265, 3575, 5854, 2450, 91, 29, 314, 60565, 448, 7354, 18663, 448, 5196, 77, 1699, 8275, 82639, 408, 3575, 5854, 2450, 91, 29, 8886, 3019, 1265, 2924, 11682, 37764, 1741, 438, 3181, 285, 7024, 4755, 11, 28285, 4849, 9760, 14613, 11, 86781, 287, 501, 6708, 11, 68863, 279, 13403, 315, 279, 1482, 7354, 11, 73185, 894, 5975, 11, 323, 17493, 5853, 3681, 7354, 13, 758, 279, 12478, 3772, 11, 3118, 389, 5257, 13553, 11, 47439, 804, 11, 323, 62751, 504, 279, 35187, 3772, 11, 59726, 3042, 279, 1590, 6291, 429, 498, 81477, 4396, 13, 576, 6291, 1265, 7146, 264, 19819, 11, 13382, 11, 63594, 7493, 1707, 323, 7716, 5871, 3019, 4362, 311, 5545, 279, 16688, 11, 23126, 438, 11017, 25, 82639, 7265, 3575, 50274, 91, 29, 314, 11822, 23126, 11, 23560, 11, 323, 2797, 6291, 92, 82639, 408, 3575, 50274, 91, 29, 4695, 11, 1430, 311, 11625, 279, 2701, 3405, 1526, 279, 3403, 17501, 25, 151645, 198, 151644, 872, 198, 31115, 458, 32156, 13027, 729, 7907, 504, 279, 2661, 9934, 13, 576, 729, 1265, 1896, 31369, 438, 1946, 323, 1173, 279, 2550, 13, 28424, 1618, 279, 729, 1283, 279, 7271, 72672, 11, 44856, 60436, 374, 2661, 308, 65623, 389, 264, 220, 16, 15, 15, 15, 553, 220, 16, 15, 15, 15, 5827, 13, 11733, 6978, 323, 8147, 315, 279, 5827, 525, 48826, 504, 220, 16, 311, 220, 16, 15, 15, 15, 13, 61706, 525, 48826, 504, 1909, 311, 5622, 11, 1393, 8147, 525, 48826, 504, 2115, 311, 1290, 382, 54, 295, 60436, 15482, 429, 1378, 65623, 3359, 1817, 1008, 421, 807, 4332, 279, 1852, 37236, 13, 7036, 11, 429, 419, 374, 279, 1172, 12890, 11, 773, 1378, 65623, 1231, 3359, 1817, 1008, 320, 73846, 311, 44856, 60436, 8, 1496, 421, 1052, 374, 2441, 53206, 7407, 1948, 1105, 13, 4695, 44856, 60436, 6801, 311, 1760, 279, 1372, 315, 13530, 315, 65623, 429, 3359, 1817, 1008, 4192, 15081, 2505, 70674, 785, 1156, 1555, 315, 279, 1946, 5610, 308, 320, 16, 37294, 308, 37294, 220, 17, 15, 15, 220, 15, 15, 15, 8, 4102, 2293, 279, 1372, 315, 65623, 382, 4854, 315, 1790, 308, 5128, 5610, 1378, 3550, 18663, 25780, 856, 15159, 72, 92, 323, 379, 15159, 72, 92, 320, 16, 37294, 856, 15159, 72, 2137, 379, 15159, 72, 92, 37294, 220, 16, 15, 15, 15, 8, 4102, 2293, 279, 1372, 315, 2802, 323, 279, 1372, 315, 3250, 1380, 600, 7563, 53206, 374, 34228, 13, 1084, 594, 19323, 429, 902, 1378, 65623, 4332, 279, 1852, 2309, 4192, 15081, 5097, 70674, 5097, 825, 7546, 4102, 2293, 279, 1372, 315, 13530, 315, 65623, 892, 3359, 1817, 1008, 13, 14731, 15081, 40381, 34764, 2505, 198, 20, 198, 16, 220, 16, 198, 16, 220, 20, 198, 18, 220, 18, 198, 20, 220, 16, 198, 20, 220, 20, 271, 5097, 198, 21, 271, 2505, 198, 18, 198, 16, 220, 16, 198, 17, 220, 18, 198, 18, 220, 20, 271, 5097, 198, 15, 1022, 15081, 9112, 70674, 641, 279, 1156, 6077, 2701, 13530, 315, 65623, 3359, 1817, 1008, 25, 320, 16, 11, 220, 18, 701, 320, 16, 11, 220, 20, 701, 320, 17, 11, 220, 18, 701, 320, 17, 11, 220, 19, 701, 320, 18, 11, 220, 19, 8, 323, 320, 18, 11, 220, 20, 568, 393, 4720, 320, 16, 11, 220, 17, 701, 320, 16, 11, 220, 19, 701, 320, 17, 11, 220, 20, 8, 323, 320, 19, 11, 220, 20, 8, 653, 537, 3359, 1817, 1008, 1576, 807, 653, 537, 4332, 279, 1852, 37236, 624, 151645, 198, 151644, 77091, 198, 151667, 271, 151668, 271, 27, 91, 7265, 3575, 5854, 2450, 91, 1339, 32313, 11, 1077, 594, 1490, 13, 576, 3491, 374, 311, 1760, 279, 1372, 315, 13530, 315, 65623, 429, 3359, 1817, 1008, 389, 264, 220, 16, 15, 15, 15, 87, 16, 15, 15, 15, 5827, 13, 425, 50109, 3359, 421, 807, 525, 389, 279, 1852, 37236, 13, 2055, 1817, 37236, 646, 16792, 1045, 13530, 11, 323, 279, 2790, 374, 279, 2629, 916, 678, 84018, 1127, 315, 279, 10601, 315, 65623, 304, 429, 37236, 4429, 1378, 518, 264, 882, 382, 5338, 11, 358, 1184, 311, 7071, 700, 1246, 311, 1874, 279, 65623, 1119, 862, 19511, 84018, 1127, 13, 2619, 525, 1378, 4494, 315, 84018, 1127, 25, 279, 6174, 429, 30232, 504, 1909, 7950, 311, 5622, 6701, 320, 1149, 594, 1618, 1105, 6785, 84018, 1127, 701, 323, 279, 6174, 429, 30232, 504, 1909, 6701, 311, 5622, 7950, 320, 42224, 84018, 1127, 3593, 2461, 264, 6785, 37236, 11, 279, 1376, 374, 856, 481, 379, 13, 9211, 3156, 1741, 264, 37236, 11, 279, 6672, 856, 481, 379, 8458, 6783, 13, 1752, 3110, 11, 320, 16, 11, 16, 8, 702, 856, 12034, 28, 15, 11, 320, 17, 11, 17, 8, 1083, 220, 15, 11, 4992, 13, 2055, 678, 65623, 389, 279, 1852, 6785, 37236, 686, 614, 279, 1852, 856, 12034, 897, 382, 2461, 279, 8225, 84018, 1127, 11, 279, 2629, 856, 488, 379, 374, 6783, 13, 8909, 320, 16, 11, 20, 8, 323, 320, 17, 11, 19, 8, 2176, 2629, 311, 220, 21, 13, 2055, 65623, 389, 279, 1852, 8225, 37236, 686, 614, 279, 1852, 856, 43010, 382, 4416, 279, 3119, 374, 1447, 16, 13, 1752, 1817, 53206, 11, 12564, 2176, 856, 12034, 323, 856, 43010, 624, 17, 13, 1752, 1817, 315, 1493, 1378, 2750, 11, 1874, 279, 65623, 553, 1105, 624, 18, 13, 1752, 1817, 1874, 448, 595, 65623, 11, 912, 595, 6599, 74, 12, 16, 5620, 17, 311, 279, 2790, 1760, 13, 9211, 1817, 6716, 304, 279, 1874, 42972, 311, 279, 2790, 382, 14190, 11, 1290, 13, 9211, 369, 894, 1874, 315, 1379, 595, 11, 279, 1372, 315, 13530, 374, 356, 5969, 11, 17, 8, 892, 374, 595, 6599, 74, 12, 16, 5620, 17, 382, 4416, 279, 7354, 525, 1447, 12, 54340, 916, 678, 65623, 624, 12, 1752, 1817, 11, 11047, 279, 1378, 6894, 25, 856, 12034, 323, 856, 43010, 624, 12, 5443, 57514, 311, 1760, 279, 1372, 315, 65623, 304, 1817, 37236, 320, 21028, 6785, 323, 8225, 4292, 12, 5005, 11, 369, 1817, 1760, 304, 279, 57514, 11, 12564, 279, 27459, 323, 2629, 1105, 678, 382, 14190, 11, 714, 279, 3491, 5302, 429, 1378, 65623, 3359, 1817, 1008, 421, 807, 4332, 279, 1852, 37236, 13, 1988, 1817, 53206, 374, 949, 315, 1378, 84018, 1127, 320, 30487, 323, 8225, 568, 4354, 11, 1378, 65623, 646, 1172, 3359, 1817, 1008, 421, 807, 525, 389, 279, 1852, 6785, 37236, 2726, 279, 1852, 8225, 37236, 13, 2055, 1817, 6716, 374, 29994, 3055, 421, 807, 4332, 2987, 315, 279, 1378, 84018, 1127, 382, 3983, 1246, 653, 582, 5648, 1990, 26580, 287, 30, 1752, 3110, 11, 1378, 65623, 1410, 387, 389, 279, 1852, 6785, 323, 1852, 8225, 37236, 13, 1988, 429, 594, 11997, 7241, 807, 525, 279, 1852, 1459, 11, 892, 4436, 944, 5420, 1588, 13, 9211, 421, 1378, 65623, 525, 389, 279, 1852, 856, 12034, 323, 1852, 856, 43010, 11, 1221, 21828, 856, 16, 488, 379, 16, 284, 856, 17, 488, 379, 17, 323, 856, 16, 481, 379, 16, 284, 856, 17, 481, 379, 17, 1035, 2968, 856, 16, 284, 856, 17, 323, 379, 16, 284, 379, 17, 11, 892, 646, 944, 3537, 2474, 279, 9892, 525, 4911, 13, 2055, 902, 1378, 65623, 646, 387, 389, 2176, 279, 1852, 6785, 323, 1852, 8225, 37236, 13, 2055, 369, 1817, 6716, 315, 65623, 11, 807, 525, 29994, 2987, 3055, 320, 333, 807, 4332, 2987, 6785, 476, 8225, 37236, 8, 476, 537, 518, 678, 13, 15277, 11, 279, 2629, 315, 279, 27459, 504, 2176, 6785, 323, 8225, 84018, 1127, 686, 2968, 279, 4396, 2790, 382, 4416, 279, 5486, 374, 4396, 13, 2055, 1431, 11, 279, 2038, 7354, 1447, 4418, 308, 382, 12209, 369, 1817, 315, 279, 308, 65623, 11, 1349, 856, 5318, 323, 379, 5318, 382, 46254, 369, 1817, 53206, 1447, 1242, 61304, 284, 856, 488, 379, 198, 13490, 61304, 284, 856, 481, 379, 271, 1654, 1184, 311, 3754, 279, 1372, 315, 65623, 304, 1817, 2629, 61304, 323, 1817, 3638, 61304, 382, 4416, 1378, 57514, 25, 825, 369, 36398, 11, 825, 369, 86947, 382, 12209, 11, 369, 1817, 1376, 304, 36398, 11, 912, 356, 11512, 11, 220, 17, 8, 311, 279, 2790, 382, 19198, 369, 279, 86947, 382, 4416, 279, 2038, 686, 387, 1447, 9928, 2629, 25977, 323, 3638, 25977, 438, 1638, 97599, 320, 269, 5792, 97599, 11, 17271, 311, 220, 15, 3593, 2461, 1817, 856, 11, 379, 1447, 1242, 3097, 284, 856, 488, 379, 271, 13490, 3097, 284, 856, 481, 379, 271, 1242, 25977, 58, 1242, 3097, 60, 1421, 220, 16, 271, 13490, 25977, 58, 13490, 3097, 60, 1421, 220, 16, 271, 12209, 11, 2790, 284, 2629, 7, 320, 66, 6599, 66, 12, 16, 27151, 17, 8, 369, 272, 304, 2629, 25977, 10656, 368, 873, 488, 2629, 7, 320, 66, 6599, 66, 12, 16, 27151, 17, 8, 369, 272, 304, 3638, 25977, 10656, 368, 5125, 4792, 594, 279, 4226, 382, 3983, 2474, 308, 646, 387, 705, 311, 220, 17, 15, 15, 11, 15, 15, 15, 11, 323, 369, 1817, 315, 1105, 582, 1882, 1378, 57514, 11, 419, 1265, 387, 11050, 3322, 13, 9211, 1496, 448, 220, 17, 15, 15, 11, 15, 15, 15, 10695, 11, 279, 57514, 646, 3705, 429, 13, 1597, 279, 2629, 374, 506, 1255, 701, 1380, 296, 374, 279, 1372, 315, 6894, 304, 1817, 10997, 11, 892, 374, 518, 1429, 220, 17, 15, 15, 11, 15, 15, 15, 353, 220, 17, 304, 279, 11785, 1142, 11, 892, 374, 70028, 382, 16451, 279, 1156, 3110, 1447, 2505, 220, 20, 1447, 45793, 525, 320, 16, 11, 16, 701, 320, 16, 11, 20, 701, 320, 18, 11, 18, 701, 320, 20, 11, 16, 701, 320, 20, 11, 20, 3593, 22464, 518, 2629, 61304, 1447, 7, 16, 10, 16, 11730, 17, 11, 320, 16, 10, 20, 11730, 21, 11, 320, 18, 10, 18, 11730, 21, 11, 320, 20, 10, 16, 11730, 21, 11, 320, 20, 10, 20, 11730, 16, 15, 382, 4416, 2629, 25977, 1447, 17, 25, 220, 16, 271, 21, 25, 220, 18, 320, 27436, 220, 16, 11, 20, 26, 220, 18, 11, 18, 26, 20, 11, 16, 57777, 14190, 3783, 11, 320, 16, 11, 20, 8, 2629, 374, 220, 21, 11, 320, 18, 11, 18, 8, 2629, 374, 220, 21, 11, 320, 20, 11, 16, 8, 2629, 374, 220, 21, 13, 2055, 2629, 220, 21, 702, 1760, 220, 18, 382, 12209, 11, 2629, 18527, 374, 220, 18, 5157, 220, 17, 284, 220, 18, 382, 785, 1008, 2629, 14579, 525, 220, 17, 25, 16, 11, 220, 16, 15, 25, 16, 13, 2055, 2629, 18527, 374, 220, 18, 382, 2461, 279, 3638, 61304, 1447, 7, 16, 12, 16, 11730, 15, 11, 320, 16, 12, 20, 8, 10829, 19, 11, 320, 18, 12, 18, 11730, 15, 11, 320, 20, 12, 16, 11730, 19, 11, 320, 20, 12, 20, 11730, 15, 382, 13490, 25977, 1447, 15, 25, 18, 320, 27436, 220, 16, 11, 16, 26, 18, 11, 18, 26, 20, 11, 20, 692, 12, 19, 25, 16, 271, 19, 25, 16, 382, 4416, 3638, 18527, 374, 220, 18, 5157, 220, 17, 284, 220, 18, 382, 4416, 2790, 2629, 374, 220, 18, 320, 1242, 8, 488, 220, 18, 320, 13490, 8, 284, 220, 21, 13, 15920, 9071, 279, 1156, 3110, 594, 2550, 382, 9454, 11, 429, 594, 4396, 382, 14037, 3110, 1447, 2505, 220, 18, 1447, 7, 16, 11, 16, 701, 320, 17, 11, 18, 701, 320, 18, 11, 20, 3593, 1242, 61304, 1447, 16, 10, 16, 28, 17, 320, 1830, 220, 16, 692, 17, 10, 18, 28, 20, 320, 1830, 220, 16, 692, 18, 10, 20, 28, 23, 320, 1830, 220, 16, 568, 8116, 18527, 220, 15, 382, 13490, 61304, 1447, 16, 12, 16, 28, 15, 320, 1830, 16, 692, 17, 12, 18, 284, 481, 16, 320, 1830, 16, 692, 18, 12, 20, 284, 481, 17, 320, 1830, 16, 568, 2055, 3638, 18527, 220, 15, 382, 7595, 220, 15, 13, 39970, 382, 4416, 279, 2038, 1265, 975, 382, 7039, 11, 10822, 419, 304, 13027, 382, 785, 2038, 1265, 1349, 504, 31369, 13, 5005, 11, 12564, 279, 2629, 323, 3638, 14579, 382, 62980, 438, 1447, 4418, 308, 382, 1242, 25977, 284, 42908, 1548, 692, 13490, 25977, 284, 42908, 1548, 692, 1958, 716, 304, 2088, 1445, 7731, 262, 856, 11, 379, 284, 2415, 1548, 11, 1946, 1005, 6960, 12367, 262, 274, 284, 856, 488, 379, 271, 262, 294, 284, 856, 481, 379, 271, 262, 2629, 25977, 11880, 60, 1421, 220, 16, 271, 262, 3638, 25977, 21427, 60, 1421, 220, 16, 271, 5035, 284, 220, 15, 271, 1958, 272, 304, 2629, 25977, 10656, 39974, 262, 2790, 1421, 272, 353, 320, 66, 12, 16, 8, 442, 220, 17, 271, 1958, 272, 304, 3638, 25977, 10656, 39974, 262, 2790, 1421, 272, 353, 320, 66, 12, 16, 8, 442, 220, 17, 271, 1350, 22842, 692, 9454, 11, 429, 594, 30339, 382, 16451, 448, 279, 1156, 3110, 1447, 1242, 25977, 369, 220, 21, 374, 220, 18, 320, 1499, 320, 16, 11, 20, 701, 320, 18, 11, 18, 701, 320, 20, 11, 16, 4579, 2055, 220, 18, 9, 17, 14, 17, 284, 18, 382, 13490, 25977, 369, 220, 15, 374, 220, 18, 320, 16, 11, 16, 26, 18, 11, 18, 26, 20, 11, 20, 8, 11397, 220, 18, 13, 2055, 2790, 220, 21, 382, 14037, 3110, 11, 438, 1573, 382, 3983, 279, 2038, 15804, 504, 31369, 13, 2055, 279, 729, 1265, 387, 5326, 311, 1349, 11127, 11, 1882, 11, 323, 2550, 382, 3983, 279, 3491, 2727, 311, 6923, 458, 32156, 13027, 729, 429, 15804, 31369, 323, 23473, 279, 2550, 382, 4416, 279, 2038, 1035, 387, 1447, 474, 5708, 198, 1499, 15302, 1159, 42908, 271, 750, 1887, 3932, 262, 308, 284, 526, 18140, 43184, 31800, 2398, 262, 2629, 25977, 284, 42908, 1548, 340, 262, 3638, 25977, 284, 42908, 1548, 340, 262, 369, 716, 304, 2088, 1445, 982, 286, 856, 11, 379, 284, 2415, 1548, 11, 5708, 43184, 31800, 1005, 6960, 2398, 286, 2629, 3097, 284, 856, 488, 379, 198, 286, 3638, 3097, 284, 856, 481, 379, 198, 286, 2629, 25977, 58, 1242, 3097, 60, 1421, 220, 16, 198, 286, 3638, 25977, 58, 13490, 3097, 60, 1421, 220, 16, 198, 262, 2790, 284, 220, 15, 198, 262, 369, 13195, 304, 2629, 25977, 10656, 3932, 286, 2790, 1421, 13195, 353, 320, 15853, 481, 16, 8, 442, 17, 198, 262, 369, 13195, 304, 3638, 25977, 10656, 3932, 286, 2790, 1421, 13195, 353, 320, 15853, 481, 16, 8, 442, 17, 198, 262, 1173, 22842, 692, 3817, 2822, 3983, 304, 279, 3491, 5114, 11, 279, 729, 1265, 387, 2598, 1283, 7271, 13, 2055, 429, 594, 4396, 382, 9454, 11, 429, 594, 279, 2038, 13, 1084, 1265, 3705, 678, 5048, 382, 11656, 46024, 1447, 77, 28, 16, 25, 2550, 220, 15, 382, 77, 28, 17, 11, 1852, 2629, 476, 1852, 3638, 25, 2550, 220, 16, 382, 3983, 279, 2038, 4278, 382, 14037, 1142, 25, 65623, 304, 1852, 2629, 714, 2155, 86947, 13, 2055, 807, 16792, 220, 16, 6716, 13, 2521, 1852, 3638, 714, 2155, 2629, 11, 16792, 220, 16, 382, 4416, 2038, 4977, 4396, 382, 27, 91, 408, 3575, 5854, 2450, 91, 1339, 27, 91, 7265, 3575, 50274, 91, 1339, 1249, 11625, 419, 3491, 11, 582, 1184, 311, 8253, 279, 1372, 315, 13530, 315, 65623, 429, 3359, 1817, 1008, 389, 264, 220, 16, 15, 15, 15, 87, 16, 15, 15, 15, 5827, 13, 425, 50109, 3359, 1817, 1008, 421, 807, 525, 389, 279, 1852, 37236, 11, 2987, 504, 1909, 7950, 311, 5622, 6701, 320, 30487, 37236, 8, 476, 504, 1909, 6701, 311, 5622, 7950, 320, 42224, 37236, 3593, 14374, 53084, 198, 16, 13, 3070, 28301, 1437, 7767, 6113, 1127, 95518, 8886, 53206, 646, 387, 949, 315, 1378, 84018, 1127, 510, 256, 481, 3070, 35490, 7767, 23450, 95518, 15633, 1506, 553, 279, 6672, 17767, 87, 481, 379, 59, 701, 892, 8458, 6783, 369, 678, 9892, 389, 279, 1852, 6785, 37236, 624, 256, 481, 3070, 38489, 7767, 23450, 95518, 15633, 1506, 553, 279, 2629, 17767, 87, 488, 379, 59, 701, 892, 8458, 6783, 369, 678, 9892, 389, 279, 1852, 8225, 37236, 382, 17, 13, 3070, 2507, 425, 50109, 389, 8886, 7767, 23450, 95518, 5443, 1378, 57514, 311, 1760, 279, 1372, 315, 65623, 389, 1817, 6785, 323, 8225, 37236, 13, 576, 1376, 369, 6785, 84018, 1127, 374, 17767, 87, 481, 379, 57758, 323, 369, 8225, 84018, 1127, 374, 17767, 87, 488, 379, 59, 3593, 18, 13, 3070, 47866, 393, 4720, 95518, 1752, 1817, 37236, 320, 21028, 6785, 323, 8225, 701, 421, 1052, 525, 17767, 74, 57758, 65623, 389, 264, 37236, 11, 279, 1372, 315, 22818, 13530, 374, 2661, 553, 279, 10601, 14806, 17767, 34, 5969, 11, 220, 17, 8, 284, 595, 1124, 15136, 320, 74, 481, 220, 16, 8, 608, 220, 17, 59, 568, 8116, 1493, 2750, 369, 678, 84018, 1127, 311, 633, 279, 2790, 1372, 315, 22818, 13530, 382, 14374, 12478, 6119, 198, 73594, 12669, 198, 474, 5708, 198, 1499, 15302, 1159, 42908, 271, 750, 1887, 3932, 262, 308, 284, 526, 18140, 43184, 31800, 2398, 262, 2629, 25977, 284, 42908, 1548, 340, 262, 3638, 25977, 284, 42908, 1548, 340, 262, 369, 716, 304, 2088, 1445, 982, 286, 856, 11, 379, 284, 2415, 1548, 11, 5708, 43184, 31800, 1005, 6960, 2398, 286, 2629, 3097, 284, 856, 488, 379, 198, 286, 3638, 3097, 284, 856, 481, 379, 198, 286, 2629, 25977, 58, 1242, 3097, 60, 1421, 220, 16, 198, 286, 3638, 25977, 58, 13490, 3097, 60, 1421, 220, 16, 198, 262, 2790, 284, 220, 15, 198, 262, 369, 13195, 304, 2629, 25977, 10656, 3932, 286, 2790, 1421, 13195, 353, 320, 15853, 481, 220, 16, 8, 442, 220, 17, 198, 262, 369, 13195, 304, 3638, 25977, 10656, 3932, 286, 2790, 1421, 13195, 353, 320, 15853, 481, 220, 16, 8, 442, 220, 17, 198, 262, 1173, 22842, 692, 3817, 741, 13874, 19324, 14374, 71287, 198, 16, 13, 3070, 31899, 5571, 95518, 576, 1946, 374, 1349, 504, 5297, 1946, 13, 576, 1156, 1555, 5610, 279, 1372, 315, 65623, 11, 8110, 553, 862, 9892, 624, 17, 13, 3070, 37119, 7767, 6113, 1127, 95518, 9043, 57514, 11, 1565, 1242, 25977, 63, 323, 1565, 13490, 25977, 7808, 525, 1483, 311, 1760, 279, 1372, 315, 65623, 389, 1817, 8225, 323, 6785, 37236, 11, 15576, 624, 18, 13, 3070, 57908, 1095, 393, 4720, 95518, 1752, 1817, 37236, 320, 1242, 323, 6672, 701, 582, 11047, 279, 1372, 315, 13530, 1667, 279, 10601, 14806, 323, 2629, 1493, 2750, 311, 633, 279, 2790, 1372, 315, 22818, 13530, 382, 1986, 5486, 29720, 5203, 65623, 553, 862, 84018, 1127, 323, 47417, 279, 22818, 13530, 304, 13482, 882, 11, 3259, 432, 14452, 369, 3460, 1946, 12282, 705, 311, 220, 17, 15, 15, 11, 15, 15, 15, 65623, 382, 27, 91, 408, 3575, 50274, 91, 29, 151645, 198]`
*   **labels**: `[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 151667, 271, 151668, 271, 27, 91, 7265, 3575, 5854, 2450, 91, 1339, 32313, 11, 1077, 594, 1490, 13, 576, 3491, 374, 311, 1760, 279, 1372, 315, 13530, 315, 65623, 429, 3359, 1817, 1008, 389, 264, 220, 16, 15, 15, 15, 87, 16, 15, 15, 15, 5827, 13, 425, 50109, 3359, 421, 807, 525, 389, 279, 1852, 37236, 13, 2055, 1817, 37236, 646, 16792, 1045, 13530, 11, 323, 279, 2790, 374, 279, 2629, 916, 678, 84018, 1127, 315, 279, 10601, 315, 65623, 304, 429, 37236, 4429, 1378, 518, 264, 882, 382, 5338, 11, 358, 1184, 311, 7071, 700, 1246, 311, 1874, 279, 65623, 1119, 862, 19511, 84018, 1127, 13, 2619, 525, 1378, 4494, 315, 84018, 1127, 25, 279, 6174, 429, 30232, 504, 1909, 7950, 311, 5622, 6701, 320, 1149, 594, 1618, 1105, 6785, 84018, 1127, 701, 323, 279, 6174, 429, 30232, 504, 1909, 6701, 311, 5622, 7950, 320, 42224, 84018, 1127, 3593, 2461, 264, 6785, 37236, 11, 279, 1376, 374, 856, 481, 379, 13, 9211, 3156, 1741, 264, 37236, 11, 279, 6672, 856, 481, 379, 8458, 6783, 13, 1752, 3110, 11, 320, 16, 11, 16, 8, 702, 856, 12034, 28, 15, 11, 320, 17, 11, 17, 8, 1083, 220, 15, 11, 4992, 13, 2055, 678, 65623, 389, 279, 1852, 6785, 37236, 686, 614, 279, 1852, 856, 12034, 897, 382, 2461, 279, 8225, 84018, 1127, 11, 279, 2629, 856, 488, 379, 374, 6783, 13, 8909, 320, 16, 11, 20, 8, 323, 320, 17, 11, 19, 8, 2176, 2629, 311, 220, 21, 13, 2055, 65623, 389, 279, 1852, 8225, 37236, 686, 614, 279, 1852, 856, 43010, 382, 4416, 279, 3119, 374, 1447, 16, 13, 1752, 1817, 53206, 11, 12564, 2176, 856, 12034, 323, 856, 43010, 624, 17, 13, 1752, 1817, 315, 1493, 1378, 2750, 11, 1874, 279, 65623, 553, 1105, 624, 18, 13, 1752, 1817, 1874, 448, 595, 65623, 11, 912, 595, 6599, 74, 12, 16, 5620, 17, 311, 279, 2790, 1760, 13, 9211, 1817, 6716, 304, 279, 1874, 42972, 311, 279, 2790, 382, 14190, 11, 1290, 13, 9211, 369, 894, 1874, 315, 1379, 595, 11, 279, 1372, 315, 13530, 374, 356, 5969, 11, 17, 8, 892, 374, 595, 6599, 74, 12, 16, 5620, 17, 382, 4416, 279, 7354, 525, 1447, 12, 54340, 916, 678, 65623, 624, 12, 1752, 1817, 11, 11047, 279, 1378, 6894, 25, 856, 12034, 323, 856, 43010, 624, 12, 5443, 57514, 311, 1760, 279, 1372, 315, 65623, 304, 1817, 37236, 320, 21028, 6785, 323, 8225, 4292, 12, 5005, 11, 369, 1817, 1760, 304, 279, 57514, 11, 12564, 279, 27459, 323, 2629, 1105, 678, 382, 14190, 11, 714, 279, 3491, 5302, 429, 1378, 65623, 3359, 1817, 1008, 421, 807, 4332, 279, 1852, 37236, 13, 1988, 1817, 53206, 374, 949, 315, 1378, 84018, 1127, 320, 30487, 323, 8225, 568, 4354, 11, 1378, 65623, 646, 1172, 3359, 1817, 1008, 421, 807, 525, 389, 279, 1852, 6785, 37236, 2726, 279, 1852, 8225, 37236, 13, 2055, 1817, 6716, 374, 29994, 3055, 421, 807, 4332, 2987, 315, 279, 1378, 84018, 1127, 382, 3983, 1246, 653, 582, 5648, 1990, 26580, 287, 30, 1752, 3110, 11, 1378, 65623, 1410, 387, 389, 279, 1852, 6785, 323, 1852, 8225, 37236, 13, 1988, 429, 594, 11997, 7241, 807, 525, 279, 1852, 1459, 11, 892, 4436, 944, 5420, 1588, 13, 9211, 421, 1378, 65623, 525, 389, 279, 1852, 856, 12034, 323, 1852, 856, 43010, 11, 1221, 21828, 856, 16, 488, 379, 16, 284, 856, 17, 488, 379, 17, 323, 856, 16, 481, 379, 16, 284, 856, 17, 481, 379, 17, 1035, 2968, 856, 16, 284, 856, 17, 323, 379, 16, 284, 379, 17, 11, 892, 646, 944, 3537, 2474, 279, 9892, 525, 4911, 13, 2055, 902, 1378, 65623, 646, 387, 389, 2176, 279, 1852, 6785, 323, 1852, 8225, 37236, 13, 2055, 369, 1817, 6716, 315, 65623, 11, 807, 525, 29994, 2987, 3055, 320, 333, 807, 4332, 2987, 6785, 476, 8225, 37236, 8, 476, 537, 518, 678, 13, 15277, 11, 279, 2629, 315, 279, 27459, 504, 2176, 6785, 323, 8225, 84018, 1127, 686, 2968, 279, 4396, 2790, 382, 4416, 279, 5486, 374, 4396, 13, 2055, 1431, 11, 279, 2038, 7354, 1447, 4418, 308, 382, 12209, 369, 1817, 315, 279, 308, 65623, 11, 1349, 856, 5318, 323, 379, 5318, 382, 46254, 369, 1817, 53206, 1447, 1242, 61304, 284, 856, 488, 379, 198, 13490, 61304, 284, 856, 481, 379, 271, 1654, 1184, 311, 3754, 279, 1372, 315, 65623, 304, 1817, 2629, 61304, 323, 1817, 3638, 61304, 382, 4416, 1378, 57514, 25, 825, 369, 36398, 11, 825, 369, 86947, 382, 12209, 11, 369, 1817, 1376, 304, 36398, 11, 912, 356, 11512, 11, 220, 17, 8, 311, 279, 2790, 382, 19198, 369, 279, 86947, 382, 4416, 279, 2038, 686, 387, 1447, 9928, 2629, 25977, 323, 3638, 25977, 438, 1638, 97599, 320, 269, 5792, 97599, 11, 17271, 311, 220, 15, 3593, 2461, 1817, 856, 11, 379, 1447, 1242, 3097, 284, 856, 488, 379, 271, 13490, 3097, 284, 856, 481, 379, 271, 1242, 25977, 58, 1242, 3097, 60, 1421, 220, 16, 271, 13490, 25977, 58, 13490, 3097, 60, 1421, 220, 16, 271, 12209, 11, 2790, 284, 2629, 7, 320, 66, 6599, 66, 12, 16, 27151, 17, 8, 369, 272, 304, 2629, 25977, 10656, 368, 873, 488, 2629, 7, 320, 66, 6599, 66, 12, 16, 27151, 17, 8, 369, 272, 304, 3638, 25977, 10656, 368, 5125, 4792, 594, 279, 4226, 382, 3983, 2474, 308, 646, 387, 705, 311, 220, 17, 15, 15, 11, 15, 15, 15, 11, 323, 369, 1817, 315, 1105, 582, 1882, 1378, 57514, 11, 419, 1265, 387, 11050, 3322, 13, 9211, 1496, 448, 220, 17, 15, 15, 11, 15, 15, 15, 10695, 11, 279, 57514, 646, 3705, 429, 13, 1597, 279, 2629, 374, 506, 1255, 701, 1380, 296, 374, 279, 1372, 315, 6894, 304, 1817, 10997, 11, 892, 374, 518, 1429, 220, 17, 15, 15, 11, 15, 15, 15, 353, 220, 17, 304, 279, 11785, 1142, 11, 892, 374, 70028, 382, 16451, 279, 1156, 3110, 1447, 2505, 220, 20, 1447, 45793, 525, 320, 16, 11, 16, 701, 320, 16, 11, 20, 701, 320, 18, 11, 18, 701, 320, 20, 11, 16, 701, 320, 20, 11, 20, 3593, 22464, 518, 2629, 61304, 1447, 7, 16, 10, 16, 11730, 17, 11, 320, 16, 10, 20, 11730, 21, 11, 320, 18, 10, 18, 11730, 21, 11, 320, 20, 10, 16, 11730, 21, 11, 320, 20, 10, 20, 11730, 16, 15, 382, 4416, 2629, 25977, 1447, 17, 25, 220, 16, 271, 21, 25, 220, 18, 320, 27436, 220, 16, 11, 20, 26, 220, 18, 11, 18, 26, 20, 11, 16, 57777, 14190, 3783, 11, 320, 16, 11, 20, 8, 2629, 374, 220, 21, 11, 320, 18, 11, 18, 8, 2629, 374, 220, 21, 11, 320, 20, 11, 16, 8, 2629, 374, 220, 21, 13, 2055, 2629, 220, 21, 702, 1760, 220, 18, 382, 12209, 11, 2629, 18527, 374, 220, 18, 5157, 220, 17, 284, 220, 18, 382, 785, 1008, 2629, 14579, 525, 220, 17, 25, 16, 11, 220, 16, 15, 25, 16, 13, 2055, 2629, 18527, 374, 220, 18, 382, 2461, 279, 3638, 61304, 1447, 7, 16, 12, 16, 11730, 15, 11, 320, 16, 12, 20, 8, 10829, 19, 11, 320, 18, 12, 18, 11730, 15, 11, 320, 20, 12, 16, 11730, 19, 11, 320, 20, 12, 20, 11730, 15, 382, 13490, 25977, 1447, 15, 25, 18, 320, 27436, 220, 16, 11, 16, 26, 18, 11, 18, 26, 20, 11, 20, 692, 12, 19, 25, 16, 271, 19, 25, 16, 382, 4416, 3638, 18527, 374, 220, 18, 5157, 220, 17, 284, 220, 18, 382, 4416, 2790, 2629, 374, 220, 18, 320, 1242, 8, 488, 220, 18, 320, 13490, 8, 284, 220, 21, 13, 15920, 9071, 279, 1156, 3110, 594, 2550, 382, 9454, 11, 429, 594, 4396, 382, 14037, 3110, 1447, 2505, 220, 18, 1447, 7, 16, 11, 16, 701, 320, 17, 11, 18, 701, 320, 18, 11, 20, 3593, 1242, 61304, 1447, 16, 10, 16, 28, 17, 320, 1830, 220, 16, 692, 17, 10, 18, 28, 20, 320, 1830, 220, 16, 692, 18, 10, 20, 28, 23, 320, 1830, 220, 16, 568, 8116, 18527, 220, 15, 382, 13490, 61304, 1447, 16, 12, 16, 28, 15, 320, 1830, 16, 692, 17, 12, 18, 284, 481, 16, 320, 1830, 16, 692, 18, 12, 20, 284, 481, 17, 320, 1830, 16, 568, 2055, 3638, 18527, 220, 15, 382, 7595, 220, 15, 13, 39970, 382, 4416, 279, 2038, 1265, 975, 382, 7039, 11, 10822, 419, 304, 13027, 382, 785, 2038, 1265, 1349, 504, 31369, 13, 5005, 11, 12564, 279, 2629, 323, 3638, 14579, 382, 62980, 438, 1447, 4418, 308, 382, 1242, 25977, 284, 42908, 1548, 692, 13490, 25977, 284, 42908, 1548, 692, 1958, 716, 304, 2088, 1445, 7731, 262, 856, 11, 379, 284, 2415, 1548, 11, 1946, 1005, 6960, 12367, 262, 274, 284, 856, 488, 379, 271, 262, 294, 284, 856, 481, 379, 271, 262, 2629, 25977, 11880, 60, 1421, 220, 16, 271, 262, 3638, 25977, 21427, 60, 1421, 220, 16, 271, 5035, 284, 220, 15, 271, 1958, 272, 304, 2629, 25977, 10656, 39974, 262, 2790, 1421, 272, 353, 320, 66, 12, 16, 8, 442, 220, 17, 271, 1958, 272, 304, 3638, 25977, 10656, 39974, 262, 2790, 1421, 272, 353, 320, 66, 12, 16, 8, 442, 220, 17, 271, 1350, 22842, 692, 9454, 11, 429, 594, 30339, 382, 16451, 448, 279, 1156, 3110, 1447, 1242, 25977, 369, 220, 21, 374, 220, 18, 320, 1499, 320, 16, 11, 20, 701, 320, 18, 11, 18, 701, 320, 20, 11, 16, 4579, 2055, 220, 18, 9, 17, 14, 17, 284, 18, 382, 13490, 25977, 369, 220, 15, 374, 220, 18, 320, 16, 11, 16, 26, 18, 11, 18, 26, 20, 11, 20, 8, 11397, 220, 18, 13, 2055, 2790, 220, 21, 382, 14037, 3110, 11, 438, 1573, 382, 3983, 279, 2038, 15804, 504, 31369, 13, 2055, 279, 729, 1265, 387, 5326, 311, 1349, 11127, 11, 1882, 11, 323, 2550, 382, 3983, 279, 3491, 2727, 311, 6923, 458, 32156, 13027, 729, 429, 15804, 31369, 323, 23473, 279, 2550, 382, 4416, 279, 2038, 1035, 387, 1447, 474, 5708, 198, 1499, 15302, 1159, 42908, 271, 750, 1887, 3932, 262, 308, 284, 526, 18140, 43184, 31800, 2398, 262, 2629, 25977, 284, 42908, 1548, 340, 262, 3638, 25977, 284, 42908, 1548, 340, 262, 369, 716, 304, 2088, 1445, 982, 286, 856, 11, 379, 284, 2415, 1548, 11, 5708, 43184, 31800, 1005, 6960, 2398, 286, 2629, 3097, 284, 856, 488, 379, 198, 286, 3638, 3097, 284, 856, 481, 379, 198, 286, 2629, 25977, 58, 1242, 3097, 60, 1421, 220, 16, 198, 286, 3638, 25977, 58, 13490, 3097, 60, 1421, 220, 16, 198, 262, 2790, 284, 220, 15, 198, 262, 369, 13195, 304, 2629, 25977, 10656, 3932, 286, 2790, 1421, 13195, 353, 320, 15853, 481, 16, 8, 442, 17, 198, 262, 369, 13195, 304, 3638, 25977, 10656, 3932, 286, 2790, 1421, 13195, 353, 320, 15853, 481, 16, 8, 442, 17, 198, 262, 1173, 22842, 692, 3817, 2822, 3983, 304, 279, 3491, 5114, 11, 279, 729, 1265, 387, 2598, 1283, 7271, 13, 2055, 429, 594, 4396, 382, 9454, 11, 429, 594, 279, 2038, 13, 1084, 1265, 3705, 678, 5048, 382, 11656, 46024, 1447, 77, 28, 16, 25, 2550, 220, 15, 382, 77, 28, 17, 11, 1852, 2629, 476, 1852, 3638, 25, 2550, 220, 16, 382, 3983, 279, 2038, 4278, 382, 14037, 1142, 25, 65623, 304, 1852, 2629, 714, 2155, 86947, 13, 2055, 807, 16792, 220, 16, 6716, 13, 2521, 1852, 3638, 714, 2155, 2629, 11, 16792, 220, 16, 382, 4416, 2038, 4977, 4396, 382, 27, 91, 408, 3575, 5854, 2450, 91, 1339, 27, 91, 7265, 3575, 50274, 91, 1339, 1249, 11625, 419, 3491, 11, 582, 1184, 311, 8253, 279, 1372, 315, 13530, 315, 65623, 429, 3359, 1817, 1008, 389, 264, 220, 16, 15, 15, 15, 87, 16, 15, 15, 15, 5827, 13, 425, 50109, 3359, 1817, 1008, 421, 807, 525, 389, 279, 1852, 37236, 11, 2987, 504, 1909, 7950, 311, 5622, 6701, 320, 30487, 37236, 8, 476, 504, 1909, 6701, 311, 5622, 7950, 320, 42224, 37236, 3593, 14374, 53084, 198, 16, 13, 3070, 28301, 1437, 7767, 6113, 1127, 95518, 8886, 53206, 646, 387, 949, 315, 1378, 84018, 1127, 510, 256, 481, 3070, 35490, 7767, 23450, 95518, 15633, 1506, 553, 279, 6672, 17767, 87, 481, 379, 59, 701, 892, 8458, 6783, 369, 678, 9892, 389, 279, 1852, 6785, 37236, 624, 256, 481, 3070, 38489, 7767, 23450, 95518, 15633, 1506, 553, 279, 2629, 17767, 87, 488, 379, 59, 701, 892, 8458, 6783, 369, 678, 9892, 389, 279, 1852, 8225, 37236, 382, 17, 13, 3070, 2507, 425, 50109, 389, 8886, 7767, 23450, 95518, 5443, 1378, 57514, 311, 1760, 279, 1372, 315, 65623, 389, 1817, 6785, 323, 8225, 37236, 13, 576, 1376, 369, 6785, 84018, 1127, 374, 17767, 87, 481, 379, 57758, 323, 369, 8225, 84018, 1127, 374, 17767, 87, 488, 379, 59, 3593, 18, 13, 3070, 47866, 393, 4720, 95518, 1752, 1817, 37236, 320, 21028, 6785, 323, 8225, 701, 421, 1052, 525, 17767, 74, 57758, 65623, 389, 264, 37236, 11, 279, 1372, 315, 22818, 13530, 374, 2661, 553, 279, 10601, 14806, 17767, 34, 5969, 11, 220, 17, 8, 284, 595, 1124, 15136, 320, 74, 481, 220, 16, 8, 608, 220, 17, 59, 568, 8116, 1493, 2750, 369, 678, 84018, 1127, 311, 633, 279, 2790, 1372, 315, 22818, 13530, 382, 14374, 12478, 6119, 198, 73594, 12669, 198, 474, 5708, 198, 1499, 15302, 1159, 42908, 271, 750, 1887, 3932, 262, 308, 284, 526, 18140, 43184, 31800, 2398, 262, 2629, 25977, 284, 42908, 1548, 340, 262, 3638, 25977, 284, 42908, 1548, 340, 262, 369, 716, 304, 2088, 1445, 982, 286, 856, 11, 379, 284, 2415, 1548, 11, 5708, 43184, 31800, 1005, 6960, 2398, 286, 2629, 3097, 284, 856, 488, 379, 198, 286, 3638, 3097, 284, 856, 481, 379, 198, 286, 2629, 25977, 58, 1242, 3097, 60, 1421, 220, 16, 198, 286, 3638, 25977, 58, 13490, 3097, 60, 1421, 220, 16, 198, 262, 2790, 284, 220, 15, 198, 262, 369, 13195, 304, 2629, 25977, 10656, 3932, 286, 2790, 1421, 13195, 353, 320, 15853, 481, 220, 16, 8, 442, 220, 17, 198, 262, 369, 13195, 304, 3638, 25977, 10656, 3932, 286, 2790, 1421, 13195, 353, 320, 15853, 481, 220, 16, 8, 442, 220, 17, 198, 262, 1173, 22842, 692, 3817, 741, 13874, 19324, 14374, 71287, 198, 16, 13, 3070, 31899, 5571, 95518, 576, 1946, 374, 1349, 504, 5297, 1946, 13, 576, 1156, 1555, 5610, 279, 1372, 315, 65623, 11, 8110, 553, 862, 9892, 624, 17, 13, 3070, 37119, 7767, 6113, 1127, 95518, 9043, 57514, 11, 1565, 1242, 25977, 63, 323, 1565, 13490, 25977, 7808, 525, 1483, 311, 1760, 279, 1372, 315, 65623, 389, 1817, 8225, 323, 6785, 37236, 11, 15576, 624, 18, 13, 3070, 57908, 1095, 393, 4720, 95518, 1752, 1817, 37236, 320, 1242, 323, 6672, 701, 582, 11047, 279, 1372, 315, 13530, 1667, 279, 10601, 14806, 323, 2629, 1493, 2750, 311, 633, 279, 2790, 1372, 315, 22818, 13530, 382, 1986, 5486, 29720, 5203, 65623, 553, 862, 84018, 1127, 323, 47417, 279, 22818, 13530, 304, 13482, 882, 11, 3259, 432, 14452, 369, 3460, 1946, 12282, 705, 311, 220, 17, 15, 15, 11, 15, 15, 15, 65623, 382, 27, 91, 408, 3575, 50274, 91, 29, 151645, 198]`

The model is trained to predict the tokens corresponding to the `assistant`'s response. Therefore, the `labels` are set to `-100` for the input part of the text, which masks them from the loss calculation. The model's task is to predict the subsequent tokens.

Here is a token-by-token comparison of the text with its corresponding `input_ids` and `labels`:

[FILE](https://huggingface.co/Team-Promptia/Qwen3-Stratos-32B/blob/main/token-by-token_comparison.txt)

### How to Use This Model

You can use the following code snippet to load the model and run inference. This example demonstrates how to structure the prompt to elicit a detailed thought process followed by a final solution.

```python
# Adapted from https://huggingface.co/Qwen/Qwen3-32B
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Team-Promptia/Qwen3-Stratos-32B"

# load the tokenizer and the model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)

SYSTEM_PROMPT = "Your role as an assistant involves thoroughly exploring questions through a systematic long thinking process before providing the final precise and accurate solutions. This requires engaging in a comprehensive cycle of analysis, summarizing, exploration, reassessment, reflection, backtracing, and iteration to develop well-considered thinking process. Please structure your response into two main sections: Thought and Solution. In the Thought section, detail your reasoning process using the specified format: <|begin_of_thought|> {thought with steps separated with '\\n\\n'} <|end_of_thought|> Each step should include detailed considerations such as analisying questions, summarizing relevant findings, brainstorming new ideas, verifying the accuracy of the current steps, refining any errors, and revisiting previous steps. In the Solution section, based on various attempts, explorations, and reflections from the Thought section, systematically present the final solution that you deem correct. The solution should remain a logical, accurate, concise expression style and detail necessary step needed to reach the conclusion, formatted as follows: <|begin_of_solution|> {final formatted, precise, and clear solution} <|end_of_solution|> Now, try to solve the following question through the above guidelines:"
# QUESTION from HuggingFaceH4/aime_2024
QUESTION = "Every morning Aya goes for a $9$-kilometer-long walk and stops at a coffee shop afterwards. When she walks at a constant speed of $s$ kilometers per hour, the walk takes her 4 hours, including $t$ minutes spent in the coffee shop. When she walks $s+2$ kilometers per hour, the walk takes her 2 hours and 24 minutes, including $t$ minutes spent in the coffee shop. Suppose Aya walks at $s+\frac{1}{2}$ kilometers per hour. Find the number of minutes the walk takes her, including the $t$ minutes spent in the coffee shop."
messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": QUESTION}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False # Switches between thinking and non-thinking modes. Default is True.
)
text += "<|begin_of_thought|>\n\n"
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

# conduct text completion
generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=16384
)
output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() 

output = tokenizer.decode(output_ids, skip_special_tokens=True).strip("\n")

print(output)
```

### Example Output

In one test run, the `output` generated by the code above was:

[FILE](https://huggingface.co/Team-Promptia/Qwen3-Stratos-32B/blob/main/output_for_display.txt)