README.md

---
library_name: transformers
base_model: DeepPavlov/rubert-base-cased
language:
  - ru
tags:
  - text-classification
  - bert
  - safetensors
  - multilabel-classification
  - requirements-engineering
  - generated_from_trainer
model-index:
  - name: rubert_level1_v2
    results:
      - task:
          type: text-classification
        metrics:
          - type: loss
            value: 0.0727
            name: Validation Loss
          - type: f1
            value: 0.9749
            name: F1 Micro
          - type: f1
            value: 0.9750
            name: F1 Macro
          - type: f1
            value: 0.9750
            name: F1 Weighted
---

# rubert_level1_v2

This model is a fine-tuned version of [DeepPavlov/rubert-base-cased](https://huggingface.co/DeepPavlov/rubert-base-cased) for multilabel classification of software requirements in Russian (Level 1).

It achieves the following results on the evaluation set:
* Loss: 0.0727
* F1 Micro: 0.9749
* F1 Macro: 0.9750
* F1 Weighted: 0.9750

## Model description

Level 1 classifier in a cascaded requirements classification pipeline. Classifies Russian-language text fragments from meeting recordings into three categories:

| Label | Description |
|---|---|
| `IsFunctional` | Functional requirements — what the system must do |
| `IsBusiness` | Business requirements — budgets, KPIs, deadlines, regulations |
| `Other (OT)` | Non-requirements — organizational remarks, transition phrases, context |

`IsNonFunctional` is derived automatically as OR over Level 2 predictions and is not predicted by this model directly.

The model is part of a cascaded pipeline:
`Audio → GigaAM-v3 (ASR) → rubert_level1_v2 (L1) → rubert_level2_v2 (L2) → Report`

Per-class classification thresholds are stored in `thresholds.json` in this repository.

## Intended uses & limitations

Intended for classification of Russian-language software requirements extracted from meeting audio recordings. Not suitable for general-purpose text classification or non-Russian languages.

## Training and evaluation data

Custom Russian-language requirements dataset compiled from:
- PROMISE dataset (translated to Russian)
- PURE dataset (parsed from XML, translated to Russian)
- Synthetically generated examples (Grok, Claude Sonnet) across 14 domain areas

Total: ~9800 labeled examples. Train/test split: 80/20, stratified, seed=42.

## Training procedure

### Training hyperparameters

* learning_rate: 2e-05
* train_batch_size: 16
* eval_batch_size: 16
* seed: 42
* optimizer: AdamW with betas=(0.9, 0.999), epsilon=1e-08
* lr_scheduler_type: linear
* lr_scheduler_warmup_ratio: 0.06
* num_epochs: 15 (early stopping patience=3)
* max_length: 96

### Training results

| Training Loss | Epoch | Validation Loss | F1 Micro | F1 Macro | F1 Weighted |
|---|---|---|---|---|---|
| 0.1007 | 1 | 0.1046 | 0.9030 | 0.8907 | 0.8906 |
| 0.0462 | 2 | 0.0471 | 0.9669 | 0.9671 | 0.9671 |
| 0.0215 | 3 | 0.0467 | 0.9698 | 0.9697 | 0.9697 |
| 0.0170 | 4 | 0.0556 | 0.9689 | 0.9689 | 0.9689 |
| 0.0072 | 5 | 0.0784 | 0.9607 | 0.9604 | 0.9605 |
| 0.0055 | 6 | 0.0608 | 0.9724 | 0.9727 | 0.9724 |

Early stopping triggered after epoch 6.

### Per-class results (test set)

| Class | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| IsFunctional | 0.934 | 0.948 | 0.941 | 420 |
| IsBusiness | 0.993 | 0.978 | 0.985 | 416 |
| Other (OT) | 1.000 | 1.000 | 1.000 | 421 |
| **micro avg** | **0.975** | **0.975** | **0.975** | 1257 |

### Framework versions

* Transformers 4.57.1
* PyTorch 2.8.0+cu128
* Datasets 4.0.0
* Tokenizers 0.22.2