README.md

---
library_name: transformers
datasets:
- dim014/ui-form-user-manual-generation-dataset-rus
language:
- ru
base_model:
- deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
---

# Model Card for Model ID

## Model Details

**Short Description**

The model is a fine-tuned version of the DeepSeek-R1-Distill-Qwen-1.5B model, specifically adapted for generating Russian user manuals and instructions for UI forms.
It is instruction-tuned and uses LoRA (Low-Rank Adaptation) for efficient parameter-efficient fine-tuning.
The model takes UI form descriptions as input and outputs step-by-step instructions in Russian, making it a valuable tool for creating clear and concise user documentation for software interfaces.

### Model Description

- **Shared by:** [dim014](https://huggingface.co/dim014)
- **Model type:** Causal language model (instruction-tuned, LoRA adapters)
- **Language(s) (NLP):** Russian (primary), limited English support
- **Finetuned from model:** [deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B)

### Model Sources

- **Repository:** [deepseek-r1-finetuned](https://huggingface.co/dim014/deepseek-r1-finetuned)
- **Demo:** TBC

## Uses

### Direct Use

- Generate Russian user manuals and step-by-step instructions for UI forms (fields, buttons, dialogs).
- Assist technical writers in drafting consistent, clear end-user documentation in Russian.
- Produce help texts, onboarding content, and tooltips for web/desktop interfaces.

### Downstream Use

- Integrate into documentation generation pipelines or help centers.
- Further fine-tune on domain-specific UIs (e.g., banking, healthcare, enterprise).
- Embed into chatbots that explain UI behavior and form usage.

### Out-of-Scope Use

- Safety-critical decision-making (medical, legal, financial).
- General-purpose creative writing or code generation.
- High-quality multilingual generation beyond Russian without additional fine-tuning.
- Any usage that conflicts with the base model or dataset licenses.

## Bias, Risks, and Limitations

- May reflect biases from the base model and training data.
- Can produce verbose or repetitive instructions for simple forms.
- May hallucinate steps or UI elements not present in the input.
- Quality may degrade on highly specialized domains or non-Russian inputs.

### Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
- Always perform human review before publishing documentation.
- Prefer Russian inputs; apply post-processing for clarity and conciseness.
- For production, implement guardrails, validation, and prompt templates tailored to your UI.

## Training Details

### Training Data

- **Dataset:** dim014/ui-form-user-manual-generation-dataset-rus
- **Content:** Alpaca-style pairs mapping UI form descriptions to Russian user manuals.
- **Size observed in run logs:** approximately 1,286 training samples.
- **Tokenization:** padding/truncation to a maximum of 1,024 tokens.

### Training Procedure

#### Preprocessing

- Alpaca-style prompt template with sections:
  - “Instruction:” + instruction
  - Optional “Input:” + input
  - “Response:” + target output + EOS
- Tokenization with the base model tokenizer; labels set equal to input_ids for causal LM.


#### Training Hyperparameters

- **Training regime:** LoRA (PEFT) fine-tuning on top of [deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B)

- LoRA configuration:
  - r: 16
  - lora_alpha: 32
  - lora_dropout: 0.05
  - target_modules: `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
  - bias: none
- Epochs: 3
- Per-device batch size: 2
- Gradient accumulation steps: 8 (effective batch size 16)
- Learning rate: 2e-4
- Scheduler: cosine with warmup ratio 0.1
- Precision: fp16
- Optimizer: AdamW (torch)
- Max sequence length: 1,024
- Gradient checkpointing: enabled
- Logging: W&B; checkpoints every 50 steps (keep last 3)

#### Speeds, Sizes, Times

- Final train loss: ~0.559
- Train runtime: ~3,256.93 seconds (~54.3 minutes)
- Train steps per second: ~0.075
- Train samples per second: ~1.185
- Total training samples: 1,286
- GPU memory (example from logs): ~3.63 GB allocated, max allocated ~7.80 GB
- Hardware model not captured in logs; trained in a single-GPU CUDA environment (e.g., Google Colab)

## Evaluation

### Testing Data, Factors & Metrics

#### Testing Data

- Qualitative testing on 5 curated prompts representing different UI forms (registration, participation details, package selection, payment, completion).

#### Factors

- Russian-only prompts.
- Varying form complexity (single-step to multi-step flows).
- Domain: general software UIs.

#### Metrics

- Automatic metrics (BLEU/ROUGE) were not computed in the provided run logs.
- Qualitative manual checks for fluency, clarity, completeness, and hallucinations.

### Results

- The model produced coherent, step-by-step Russian instructions across the 5 example prompts.
- Occasional minor terminology drift or verbosity observed; human editing recommended.

## Technical Specifications

### Model Architecture and Objective

- Architecture: decoder-only causal transformer (Qwen-style, distilled in DeepSeek-R1).
- Objective: instruction tuning for Russian UI form user manual generation.

### Compute Infrastructure

- Single-node, single GPU (CUDA), automatic device placement (`device_map="auto"`).

#### Hardware

- GPU name not captured in logs; runtime memory usage indicates a consumer/pro cloud GPU.
- Example memory: ~3.63 GB allocated, up to ~7.80 GB max allocated.

#### Software

- Libraries: `transformers`, `datasets`, `peft`, `torch`, `accelerate`, `bitsandbytes`, `wandb`
- Tokenizer and model weights from the Hugging Face Hub.

## Glossary

- **LoRA (Low-Rank Adaptation):** Parameter-efficient fine-tuning adding low-rank adapters to selected layers.
- **Causal LM:** Autoregressive language model generating the next token from previous context.
- **Instruction tuning:** Fine-tuning to follow natural-language instructions.

## Model Card Authors

[dim014](https://huggingface.co/dim014)

## Model Card Contact

[dim014](https://huggingface.co/dim014)