Training in progress, epoch 1

README.md

@@ -1,73 +1,67 @@
 ---
 library_name: peft
 license: apache-2.0
+base_model: answerdotai/ModernBERT-base
 tags:
-- json-extraction
-- modernbert
+- base_model:adapter:answerdotai/ModernBERT-base
 - lora
-- diffuberta
-language: en
-metrics:
-- name: train_loss
-  value: 4.7773
-- name: eval_loss
-  value: 4.318033695220947
-datasets:
-- generated-json-pairs
+- transformers
+model-index:
+- name: DiffuBERTa
+  results: []
 ---
 
----
-datasets:
-- generated-json-pairs
-language: en
-library_name: peft
-license: apache-2.0
-metrics:
-- name: train_loss
-  value: 4.7773
-- name: eval_loss
-  value: 4.318033695220947
-tags:
-- json-extraction
-- modernbert
-- lora
-- diffuberta
----
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# DiffuBERTa
+
+This model is a fine-tuned version of [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 4.3180
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
 
-# DiffuBERTa: JSON Extraction Adapter
+More information needed
 
-This model is a Fine-tuned version of **answerdotai/ModernBERT-base** using LoRA. It is designed to extract structured JSON data from unstructured text using a parallel decoding approach.
+## Training procedure
 
-## Model Performance
-- **Final Training Loss**: 4.7773
-- **Final Evaluation Loss**: 4.318033695220947
-- **Training Epochs**: 5
-- **Date Trained**: 2025-11-28
+### Training hyperparameters
 
-## 🚀 Live Demo Output
-*(Generated automatically after training)*
+The following hyperparameters were used during training:
+- learning_rate: 3e-05
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 42
+- gradient_accumulation_steps: 2
+- total_train_batch_size: 16
+- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 500
+- num_epochs: 5
 
-**Input Text:**
-> "We are excited to welcome Dr. Sarah to our Paris office as Senior Data Scientist."
+### Training results
 
-**Template:**
-> `{'name': '[1]', 'job': '[2]', 'city': '[1]'}`
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 15.3944       | 1.0   | 63   | 14.6510         |
+| 13.7016       | 2.0   | 126  | 10.3954         |
+| 10.2371       | 3.0   | 189  | 6.2723          |
+| 5.5815        | 4.0   | 252  | 5.0812          |
+| 4.7773        | 5.0   | 315  | 4.3180          |
 
-**Model Output:**
-```json
-{
-  "name": "Sarah",
-  "job": "Data scientist",
-  "city": "Paris"
-}
-```
 
-## Usage
-```python
-from transformers import AutoModelForMaskedLM, AutoTokenizer
-from peft import PeftModel
+### Framework versions
 
-base_model = AutoModelForMaskedLM.from_pretrained("answerdotai/ModernBERT-base")
-model = PeftModel.from_pretrained(base_model, "philipp-zettl/DiffuBERTa")
-# ... use extract_parallel helper ...
-```
+- PEFT 0.18.0
+- Transformers 4.57.3
+- Pytorch 2.9.1+cu128
+- Tokenizers 0.22.1

adapter_config.json

@@ -29,10 +29,10 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "W1",
-    "Wqkv",
     "Wo",
-    "W2"
+    "W2",
+    "W1",
+    "Wqkv"
   ],
   "target_parameters": null,
   "task_type": "FEATURE_EXTRACTION",

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:27b153a25bf3d5e38441ccca5c52e8455ab6d353be028e0ebe8b95dab0073673
+oid sha256:0e0e49115972bdf0ea99a58aceac7ead32207c21ef5f8367bd433253d2eb1553
 size 9207688

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8a1676b1da10f7730e9075290eef1895a2e69f3e8726e3b9cda4f4f068be0c40
+oid sha256:60dd10c2a863cb1987768c6c8035956b4d7d4bdee636f5e9eeba2b53a47c0cef
 size 5905