MB55 commited on
Commit
ba30394
·
verified ·
1 Parent(s): 0630497

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -40
README.md CHANGED
@@ -1,57 +1,26 @@
1
  ---
2
  base_model: LSX-UniWue/LLaMmlein_7B_chat
3
- library_name: peft
4
  license: mit
5
- language:
6
- - de
7
  ---
8
 
9
- # Model Card for Model ID
10
 
11
- <!-- Provide a quick summary of what the model is/does. -->
12
 
13
-
14
-
15
- ## Model Details
16
-
17
-
18
-
19
- ### Model Description
20
-
21
- <!-- Provide a longer summary of what this model is. -->
22
-
23
- # QLoRA Fine-Tuned Model: Advance vs. Non-Advance Classification
24
-
25
- This model is a fine-tuned version of [LSX-UniWue/LLaMmlein_7B_chat](https://huggingface.co/LSX-UniWue/LLaMmlein_7B_chat) using QLoRA and a classification head.
26
-
27
- It was trained to classify short utterances (e.g., student or teacher dialogue) into two categories:
28
- - **advance**: utterances that move the conversation forward (e.g., answering, explaining)
29
- - **non_advance**: utterances that do not move the conversation forward (e.g., hesitations, misunderstandings)
30
-
31
- ---
32
-
33
- ## 🔧 Fine-tuning Details
34
-
35
- - **Base model**: LSX-UniWue/LLaMmlein_7B_chat
36
- - **Method**: QLoRA (4-bit quantization, LoRA adapters)
37
- - **Task**: Sequence Classification (2 classes)
38
- - **Training dataset size**: 1200 examples
39
- - **Validation dataset size**: 120 examples
40
- - **Training epochs**: 2
41
- - **Learning rate**: 2e-4
42
- - **Weight decay**: 0.01
43
 
44
  ---
45
 
46
- ## 📊 Performance
47
 
48
- - Final training loss: ~0.10
49
- - Final validation loss: ~0.35
50
- - No overfitting observed during training
51
 
52
  ---
53
 
54
- ## 📚 How to Use
55
 
56
  ```python
57
  from transformers import AutoModelForSequenceClassification, AutoTokenizer
 
1
  ---
2
  base_model: LSX-UniWue/LLaMmlein_7B_chat
 
3
  license: mit
 
 
4
  ---
5
 
6
+ # LLäMmlein QLoRA Grounding Act Classification
7
 
8
+ This model is a fine-tuned version of [LSX-UniWue/LLaMmlein_7B_chat](https://huggingface.co/LSX-UniWue/LLaMmlein_7B_chat), optimized using QLoRA for efficient binary classification of German dialogue utterances into:
9
 
10
+ - **advance**: Contribution that moves the dialogue forward (e.g. confirmations, follow-ups, elaborations)
11
+ - **non_advance**: Other utterances (e.g. vague responses, misunderstandings, irrelevant comments)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
 
13
  ---
14
 
15
+ ## Use Cases
16
 
17
+ - Dialogue system analysis
18
+ - Teacher-student interaction classification
19
+ - Grounding in institutional advising or classroom discourse
20
 
21
  ---
22
 
23
+ ## How to Use
24
 
25
  ```python
26
  from transformers import AutoModelForSequenceClassification, AutoTokenizer