Text Classification
Transformers
PyTorch
English
promptforge_quality
promptforge
prompt-engineering
prompt-quality
modernbert
regression
llm
Instructions to use ArjunShukla/PromptForge-Quality with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ArjunShukla/PromptForge-Quality with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="ArjunShukla/PromptForge-Quality")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("ArjunShukla/PromptForge-Quality", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Upload folder using huggingface_hub
Browse files- README.md +37 -0
- config.json +20 -0
- pytorch_model.bin +3 -0
- tokenizer.json +0 -0
- tokenizer_config.json +17 -0
- training_config.yaml +37 -0
README.md
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language: en
|
| 3 |
+
license: mit
|
| 4 |
+
library_name: transformers
|
| 5 |
+
tags:
|
| 6 |
+
- prompt-engineering
|
| 7 |
+
- prompt-quality
|
| 8 |
+
- modernbert
|
| 9 |
+
- regression
|
| 10 |
+
- promptforge
|
| 11 |
+
pipeline_tag: text-classification
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
# PromptForge-Quality
|
| 15 |
+
|
| 16 |
+
Scores LLM prompts across multiple quality dimensions:
|
| 17 |
+
|
| 18 |
+
- clarity
|
| 19 |
+
- specificity
|
| 20 |
+
- context
|
| 21 |
+
- goal_definition
|
| 22 |
+
- constraints
|
| 23 |
+
- completeness
|
| 24 |
+
- actionability
|
| 25 |
+
|
| 26 |
+
## Usage
|
| 27 |
+
|
| 28 |
+
```python
|
| 29 |
+
from promptforge import PromptForge
|
| 30 |
+
|
| 31 |
+
pf = PromptForge(quality_model_path="YOUR_HF_REPO_OR_LOCAL_DIR")
|
| 32 |
+
print(pf.analyze("Build me a website"))
|
| 33 |
+
```
|
| 34 |
+
|
| 35 |
+
## Training
|
| 36 |
+
|
| 37 |
+
Phase 1 of [PromptForge](https://github.com/promptforge/promptforge) — ModernBERT encoder with dual regression heads.
|
config.json
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_type": "promptforge_quality",
|
| 3 |
+
"base_model_name": "answerdotai/ModernBERT-base",
|
| 4 |
+
"num_labels": 7,
|
| 5 |
+
"dropout": 0.1,
|
| 6 |
+
"dimension_loss_weight": 0.8,
|
| 7 |
+
"quality_loss_weight": 0.2,
|
| 8 |
+
"label_names": [
|
| 9 |
+
"clarity",
|
| 10 |
+
"specificity",
|
| 11 |
+
"context",
|
| 12 |
+
"goal_definition",
|
| 13 |
+
"constraints",
|
| 14 |
+
"completeness",
|
| 15 |
+
"actionability"
|
| 16 |
+
],
|
| 17 |
+
"architectures": [
|
| 18 |
+
"PromptForgeQualityModel"
|
| 19 |
+
]
|
| 20 |
+
}
|
pytorch_model.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ab3388a771b84cdf75cadd7a3c80a0dc9a679efc7c4cb88cce6f5832c145d816
|
| 3 |
+
size 598492635
|
tokenizer.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
tokenizer_config.json
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"backend": "tokenizers",
|
| 3 |
+
"clean_up_tokenization_spaces": true,
|
| 4 |
+
"cls_token": "[CLS]",
|
| 5 |
+
"is_local": false,
|
| 6 |
+
"local_files_only": false,
|
| 7 |
+
"mask_token": "[MASK]",
|
| 8 |
+
"model_input_names": [
|
| 9 |
+
"input_ids",
|
| 10 |
+
"attention_mask"
|
| 11 |
+
],
|
| 12 |
+
"model_max_length": 8192,
|
| 13 |
+
"pad_token": "[PAD]",
|
| 14 |
+
"sep_token": "[SEP]",
|
| 15 |
+
"tokenizer_class": "TokenizersBackend",
|
| 16 |
+
"unk_token": "[UNK]"
|
| 17 |
+
}
|
training_config.yaml
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
model_name: answerdotai/ModernBERT-base
|
| 2 |
+
max_length: 512
|
| 3 |
+
num_labels: 7
|
| 4 |
+
label_names:
|
| 5 |
+
- clarity
|
| 6 |
+
- specificity
|
| 7 |
+
- context
|
| 8 |
+
- goal_definition
|
| 9 |
+
- constraints
|
| 10 |
+
- completeness
|
| 11 |
+
- actionability
|
| 12 |
+
num_examples: 25000
|
| 13 |
+
seed: 42
|
| 14 |
+
train_ratio: 0.8
|
| 15 |
+
val_ratio: 0.1
|
| 16 |
+
test_ratio: 0.1
|
| 17 |
+
num_train_epochs: 3
|
| 18 |
+
per_device_train_batch_size: 8
|
| 19 |
+
per_device_eval_batch_size: 16
|
| 20 |
+
gradient_accumulation_steps: 2
|
| 21 |
+
learning_rate: 2.0e-05
|
| 22 |
+
weight_decay: 0.01
|
| 23 |
+
warmup_steps: 500
|
| 24 |
+
logging_steps: 100
|
| 25 |
+
eval_steps: 500
|
| 26 |
+
save_steps: 500
|
| 27 |
+
save_total_limit: 2
|
| 28 |
+
early_stopping_patience: 2
|
| 29 |
+
dropout: 0.1
|
| 30 |
+
dimension_loss_weight: 0.8
|
| 31 |
+
quality_loss_weight: 0.2
|
| 32 |
+
prefer_gpu: true
|
| 33 |
+
use_fp16: true
|
| 34 |
+
use_bf16: false
|
| 35 |
+
output_dir: outputs/promptforge-quality
|
| 36 |
+
dataset_path: data/promptforge_dataset.csv
|
| 37 |
+
final_model_dir: outputs/promptforge-quality-model
|