README.md

metadata

title: Prompt A/B Tester
emoji: 🔬
colorFrom: blue
colorTo: purple
sdk: streamlit
sdk_version: 1.52.1
app_file: app.py
pinned: false
license: mit

Prompt A/B Tester

Web application for testing and comparing the performance of two OpenAI GPT prompt versions with A/B testing methodology.

Author

Marek Staniszewski Heuristica 🌐 www.heuristica.pl 📧 staniszewski@heuristica.pl For questions, suggestions, or support, please contact the author.

Features

• API Key Management: Securely store OpenAI API key in Streamlit secrets
• Model Selection: Choose from available GPT models (gpt-4o, gpt-4, gpt-3.5-turbo)
• Parameter Configuration:
  • Temperature (0.0 - 2.0)
  • Max Tokens (100 - 4000)
  • Number of responses (1-50)
• Two Input Methods:
  • Upload .md or .txt files
  • Paste text directly
• Progress Tracking: Real-time progress bars and logs
• Interactive Rating: Rate responses on a 1-5 scale
• Results Visualization: Compare prompts with statistics
• CSV Export: Download results with metadata

Usage

1. Configuration

1. Go to "⚙️ Configuration" tab
2. Enter your OpenAI API key
3. Select model and adjust parameters
4. Save settings

2. Run Test

1. Go to "🚀 Test" tab
2. Either upload two files OR paste two prompts
3. Click "Run Test"
4. Monitor progress

3. Rate Responses

1. After generation, go to "⭐ Rating" tab
2. Rate each response (1-5)
3. Click "Calculate Results"

4. View Results

1. Review statistics in "📊 Results" tab
2. Export to CSV if needed

Configuration on Hugging Face Spaces

To use this app on Hugging Face Spaces:

1. Go to Settings → Repository secrets
2. Add a new secret:
   • Name: OPENAI_API_KEY
   • Value: Your OpenAI API key

Cost Estimation

Approximate costs per test (2025 pricing):

Model	10 responses (5A + 5B)	20 responses (10A + 10B)
gpt-4o	~$0.05	~$0.10
gpt-4-turbo	~$0.15	~$0.30
gpt-4	~$0.30	~$0.60
gpt-3.5-turbo	~$0.01	~$0.02

Project Information

Version: 2.0 (Streamlit) Created: 2025-10-14 License: MIT Contact: www.Heuristica.pl