File size: 2,806 Bytes
e54fd17 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 | # CLI Usage
PromptLab provides a command-line interface (CLI) for managing prompts, versions, tests, and evaluations.
## Basic Commands
### Prompt Management
```bash
# Create a prompt
promptlab prompt create "Weather Forecast" --content "Provide a weather forecast for {location} on {date}" --tags "weather,forecast"
# List all prompts
promptlab prompt list
# Get prompt details
promptlab prompt get <prompt_id>
# Update a prompt
promptlab prompt update <prompt_id> --content "New content" --tags "new,tags"
# Delete a prompt
promptlab prompt delete <prompt_id>
```
### Version Control
```bash
# Commit a version
promptlab version commit <prompt_id> --message "Version description"
# List versions
promptlab version list <prompt_id>
# Check out (revert to) a specific version
promptlab version checkout <prompt_id> <version_number>
# Compare versions
promptlab version diff <prompt_id> <version1> <version2>
```
### Testing
```bash
# Create a test case
promptlab test create <prompt_id> --input '{"location": "New York", "date": "tomorrow"}' --expected "Expected output"
# List test cases
promptlab test list <prompt_id>
# Run a specific test case
promptlab test run <test_case_id> --llm openai
# Run all test cases for a prompt
promptlab test run-all <prompt_id> --llm openai
# Run an A/B test between two prompts
promptlab test ab <prompt_id_a> <prompt_id_b> --inputs '[{"var": "value1"}, {"var": "value2"}]' --llm openai
```
### Evaluation
```bash
# Evaluate a prompt
promptlab eval run <prompt_id> --inputs '[{"var": "value1"}, {"var": "value2"}]' --llm openai
# List available metrics
promptlab eval metrics
# Register a custom metric
promptlab eval register-metric <metric_file.py>
```
## Environment Configuration
The CLI supports environment variables for configuration:
- `PROMPTLAB_STORAGE`: Path to store prompts and related data
- `PROMPTLAB_OPENAI_API_KEY`: OpenAI API key for built-in LLM support
- `PROMPTLAB_DEFAULT_LLM`: Default LLM to use for testing and evaluation
You can also create a config file at `~/.promptlab/config.json`:
```json
{
"storage_path": "/path/to/storage",
"default_llm": "openai",
"api_keys": {
"openai": "your-openai-key"
}
}
```
## Advanced Usage
### Multiple Storage Locations
```bash
# Specify a storage location for a command
promptlab --storage /path/to/storage prompt list
# Export a prompt to another storage
promptlab prompt export <prompt_id> --output /path/to/output.json
# Import a prompt from a file
promptlab prompt import /path/to/prompt.json
```
### Automation and Scripting
```bash
# Get output in JSON format
promptlab --json prompt list
# Use in shell scripts
PROMPT_ID=$(promptlab --json prompt create "Script Prompt" --content "Content" | jq -r '.id')
echo "Created prompt with ID: $PROMPT_ID"
```
|