| # CLI Usage | |
| PromptLab provides a command-line interface (CLI) for managing prompts, versions, tests, and evaluations. | |
| ## Basic Commands | |
| ### Prompt Management | |
| ```bash | |
| # Create a prompt | |
| promptlab prompt create "Weather Forecast" --content "Provide a weather forecast for {location} on {date}" --tags "weather,forecast" | |
| # List all prompts | |
| promptlab prompt list | |
| # Get prompt details | |
| promptlab prompt get <prompt_id> | |
| # Update a prompt | |
| promptlab prompt update <prompt_id> --content "New content" --tags "new,tags" | |
| # Delete a prompt | |
| promptlab prompt delete <prompt_id> | |
| ``` | |
| ### Version Control | |
| ```bash | |
| # Commit a version | |
| promptlab version commit <prompt_id> --message "Version description" | |
| # List versions | |
| promptlab version list <prompt_id> | |
| # Check out (revert to) a specific version | |
| promptlab version checkout <prompt_id> <version_number> | |
| # Compare versions | |
| promptlab version diff <prompt_id> <version1> <version2> | |
| ``` | |
| ### Testing | |
| ```bash | |
| # Create a test case | |
| promptlab test create <prompt_id> --input '{"location": "New York", "date": "tomorrow"}' --expected "Expected output" | |
| # List test cases | |
| promptlab test list <prompt_id> | |
| # Run a specific test case | |
| promptlab test run <test_case_id> --llm openai | |
| # Run all test cases for a prompt | |
| promptlab test run-all <prompt_id> --llm openai | |
| # Run an A/B test between two prompts | |
| promptlab test ab <prompt_id_a> <prompt_id_b> --inputs '[{"var": "value1"}, {"var": "value2"}]' --llm openai | |
| ``` | |
| ### Evaluation | |
| ```bash | |
| # Evaluate a prompt | |
| promptlab eval run <prompt_id> --inputs '[{"var": "value1"}, {"var": "value2"}]' --llm openai | |
| # List available metrics | |
| promptlab eval metrics | |
| # Register a custom metric | |
| promptlab eval register-metric <metric_file.py> | |
| ``` | |
| ## Environment Configuration | |
| The CLI supports environment variables for configuration: | |
| - `PROMPTLAB_STORAGE`: Path to store prompts and related data | |
| - `PROMPTLAB_OPENAI_API_KEY`: OpenAI API key for built-in LLM support | |
| - `PROMPTLAB_DEFAULT_LLM`: Default LLM to use for testing and evaluation | |
| You can also create a config file at `~/.promptlab/config.json`: | |
| ```json | |
| { | |
| "storage_path": "/path/to/storage", | |
| "default_llm": "openai", | |
| "api_keys": { | |
| "openai": "your-openai-key" | |
| } | |
| } | |
| ``` | |
| ## Advanced Usage | |
| ### Multiple Storage Locations | |
| ```bash | |
| # Specify a storage location for a command | |
| promptlab --storage /path/to/storage prompt list | |
| # Export a prompt to another storage | |
| promptlab prompt export <prompt_id> --output /path/to/output.json | |
| # Import a prompt from a file | |
| promptlab prompt import /path/to/prompt.json | |
| ``` | |
| ### Automation and Scripting | |
| ```bash | |
| # Get output in JSON format | |
| promptlab --json prompt list | |
| # Use in shell scripts | |
| PROMPT_ID=$(promptlab --json prompt create "Script Prompt" --content "Content" | jq -r '.id') | |
| echo "Created prompt with ID: $PROMPT_ID" | |
| ``` | |