nlproxy/docs/cli.md

NLProxy CLI Module Reference

This document covers the command-line interface implemented under cli/.

Purpose

The CLI package provides operational tooling for NLProxy:

• Starting the server
• Downloading and validating local models
• Running compression tasks from the command line
• Running project tests

Files

cli/runserver.py

Responsibilities

• Parses CLI arguments for server host, port, worker count, provider selection, and logging.
• Loads .env values from the current working directory.
• Configures provider API keys and exports them to environment variables.
• Starts uvicorn with production-ready settings.

Notable CLI Behavior

• --reload forces workers=1 to avoid conflicts with auto-reload.
• Default provider is resolved from CLI flag, environment variables, or built-in fallback.
• Logs an error and exits if provider credentials are missing.
• Uses uvicorn.run(..., http="httptools", ws="websockets") for performance.

Configuration Options

• --host, --port
• --workers
• --llm-client
• --model
• --api-key-client
• --reload
• --log-level
• --access-log
• --list-models

Environment Variables

• NLPROXY_HOST
• NLPROXY_PORT
• NLPROXY_LOG_LEVEL
• NLPROXY_DEFAULT_LLM_PROVIDER
• NLPROXY_DEFAULT_LLM_MODEL
• NLPROXY_MODELS_URL

cli/download_models.py

Responsibilities

• Downloads the repository model archive from a configurable URL.
• Extracts model files safely and validates required directory structure.
• Supports checksum verification for nlproxy_models.zip.
• Reports progress via tqdm when available.

Primary Functions

• ensure_models_dir(models_dir: Path) -> None
• download_with_retries(url: str, dest_path: Path) -> bool
• extract_and_validate(zip_path: Path, models_dir: Path, expected_dirs: List[str]) -> bool
• cleanup_temp(zip_path: Path, keep_zip: bool) -> None

Validation Rules

• Rejects archives containing absolute paths or .. entries.
• Ensures expected directories exist after extraction:
  • all-MiniLM-L6-v2
  • nli-distilroberta-base
  • distilgpt2
• Fails if any extracted model directory lacks valid model artifact files.

cli/compress.py

Responsibilities

• Provides a terminal wrapper for one-off compression operations.
• Accepts prompt text, compression mode, aggressiveness, and output formatting.
• Integrates with the service layer for the same pipeline used by the server.

General Behavior

• Supports --quiet to reduce console output.
• Outputs JSON by default.
• Reuses the same compressor logic as the FastAPI route.

cli/help.py

Responsibilities

• Prints CLI usage help text.
• Acts as a lightweight documentation helper for users who invoke the CLI with --help.

cli/tests.py

Responsibilities

• Wraps pytest invocation for repository tests.
• Builds pytest command line based on CLI flags.
• Supports coverage and custom test selection.

Usage Notes

• The CLI is intentionally thin and delegates heavy work to server, service, and core modules.
• The downloader is the only CLI entrypoint that performs local filesystem writes and network I/O.
• runserver is the recommended path for production deployments.