docs/user_guide.md

User Guide

Complete operational guide for the Hopcroft Skill Classification system covering all components: API, GUI, load testing, and monitoring.

---

Table of Contents

1. System Setup
2. API Usage
3. GUI (Streamlit)
4. Load Testing (Locust)
5. Monitoring (Prometheus & Grafana)

---

1. System Setup (Local)

Prerequisites

Requirement	Version	Purpose
Python	3.10+	Runtime environment
Docker	20.10+	Containerization
Docker Compose	2.0+	Multi-service orchestration
Git	2.30+	Version control

Option A: Docker Setup

1. Clone and Configure

git clone https://github.com/se4ai2526-uniba/Hopcroft.git
cd Hopcroft

# Create environment file
cp .env.example .env

2. Edit .env with Your Credentials

MLFLOW_TRACKING_URI=https://dagshub.com/se4ai2526-uniba/Hopcroft.mlflow
MLFLOW_TRACKING_USERNAME=your_dagshub_username
MLFLOW_TRACKING_PASSWORD=your_dagshub_token

Get your DagsHub token at: https://dagshub.com/user/settings/tokens

3. Start All Services

docker compose -f docker/docker-compose.yml up -d --build

4. Verify Services

Service	URL	Purpose
API (Swagger)	http://localhost:8080/docs	Interactive API documentation
GUI (Streamlit)	http://localhost:8501	Web interface
Health Check	http://localhost:8080/health	Service status

Option B: Virtual Environment Setup

1. Create Virtual Environment

python -m venv venv

# Windows
venv\Scripts\activate

# Linux/macOS
source venv/bin/activate

2. Install Dependencies

pip install -r requirements.txt
pip install -e .

3. Configure DVC (for Model Access)

dvc remote modify origin --local auth basic
dvc remote modify origin --local user YOUR_DAGSHUB_USERNAME
dvc remote modify origin --local password YOUR_DAGSHUB_TOKEN
dvc pull

4. Start Services Manually

# Terminal 1: Start API
make api-dev

# Terminal 2: Start Streamlit
streamlit run hopcroft_skill_classification_tool_competition/streamlit_app.py

Docker Compose Commands Reference

Command	Description
docker compose -f docker/docker-compose.yml up -d	Start in background
docker compose -f docker/docker-compose.yml down	Stop all services
docker compose -f docker/docker-compose.yml logs -f	Stream logs
docker compose -f docker/docker-compose.yml ps	Check status
docker compose -f docker/docker-compose.yml restart	Restart services

---

2. API Usage

Base URLs

Environment	URL
Local (Docker)	http://localhost:8080
Local (Dev)	http://localhost:8000
Production (HF Spaces)	https://dacrow13-hopcroft-skill-classification.hf.space/docs

Endpoints Overview

Method	Endpoint	Description
POST	/predict	Predict skills for single issue
POST	/predict/batch	Batch prediction (max 100)
GET	/predictions	List recent predictions
GET	/predictions/{run_id}	Get prediction by ID
GET	/health	Health check
GET	/metrics	Prometheus metrics

Interactive Documentation

Access Swagger UI for interactive testing:

• Swagger: http://localhost:8080/docs
• ReDoc: http://localhost:8080/redoc

Example Requests

Single Prediction

curl -X POST "http://localhost:8080/predict" \
  -H "Content-Type: application/json" \
  -d '{
    "issue_text": "Fix authentication bug in OAuth2 login flow",
    "repo_name": "my-project",
    "pr_number": 42
  }'

Response:

{
  "run_id": "abc123...",
  "predictions": [
    {"skill": "authentication", "confidence": 0.92},
    {"skill": "security", "confidence": 0.78},
    {"skill": "oauth", "confidence": 0.65}
  ],
  "model_version": "1.0.0",
  "timestamp": "2025-01-05T15:00:00Z"
}

Batch Prediction

curl -X POST "http://localhost:8080/predict/batch" \
  -H "Content-Type: application/json" \
  -d '{
    "issues": [
      {"issue_text": "Database connection timeout"},
      {"issue_text": "UI button not responding"}
    ]
  }'

List Predictions

curl "http://localhost:8080/predictions?limit=10&skip=0"

Health Check

curl "http://localhost:8080/health"

Response:

{
  "status": "healthy",
  "model_loaded": true,
  "model_version": "1.0.0"
}

Makefile Shortcuts

make test-api-health      # Test health endpoint
make test-api-predict     # Test prediction
make test-api-list        # List predictions
make test-api-all         # Run all API tests

---

3. GUI (Streamlit)

Access Points

Environment	URL
Local (Docker)	http://localhost:8501
Production	https://dacrow13-hopcroft-skill-classification.hf.space

Features

• Real-time Prediction: Instant skill classification
• Confidence Scores: Probability for each predicted skill
• Multiple Input Modes: Quick input, detailed input, examples
• API Health Indicator: Connection status in sidebar

User Interface

Main Dashboard

The sidebar displays:

• API connection status
• Confidence threshold slider
• Model information

Quick Input Mode

1. Paste GitHub issue text
2. Click "Predict Skills"
3. View results instantly

Detailed Input Mode

Optional metadata fields:

• Repository name
• PR number
• Extended description

Prediction Results

Results display:

• Top-5 predicted skills with confidence bars
• Full predictions table with filtering
• Processing time metrics
• Raw JSON response (expandable)

Example Gallery

Pre-loaded test cases:

• Authentication bugs
• ML feature requests
• Database issues
• UI enhancements

---

4. Load Testing (Locust)

Installation

pip install locust

Configuration

The Locust configuration is in monitoring/locust/locustfile.py:

Task	Weight	Endpoint
Single Prediction	60% (weight: 3)	POST /predict
Batch Prediction	20% (weight: 1)	POST /predict/batch
Monitoring	20% (weight: 1)	GET /health, /predictions

Running Load Tests

Web UI Mode

cd monitoring/locust
locust

Then open: http://localhost:8089

Configure in the Web UI:

• Number of users: Total concurrent users
• Spawn rate: Users per second to add
• Host: Target URL (e.g., http://localhost:8080)

Headless Mode

locust --headless \
  --users 50 \
  --spawn-rate 10 \
  --run-time 5m \
  --host http://localhost:8080 \
  --csv results

Target URLs

Environment	Host URL
Local Docker	http://localhost:8080
Local Dev	http://localhost:8000
HF Spaces	https://dacrow13-hopcroft-skill-classification.hf.space

Interpreting Results

Metric	Description	Target
RPS	Requests per second	Higher = better
Median Response Time	50th percentile latency	< 500ms
95th Percentile	Worst-case latency	< 2s
Failure Rate	Percentage of errors	< 1%

---

5. Monitoring (Prometheus & Grafana)

Access Points

Local Development:

Service	URL
Prometheus	http://localhost:9090
Grafana	http://localhost:3000
Pushgateway	http://localhost:9091

Hugging Face Spaces (Production):

Service	URL
Prometheus	https://dacrow13-hopcroft-skill-classification.hf.space/prometheus/
Grafana	https://dacrow13-hopcroft-skill-classification.hf.space/grafana/

Prometheus Metrics

Access the metrics endpoint: http://localhost:8080/metrics

Available Metrics

Metric	Type	Description
hopcroft_requests_total	Counter	Total requests by method/endpoint
hopcroft_request_duration_seconds	Histogram	Request latency distribution
hopcroft_in_progress_requests	Gauge	Currently processing requests
hopcroft_prediction_processing_seconds	Summary	Model inference time

Useful PromQL Queries

Request Rate (per second)

rate(hopcroft_requests_total[1m])

Average Latency

rate(hopcroft_request_duration_seconds_sum[5m]) / rate(hopcroft_request_duration_seconds_count[5m])

In-Progress Requests

hopcroft_in_progress_requests

Model Prediction Time (P90)

hopcroft_prediction_processing_seconds{quantile="0.9"}

Grafana Dashboards

The pre-configured dashboard includes:

Panel	Description
API Request Rate	Real-time requests per endpoint
API Latency	Response time distribution
Drift Detection Status	Binary indicator (0=No Drift, 1=Drift)
Drift P-Value	Statistical significance
Drift Distance	KS test distance metric

Data Drift Detection

Prepare Baseline (One-time)

cd monitoring/drift/scripts
python prepare_baseline.py

Run Drift Check

python run_drift_check.py

Verify Results

# Check Pushgateway
curl http://localhost:9091/metrics | grep drift

# PromQL queries
drift_detected
drift_p_value
drift_distance

Alerting Rules

Pre-configured alerts in monitoring/prometheus/alert_rules.yml:

Alert	Condition	Severity
ServiceDown	Target down for 5m	Critical
HighErrorRate	5xx > 10% for 5m	Warning
SlowRequests	P95 > 2s	Warning

Starting Monitoring Stack

# Start all monitoring services
docker compose up -d

# Verify containers
docker compose ps

# Check Prometheus targets
curl http://localhost:9090/targets

---

Troubleshooting

Common Issues

API Returns 500 Error

1. Check .env credentials are correct
2. Restart services: docker compose down && docker compose up -d
3. Verify model files: docker exec hopcroft-api ls -la /app/models/

GUI Shows "API Unavailable"

1. Wait 30-60 seconds for API initialization
2. Check API health: curl http://localhost:8080/health
3. View logs: docker compose logs hopcroft-api

Port Already in Use

# Check port usage
netstat -ano | findstr :8080

# Stop conflicting containers
docker compose down

DVC Pull Fails

# Clean cache and retry
rm -rf .dvc/cache
dvc pull