File size: 2,015 Bytes
61ba51e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 | # SGLang Monitoring Setup
This directory contains a ready-to-use monitoring setup for SGLang using Prometheus and Grafana.
## Prerequisites
- Docker and Docker Compose installed
- SGLang server running with metrics enabled
## Usage
1. Start your SGLang server with metrics enabled:
```bash
python -m sglang.launch_server --model-path meta-llama/Meta-Llama-3.1-8B-Instruct --port 30000 --enable-metrics
```
By default, the metrics server will run on `127.0.0.1:30000`.
2. Start the monitoring stack:
```bash
cd examples/monitoring
docker compose up
```
3. Access the monitoring interfaces:
- Grafana: [http://localhost:3000](http://localhost:3000)
- Prometheus: [http://localhost:9090](http://localhost:9090)
Default Grafana login credentials:
- Username: `admin`
- Password: `admin`
You'll be prompted to change the password on first login.
4. The SGLang dashboard will be automatically available in the "SGLang Monitoring" folder.
## Troubleshooting
### Port Conflicts
If you see errors like "port is already allocated":
1. Check if you already have Prometheus or Grafana running:
```bash
docker ps | grep -E 'prometheus|grafana'
```
2. Stop any conflicting containers:
```bash
docker stop <container_id>
```
3. Ensure no other services are using ports 9090 and 3000:
```bash
lsof -i :9090
lsof -i :3000
```
### Connection Issues
If Grafana cannot connect to Prometheus:
1. Check that both services are running
2. Verify the datasource configuration in Grafana
3. Check that your SGLang server is properly exposing metrics
## Configuration
- Prometheus configuration: `prometheus.yaml`
- Docker Compose configuration: `docker-compose.yaml`
- Grafana datasource: `grafana/datasources/datasource.yaml`
- Grafana dashboard configuration: `grafana/dashboards/config/dashboard.yaml`
- SGLang dashboard JSON: `grafana/dashboards/json/sglang-dashboard.json`
## Customization
You can customize the monitoring setup by modifying the configuration files as needed.
|