docs/development.md

Development Guide

Complete guide for setting up and developing the Smart LINE Bot + Dashboard + Scraper Pipeline.

Prerequisites

• Python 3.11+
• PostgreSQL 15+
• Redis 7+
• Docker & Docker Compose (optional)
• LINE Developer Account
• OpenAI Account

Local Development Setup

1. Clone Repository

git clone https://github.com/Smiledangers/smart-line-automation-suite.git
cd smart-line-automation-suite

2. Create Virtual Environment

# Windows
python -m venv venv
venv\Scripts\activate

# Linux/Mac
python -m venv venv
source venv/bin/activate

3. Install Dependencies

# Install all dependencies
pip install -r requirements/base.txt
pip install -r requirements/dev.txt

# Or use make
make install
make install-dev

4. Environment Setup

# Copy environment template
cp .env.example .env

# Edit .env with your settings
# Required:
# - LINE_CHANNEL_ACCESS_TOKEN
# - LINE_CHANNEL_SECRET
# - OPENAI_API_KEY
# - SECRET_KEY
# - DATABASE_URL
# - REDIS_URL

5. Database Setup

# Initialize database
python scripts/init_db.py

# Run migrations
alembic upgrade head

# Create superuser
python scripts/create_superuser.py

# Seed test data (optional)
python scripts/seed_data.py --all

6. Run Development Server

# Start Redis (if not using Docker)
redis-server

# Start PostgreSQL (if not using Docker)
# Or use Docker:
docker run -d -p 5432:5432 -e POSTGRES_PASSWORD=postgres postgres:15

# Start FastAPI server
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

# Start Celery worker (in separate terminal)
celery -A app.tasks.celery_app worker --loglevel=info

# Start Celery beat (in separate terminal)
celery -A app.tasks.celery_app beat --loglevel=info

7. Access Application

• API Docs: http://localhost:8000/docs
• ReDoc: http://localhost:8000/redoc
• Health: http://localhost:8000/health

Docker Development

Quick Start

# Start all services
docker compose up -d --build

# View logs
docker compose logs -f

# Stop services
docker compose down

Development Commands

# Rebuild containers
docker compose build --no-cache

# Run migrations
docker compose exec web alembic upgrade head

# Create superuser
docker compose exec web python scripts/create_superuser.py

# Access shell
docker compose exec web bash

# Run tests
docker compose exec web pytest

Project Structure

app/
├── api/
│   └── v1/
│       └── endpoints/     # API route handlers
├── core/                   # Core configuration
├── models/                 # SQLAlchemy models
├── schemas/                # Pydantic schemas
├── services/               # Business logic
├── tasks/                  # Celery tasks
└── utils/                  # Utilities

tests/
├── unit/                   # Unit tests
├── integration/            # Integration tests
└── conftest.py             # Pytest fixtures

deploy/
├── k8s/                    # Kubernetes manifests
├── helm/                   # Helm charts
├── terraform/              # Terraform configs
└── cloud/                  # Cloud deployment configs

Code Style

Formatting

# Format code with Black
black app/ tests/

# Sort imports with isort
isort app/ tests/

Type Checking

# Run mypy
mypy app/

Linting

# Run flake8
flake8 app/ tests/

Testing

Run All Tests

pytest

Run Specific Tests

# Unit tests
pytest tests/unit/

# Integration tests
pytest tests/integration/

# With coverage
pytest --cov=app --cov-report=term-missing

Test Fixtures

See tests/conftest.py for available fixtures:

• client - FastAPI test client
• db_session - Database session
• test_user - Test user
• auth_headers - Auth headers
• sample_line_message - LINE webhook sample
• sample_scraping_job - Scraping job sample

Common Issues

Database Connection Error

# Check PostgreSQL is running
pg_isready

# Check connection string in .env

Redis Connection Error

# Check Redis is running
redis-cli ping

# Check REDIS_URL in .env

LINE Webhook Not Working

# Verify credentials
python -c "from app.core.config import settings; print(settings.LINE_CHANNEL_ACCESS_TOKEN)"

# Use ngrok for local development
ngrok http 8000
# Update LINE webhook URL to ngrok URL

Import Errors

# Reinstall dependencies
pip install -r requirements/base.txt

# Check PYTHONPATH
echo $PYTHONPATH

Development Tips

1. Use Docker for consistent environment
2. Use make commands for common tasks (see Makefile)
3. Write tests before implementing features
4. Use type hints for better code documentation
5. Keep secrets in .env, never commit them
6. Use pre-commit hooks for code quality

Contributing

1. Create feature branch
2. Write tests first
3. Implement feature
4. Run code formatters
5. Run full test suite
6. Push and create PR

Resources

• FastAPI Docs
• SQLAlchemy Docs
• Celery Docs
• LINE Messaging API
• OpenAI API