ONLINE_OFFLINE_MILESTONES.md

## MozDef Online → Offline Setup Milestones (Frontend / Backend / Infra)

This file gives you a **checklist-style plan** for:
- Bringing MozDef up **online** (with internet)
- Packing everything
- Running it **offline**

Each milestone is split into **infrastructure**, **backend**, and **frontend** where it makes sense, so you can see exactly what layer you’re validating.

---

## Milestone 1 – Hosts & Tools Ready

### 1.1 Online host (build + test)
- **Do**
  - Install Docker + Docker Compose.
  - Ensure ~50 GB free disk.
- **Check**
  ```bash
  docker --version
  docker-compose --version
  docker run --rm hello-world
  df -h              # check free space
  uname -m           # should be x86_64
  ```

### 1.2 Offline host (final runtime)
- **Do**
  - Install Docker + Docker Compose from local media / mirror.
- **Check**
  ```bash
  docker --version
  docker-compose --version
  uname -m           # x86_64
  ```

---

## Milestone 2 – Online: Get Source & Build Images

This milestone builds **all layers** inside Docker (isolated from host OS).

### 2.1 Get MozDef source
- **Do**
  ```bash
  cd /root
  git clone https://github.com/mozilla/MozDef.git   # or use your existing /root/MozDef
  cd MozDef
  ```
- **Check**
  ```bash
  ls
  # expect: alerts bot config docker docs loginput meteor mq rest ... Makefile
  ```

### 2.2 Build all images (infra + backend + frontend)
- **Do**
  ```bash
  cd /root/MozDef
  make build
  ```
- **Check**
  ```bash
  docker images | grep mozdef
  # expect at least:
  #  mozdef/mozdef_base
  #  mozdef/mozdef_elasticsearch
  #  mozdef/mozdef_rabbitmq
  #  mozdef/mozdef_mongodb
  #  mozdef/mozdef_kibana
  #  mozdef/mozdef_nginx
  #  mozdef/mozdef_bootstrap
  #  mozdef/mozdef_loginput
  #  mozdef/mozdef_mq_worker
  #  mozdef/mozdef_rest
  #  mozdef/mozdef_meteor
  #  mozdef/mozdef_alerts
  #  mozdef/mozdef_alertactions
  #  mozdef/mozdef_cron
  #  mozdef/mozdef_syslog
  ```

### 2.3 Confirm runtime isolation (critical OS/Node concern)
- **Do**
  ```bash
  # host versions (for info only)
  node --version || echo "no host node"
  python3 --version || echo "no host python"

  # container runtimes (REAL runtime MozDef will use)
  docker run --rm mozdef/mozdef_meteor node --version      # expect v8.11.4
  docker run --rm mozdef/mozdef_base python3 --version     # expect Python 3.6.x
  ```
- **Goal**
  - Prove that **Node 8** and **Python 3.6** live inside containers, independent of CentOS/RHEL 8 host packages.

---

## Milestone 3 – Online: Full Stack Test (Infra → Backend → Frontend)

### 3.1 Start everything
- **Do**
  ```bash
  cd /root/MozDef
  make run        # starts all infra + backend + frontend
  sleep 300       # wait 5–8 minutes
  ```
- **Check**
  ```bash
  docker-compose -f docker/compose/docker-compose.yml -p mozdef ps
  # all services should show "Up" (no "unhealthy")
  ```

### 3.2 Infra checks (shared for backend + frontend)
- **Elasticsearch**
  ```bash
  docker exec -it mozdef_elasticsearch_1 curl \
    http://127.0.0.1:9200/_cluster/health?pretty
  # status: "green" or "yellow"
  ```
- **RabbitMQ**
  ```bash
  docker exec -it mozdef_rabbitmq_1 curl \
    http://127.0.0.1:15672 || echo "mgmt UI ok if 200/redirect"
  ```
- **MongoDB**
  ```bash
  docker exec -it mozdef_mongodb_1 \
    mongo --port 3002 --eval "db.adminCommand('ping')"
  ```

### 3.3 Backend checks
- **loginput** (ingest API)
  ```bash
  curl http://localhost:8080/status
  # {"status":"ok","service":"loginput"}
  ```
- **rest** (REST API for Meteor)
  ```bash
  curl http://localhost:8081/status
  # {"status":"ok","service":"restapi"}
  ```

### 3.4 Frontend checks
- **Meteor UI**
  ```bash
  curl -I http://localhost
  # HTTP/1.1 200 OK
  ```
- **Kibana**
  ```bash
  curl -I http://localhost:9090/app/kibana
  # HTTP/1.1 200 OK
  ```

### 3.5 End‑to‑end event test (frontend → backend → Elasticsearch)
- **Do**
  ```bash
  curl -X POST http://localhost:8080/events \
    -H "Content-Type: application/json" \
    -d '{
      "timestamp": "2024-01-15T10:00:00+00:00",
      "utctimestamp": "2024-01-15T10:00:00+00:00",
      "hostname": "test.example.com",
      "processname": "test.py",
      "processid": 1234,
      "severity": "INFO",
      "summary": "Online pre-pack test event",
      "category": "test",
      "source": "test",
      "tags": ["test"],
      "details": {}
    }'
  sleep 10
  ```
- **Check**
  ```bash
  docker exec -it mozdef_elasticsearch_1 curl \
    "http://127.0.0.1:9200/events-*/_search?q=category:test&size=1&pretty"
  # expect your test event in hits
  ```

### 3.6 Stop stack (ready to freeze for offline)
- **Do**
  ```bash
  cd /root/MozDef
  make stop
  docker-compose -f docker/compose/docker-compose.yml -p mozdef ps
  # no containers "Up"
  ```

---

## Milestone 4 – Online: Package for Offline

### 4.1 Package Docker images (all layers)
- **Do (single tar for all)**  
  ```bash
  mkdir -p ~/mozdef-offline-package/docker-images
  cd /root/MozDef

  docker save $(docker images mozdef/* --format "{{.Repository}}:{{.Tag}}") \
    -o ~/mozdef-offline-package/docker-images/mozdef-all-images.tar
  ```
- **Check**
  ```bash
  ls -lh ~/mozdef-offline-package/docker-images/mozdef-all-images.tar
  # ~10–20 GB
  ```

### 4.2 (Optional safety) cache Python / npm
- **Python (backend libs)**
  ```bash
  mkdir -p ~/mozdef-offline-package/python-packages
  cd /root/MozDef

  python3 -m venv /tmp/mozdef-download-env
  source /tmp/mozdef-download-env/bin/activate

  pip download -r requirements.txt \
    -d ~/mozdef-offline-package/python-packages \
    --platform linux_x86_64 --python-version 36 --only-binary=:all: || true
  ```
- **npm (frontend libs)** – only if you expect to rebuild Meteor offline:
  ```bash
  mkdir -p ~/mozdef-offline-package/npm-packages
  cd /root/MozDef/meteor

  if [ ! -f package-lock.json ]; then
    npm install --package-lock-only
  fi

  npm ci --cache ~/mozdef-offline-package/npm-packages/.npm \
    --prefer-offline=false || npm install --cache ~/mozdef-offline-package/npm-packages/.npm
  ```

### 4.3 Package source code
- **Do**
  ```bash
  mkdir -p ~/mozdef-offline-package/source-code
  tar -czf ~/mozdef-offline-package/source-code/MozDef-source.tar.gz /root/MozDef
  ```

### 4.4 Create final archive to move
- **Do**
  ```bash
  cd ~/mozdef-offline-package
  tar -czf ../mozdef-offline-package-$(date +%Y%m%d).tar.gz .
  du -sh ../mozdef-offline-package-*.tar.gz
  ```

---

## Milestone 5 – Transfer to Offline

- **Option: USB / disk**
  ```bash
  cp ../mozdef-offline-package-*.tar.gz /mnt/usb/
  ```

- **Option: split if too big**
  ```bash
  cd ~
  split -b 4G mozdef-offline-package-*.tar.gz mozdef-part-
  # rejoin on offline machine:
  # cat mozdef-part-* > mozdef-offline-package.tar.gz
  ```

---

## Milestone 6 – Offline: Load Images & Run

### 6.1 Extract package
- **Do**
  ```bash
  mkdir -p ~/mozdef-offline-install
  cd ~/mozdef-offline-install

  tar -xzf /path/to/mozdef-offline-package-*.tar.gz
  cd mozdef-offline-package
  ```

### 6.2 Load Docker images (infra + backend + frontend)
- **Do**
  ```bash
  cd docker-images

  docker load -i mozdef-all-images.tar

  docker images | grep mozdef
  # should list all mozdef/* images
  ```

### 6.3 Install source code
- **Do**
  ```bash
  cd ~/mozdef-offline-install/mozdef-offline-package
  mkdir -p /opt/mozdef

  tar -xzf source-code/MozDef-source.tar.gz -C /opt/mozdef
  cd /opt/mozdef/MozDef
  ```

### 6.4 Quick isolation sanity check (same as online)
- **Do**
  ```bash
  node --version || echo "host node (irrelevant)"
  docker run --rm mozdef/mozdef_meteor node --version     # expect v8.11.4

  python3 --version || echo "host python (irrelevant)"
  docker run --rm mozdef/mozdef_base python3 --version    # expect Python 3.6.x
  ```

---

## Milestone 7 – Offline: Start Stack & Validate

### 7.1 Start all services offline
- **Do**
  ```bash
  cd /opt/mozdef/MozDef
  make run
  sleep 300
  ```
- **Check**
  ```bash
  docker-compose -f docker/compose/docker-compose.yml -p mozdef ps
  # all services "Up"
  ```

### 7.2 Infra checks (same as online)
- **Elasticsearch / RabbitMQ / MongoDB** – same commands as Milestone 3.2.

### 7.3 Backend checks
- **loginput**
  ```bash
  curl http://localhost:8080/status
  ```
- **rest**
  ```bash
  curl http://localhost:8081/status
  ```

### 7.4 Frontend checks
- **Meteor UI**
  ```bash
  curl -I http://localhost
  ```
- **Kibana**
  ```bash
  curl -I http://localhost:9090/app/kibana
  ```

### 7.5 Offline end‑to‑end event test
- **Do**
  ```bash
  curl -X POST http://localhost:8080/events \
    -H "Content-Type: application/json" \
    -d '{
      "timestamp": "2024-01-15T10:00:00+00:00",
      "utctimestamp": "2024-01-15T10:00:00+00:00",
      "hostname": "offline.example.com",
      "processname": "test.py",
      "processid": 1234,
      "severity": "INFO",
      "summary": "Offline test event",
      "category": "offline-test",
      "source": "offline",
      "tags": ["offline","test"],
      "details": {}
    }'
  sleep 10
  ```
- **Check**
  ```bash
  docker exec -it mozdef_elasticsearch_1 curl \
    "http://127.0.0.1:9200/events-*/_search?q=category:offline-test&size=1&pretty"
  # expect your offline event
  ```

---

## How to Use This Milestone File

- Go milestone by milestone.
- After each **Check** section passes, you know that layer (infra/backend/frontend) is good.
- The detailed command reference for packaging, manifests, and scripts lives in `OFFLINE_DEPLOYMENT_GUIDE.md`.  
  This file is your **progress checklist**; the guide is the **full cookbook**.