| # MozDef - Project Packaging & Transfer Guide |
|
|
| ## π¦ Project Size & Location |
|
|
| ### Project Structure |
| ``` |
| /root/MozDef/ |
| βββ Source Code (application files) |
| βββ docker/ (Docker configurations) |
| β βββ compose/ (Docker Compose files) |
| β βββ Dockerfiles (for each service) |
| βββ Documentation (*.md files) |
| βββ Configuration files |
| ``` |
|
|
| --- |
|
|
| ## π Project Size Analysis |
|
|
| ### 1. Source Code Size |
| **Location:** `/root/MozDef` |
|
|
| **Size Breakdown:** |
| - Source code: ~50-200 MB (varies) |
| - Documentation: ~1-5 MB |
| - Configuration files: ~1-10 MB |
|
|
| **Check size:** |
| ```bash |
| du -sh /root/MozDef |
| du -sh /root/MozDef/* | sort -h |
| ``` |
|
|
| ### 2. Docker Images Size |
| **Location:** Docker's storage directory (usually `/var/lib/docker/`) |
|
|
| **Total Images:** 17 MozDef images |
|
|
| **Estimated Size:** |
| - Base images: ~500 MB - 2 GB each |
| - Application images: ~200 MB - 1 GB each |
| - **Total: ~5-15 GB** (depends on base images) |
|
|
| **Check Docker images:** |
| ```bash |
| docker images mozdef/* --format "{{.Repository}}:{{.Tag}} {{.Size}}" |
| docker system df |
| ``` |
|
|
| ### 3. Docker Volumes (Data) |
| **Location:** `/var/lib/docker/volumes/` |
|
|
| **Volumes:** |
| - `mozdef_elasticsearch` (event data) |
| - `mozdef_mongodb` (Meteor data) |
| - `mozdef_rabbitmq` (queue data) |
| - `mozdef_geolite_db` (GeoIP data) |
|
|
| **Size:** Varies based on data (can be GBs for production) |
|
|
| **Check volumes:** |
| ```bash |
| docker volume ls | grep mozdef |
| docker system df -v |
| ``` |
|
|
| --- |
|
|
| ## π Key File Locations |
|
|
| ### Docker Files |
| ``` |
| /root/MozDef/docker/compose/ |
| βββ docker-compose.yml # Main compose file |
| βββ */Dockerfile # Service Dockerfiles |
| βββ */files/ # Configuration files |
| ``` |
|
|
| ### Source Code |
| ``` |
| /root/MozDef/ |
| βββ loginput/ # Loginput service |
| βββ rest/ # REST API service |
| βββ mq/ # MQ Worker |
| βββ alerts/ # Alerts service |
| βββ meteor/ # Meteor frontend |
| βββ config/ # Configuration files |
| ``` |
|
|
| ### Docker Images (Runtime) |
| **Location:** Docker's internal storage |
| - Default: `/var/lib/docker/` |
| - Check: `docker info | grep "Docker Root Dir"` |
|
|
| --- |
|
|
| ## π¦ Creating Archive for Transfer |
|
|
| ### Method 1: Complete Archive (Recommended) |
|
|
| #### Step 1: Save Docker Images |
| ```bash |
| # Create archive directory |
| ARCHIVE_DIR="$HOME/mozdef-archive-$(date +%Y%m%d)" |
| mkdir -p "$ARCHIVE_DIR/docker-images" |
| |
| # Save all MozDef images |
| docker save $(docker images mozdef/* --format "{{.Repository}}:{{.Tag}}") \ |
| -o "$ARCHIVE_DIR/docker-images/mozdef-all-images.tar" |
| |
| # Check size |
| ls -lh "$ARCHIVE_DIR/docker-images/mozdef-all-images.tar" |
| ``` |
|
|
| #### Step 2: Archive Source Code |
| ```bash |
| # Archive source code |
| tar -czf "$ARCHIVE_DIR/source-code/MozDef-source.tar.gz" \ |
| -C /root MozDef \ |
| --exclude='MozDef/.git' \ |
| --exclude='MozDef/node_modules' \ |
| --exclude='MozDef/.meteor/local' \ |
| --exclude='MozDef/__pycache__' |
| |
| # Check size |
| ls -lh "$ARCHIVE_DIR/source-code/MozDef-source.tar.gz" |
| ``` |
|
|
| #### Step 3: Save Docker Volumes (Optional - for data) |
| ```bash |
| # Save Elasticsearch data |
| docker run --rm \ |
| -v mozdef_elasticsearch:/data \ |
| -v "$ARCHIVE_DIR":/backup \ |
| alpine tar czf /backup/elasticsearch-data.tar.gz -C /data . |
| |
| # Save MongoDB data |
| docker run --rm \ |
| -v mozdef_mongodb:/data \ |
| -v "$ARCHIVE_DIR":/backup \ |
| alpine tar czf /backup/mongodb-data.tar.gz -C /data . |
| ``` |
|
|
| #### Step 4: Copy Documentation |
| ```bash |
| # Copy all documentation |
| cp /root/MozDef/*.md "$ARCHIVE_DIR/documentation/" 2>/dev/null |
| ``` |
|
|
| #### Step 5: Create Final Archive |
| ```bash |
| # Create single archive |
| cd "$(dirname "$ARCHIVE_DIR")" |
| tar -czf "mozdef-complete-$(date +%Y%m%d).tar.gz" "$(basename "$ARCHIVE_DIR")" |
| |
| # Check final size |
| du -sh "mozdef-complete-$(date +%Y%m%d).tar.gz" |
| ``` |
|
|
| ### Method 2: Split Archive (for large files) |
|
|
| If archive is too large, split it: |
|
|
| ```bash |
| # Split into 2GB chunks |
| split -b 2G mozdef-complete.tar.gz mozdef-complete.tar.gz.part |
| |
| # Reassemble on target: |
| # cat mozdef-complete.tar.gz.part* > mozdef-complete.tar.gz |
| ``` |
|
|
| --- |
|
|
| ## π Transferring to Another Device |
|
|
| ### Option 1: USB Drive / External Storage |
|
|
| ```bash |
| # 1. Mount USB drive |
| sudo mkdir -p /mnt/usb |
| sudo mount /dev/sdX1 /mnt/usb |
| |
| # 2. Copy archive |
| cp mozdef-complete-*.tar.gz /mnt/usb/ |
| |
| # 3. Unmount |
| sudo umount /mnt/usb |
| ``` |
|
|
| ### Option 2: Network Transfer (SCP) |
|
|
| ```bash |
| # From source machine |
| scp mozdef-complete-*.tar.gz user@target-machine:/path/to/destination/ |
| |
| # Or using rsync (better for large files) |
| rsync -avz --progress mozdef-complete-*.tar.gz user@target-machine:/path/ |
| ``` |
|
|
| ### Option 3: Network Share (NFS/SMB) |
|
|
| ```bash |
| # Mount network share |
| sudo mount -t nfs server:/share /mnt/nfs |
| |
| # Copy archive |
| cp mozdef-complete-*.tar.gz /mnt/nfs/ |
| ``` |
|
|
| ### Option 4: Cloud Storage (if allowed) |
|
|
| ```bash |
| # Upload to cloud storage |
| # Example with AWS S3: |
| aws s3 cp mozdef-complete-*.tar.gz s3://bucket-name/ |
| |
| # Download on target: |
| aws s3 cp s3://bucket-name/mozdef-complete-*.tar.gz ./ |
| ``` |
|
|
| --- |
|
|
| ## π₯ Restoring on Target Device |
|
|
| ### Step 1: Extract Archive |
| ```bash |
| # Extract archive |
| tar -xzf mozdef-complete-*.tar.gz |
| cd mozdef-archive-*/ |
| ``` |
|
|
| ### Step 2: Load Docker Images |
| ```bash |
| # Load Docker images |
| docker load -i docker-images/mozdef-all-images.tar |
| |
| # Verify images loaded |
| docker images mozdef/* |
| ``` |
|
|
| ### Step 3: Extract Source Code |
| ```bash |
| # Extract source code |
| tar -xzf source-code/MozDef-source.tar.gz -C /root/ |
| |
| # Verify |
| ls -la /root/MozDef |
| ``` |
|
|
| ### Step 4: Restore Volumes (if needed) |
| ```bash |
| # Create volumes first |
| docker volume create mozdef_elasticsearch |
| docker volume create mozdef_mongodb |
| docker volume create mozdef_rabbitmq |
| docker volume create mozdef_geolite_db |
| |
| # Restore data |
| docker run --rm \ |
| -v mozdef_elasticsearch:/data \ |
| -v "$(pwd)":/backup \ |
| alpine tar xzf /backup/elasticsearch-data.tar.gz -C /data |
| ``` |
|
|
| ### Step 5: Start Services |
| ```bash |
| # Navigate to project |
| cd /root/MozDef |
| |
| # Start services |
| docker-compose -f docker/compose/docker-compose.yml -p mozdef up -d |
| |
| # Verify |
| docker-compose -f docker/compose/docker-compose.yml -p mozdef ps |
| ``` |
|
|
| --- |
|
|
| ## π Quick Archive Script |
|
|
| Save this script as `archive_mozdef.sh`: |
|
|
| ```bash |
| #!/bin/bash |
| # MozDef Complete Archive Script |
| |
| ARCHIVE_DIR="$HOME/mozdef-archive-$(date +%Y%m%d)" |
| mkdir -p "$ARCHIVE_DIR"/{docker-images,source-code,documentation} |
| |
| echo "=== Creating MozDef Archive ===" |
| echo "" |
| |
| # 1. Save Docker Images |
| echo "1/4 Saving Docker images..." |
| docker save $(docker images mozdef/* --format "{{.Repository}}:{{.Tag}}") \ |
| -o "$ARCHIVE_DIR/docker-images/mozdef-all-images.tar" |
| echo " β
Images saved: $(du -sh "$ARCHIVE_DIR/docker-images/mozdef-all-images.tar" | awk '{print $1}')" |
| |
| # 2. Archive Source Code |
| echo "2/4 Archiving source code..." |
| tar -czf "$ARCHIVE_DIR/source-code/MozDef-source.tar.gz" \ |
| -C /root MozDef \ |
| --exclude='MozDef/.git' \ |
| --exclude='MozDef/node_modules' \ |
| --exclude='MozDef/.meteor/local' |
| echo " β
Source archived: $(du -sh "$ARCHIVE_DIR/source-code/MozDef-source.tar.gz" | awk '{print $1}')" |
| |
| # 3. Copy Documentation |
| echo "3/4 Copying documentation..." |
| cp /root/MozDef/*.md "$ARCHIVE_DIR/documentation/" 2>/dev/null |
| echo " β
Documentation copied" |
| |
| # 4. Create Final Archive |
| echo "4/4 Creating final archive..." |
| cd "$(dirname "$ARCHIVE_DIR")" |
| tar -czf "mozdef-complete-$(date +%Y%m%d).tar.gz" "$(basename "$ARCHIVE_DIR")" |
| echo " β
Final archive: $(du -sh "mozdef-complete-$(date +%Y%m%d).tar.gz" | awk '{print $1}')" |
| |
| echo "" |
| echo "=== Archive Complete ===" |
| echo "Location: $(pwd)/mozdef-complete-$(date +%Y%m%d).tar.gz" |
| ``` |
|
|
| **Usage:** |
| ```bash |
| chmod +x archive_mozdef.sh |
| ./archive_mozdef.sh |
| ``` |
|
|
| --- |
|
|
| ## π Estimated Archive Sizes |
|
|
| ### Minimal Archive (Images + Source) |
| - **Docker Images:** ~5-15 GB |
| - **Source Code:** ~100-500 MB |
| - **Total:** ~5-16 GB |
|
|
| ### Complete Archive (with data) |
| - **Docker Images:** ~5-15 GB |
| - **Source Code:** ~100-500 MB |
| - **Volumes (data):** Varies (can be GBs) |
| - **Total:** Depends on data size |
|
|
| --- |
|
|
| ## β
Pre-Transfer Checklist |
|
|
| - [ ] All Docker images saved |
| - [ ] Source code archived |
| - [ ] Documentation included |
| - [ ] Volumes backed up (if needed) |
| - [ ] Archive size verified |
| - [ ] Archive integrity checked (optional: `md5sum`) |
|
|
| --- |
|
|
| ## π Verification Commands |
|
|
| ### Check Archive Contents |
| ```bash |
| # List archive contents |
| tar -tzf mozdef-complete-*.tar.gz | head -20 |
| |
| # Check archive integrity |
| tar -tzf mozdef-complete-*.tar.gz > /dev/null && echo "Archive OK" |
| ``` |
|
|
| ### Verify Docker Images |
| ```bash |
| # After loading |
| docker images mozdef/* | wc -l |
| # Should show 17 images |
| ``` |
|
|
| ### Verify Source Code |
| ```bash |
| # After extracting |
| ls -la /root/MozDef |
| # Should show all project files |
| ``` |
|
|
| --- |
|
|
| ## π Summary |
|
|
| **Project Location:** `/root/MozDef` |
|
|
| **Key Components:** |
| 1. **Source Code:** `/root/MozDef` (~100-500 MB) |
| 2. **Docker Images:** Docker storage (~5-15 GB) |
| 3. **Docker Files:** `/root/MozDef/docker/` |
| 4. **Volumes (Data):** `/var/lib/docker/volumes/` (varies) |
|
|
| **Archive Creation:** |
| - Save images: `docker save` |
| - Archive source: `tar -czf` |
| - Create final: Single tar.gz file |
|
|
| **Transfer Methods:** |
| - USB drive |
| - Network (SCP/rsync) |
| - Network share |
| - Cloud storage (if allowed) |
|
|
| **Restore Process:** |
| 1. Extract archive |
| 2. Load Docker images |
| 3. Extract source code |
| 4. Start services |
|
|
| --- |
|
|
| **Last Updated:** $(date) |
|
|