QUICK_START.md

Quick Start Guide - VQA Mobile App

This guide will help you get the VQA mobile app running quickly.

Prerequisites Checklist

• Python 3.8+ installed
• Node.js 16+ installed
• VQA model checkpoints available
• Smartphone with Expo Go app installed
• Computer and phone on same WiFi network

Step-by-Step Setup

Step 1: Start the Backend Server

# Open terminal/PowerShell
cd c:\Users\rdeva\Downloads\vqa_coes

# Install backend dependencies (first time only)
pip install -r requirements_api.txt

# Start the server
python backend_api.py

Expected output:

🚀 INITIALIZING ENSEMBLE VQA SYSTEM
⚙️  Device: cuda
📥 Loading models...
✅ Ensemble ready!

Important: Keep this terminal window open! The server must keep running.

Step 2: Find Your Local IP Address

Windows:

ipconfig

Look for "IPv4 Address" under your WiFi adapter (e.g., 192.168.1.100)

Mac/Linux:

ifconfig
# or
ip addr

Step 3: Configure the Mobile App

1. Open ui/src/config/api.js
2. Replace the IP address:

   export const API_BASE_URL = 'http://YOUR_IP_HERE:8000';
   // Example: export const API_BASE_URL = 'http://192.168.1.100:8000';
   

Step 4: Configure Google OAuth (Optional for Testing)

For testing without Google login, you can skip this and modify the app to bypass authentication.

For full Google login:

1. Go to Google Cloud Console
2. Create a project
3. Enable Google+ API
4. Create OAuth 2.0 credentials
5. Update ui/src/config/google.js with your client IDs

Step 5: Start the Mobile App

# Open a NEW terminal/PowerShell
cd c:\Users\rdeva\Downloads\vqa_coes\ui

# Start Expo
npm start

Expected output:

Metro waiting on exp://192.168.1.100:8081
› Scan the QR code above with Expo Go (Android) or the Camera app (iOS)

Step 6: Run on Your Phone

1. Install Expo Go on your phone:

   • Android - Play Store
   • iOS - App Store

2. Scan the QR code:

   • Android: Open Expo Go app → Scan QR code
   • iOS: Open Camera app → Scan QR code → Tap notification

3. Wait for app to load (first time may take 1-2 minutes)

Testing Without Google Login

If you want to test the VQA functionality without setting up Google OAuth:

1. Open ui/App.js
2. Temporarily modify the navigation to always show HomeScreen:

// Replace this:
{user ? (
  <Stack.Screen name="Home" component={HomeScreen} />
) : (
  <Stack.Screen name="Login" component={LoginScreen} />
)}

// With this:
<Stack.Screen name="Home" component={HomeScreen} />

3. Restart the Expo server

Testing the App

Test 1: General Question (Base Model)

1. Tap "Gallery" and select an image
2. Enter question: "What color is the car?"
3. Tap "Ask Question"
4. Should show: 🔍 Base Model

Test 2: Spatial Question (Spatial Model)

1. Select an image with multiple objects
2. Enter question: "What is to the right of the table?"
3. Tap "Ask Question"
4. Should show: 📍 Spatial Model

Troubleshooting

"Cannot connect to server"

• ✅ Check backend is running
• ✅ Verify IP address in api.js is correct
• ✅ Ensure phone and computer on same WiFi
• ✅ Check firewall isn't blocking port 8000

"Model not loaded"

• ✅ Check checkpoint files are in project root
• ✅ Verify file names: vqa_checkpoint.pt and vqa_spatial_checkpoint.pt
• ✅ Check backend terminal for error messages

App won't load on phone

• ✅ Ensure Expo Go is installed
• ✅ Check both devices on same network
• ✅ Try restarting Expo server (Ctrl+C, then npm start)
• ✅ Clear Expo cache: npm start -- --clear

"Permission denied" for camera/gallery

• ✅ Grant permissions when prompted
• ✅ Check phone settings → App permissions

Next Steps

Once everything works:

1. Set up Google OAuth for production use
2. Customize the UI in src/styles/theme.js
3. Add custom icons in assets/ folder
4. Build standalone app with eas build

Quick Commands Reference

# Start backend
cd c:\Users\rdeva\Downloads\vqa_coes
python backend_api.py

# Start mobile app
cd c:\Users\rdeva\Downloads\vqa_coes\ui
npm start

# Clear Expo cache
npm start -- --clear

# Install new package
npm install package-name

# Check backend health
curl http://localhost:8000/health

Support

If you encounter issues:

1. Check the main README.md
2. Review backend terminal logs
3. Check Expo console for errors
4. Verify all prerequisites are met