SECRETS_SETUP.md

# Hugging Face Spaces Secrets Setup Guide

This guide explains how to configure and deploy your Telugu Dialect Map to Hugging Face Spaces with secure secrets management.

## Overview

Your application requires two secret files:
- **`config.json`**: Configuration for Google Sheets sync and automation
- **`credentials.json`**: Google Cloud service account credentials

These files contain sensitive information and should NEVER be committed to git. Instead, we'll use Hugging Face Spaces **secrets** (environment variables) to store them securely.

---

## Step 1: Obtain Google Service Account Credentials

### 1.1 Create a Google Cloud Project

1. Go to [Google Cloud Console](https://console.cloud.google.com/)
2. Create a new project or select an existing one
3. Note your project ID

### 1.2 Enable Google Sheets API

1. In your project, go to **APIs & Services** → **Library**
2. Search for "Google Sheets API"
3. Click **Enable**

### 1.3 Create a Service Account

1. Go to **APIs & Services** → **Credentials**
2. Click **Create Credentials** → **Service Account**
3. Fill in the details:
   - **Service account name**: `dialect-map-automation`
   - **Service account ID**: (auto-generated)
   - **Description**: "Service account for dialect map Google Sheets automation"
4. Click **Create and Continue**
5. Skip the optional steps (roles and user access)
6. Click **Done**

### 1.4 Create and Download Service Account Key

1. Click on the service account you just created
2. Go to the **Keys** tab
3. Click **Add Key** → **Create new key**
4. Select **JSON** format
5. Click **Create**
6. A JSON file will be downloaded - this is your `credentials.json`
7. **Keep this file secure!** It provides full access to your Google Sheets

### 1.5 Share Your Google Sheets with the Service Account

1. Open your `credentials.json` file
2. Find the `client_email` field (e.g., `dialect-map-automation@your-project.iam.gserviceaccount.com`)
3. Copy this email address
4. Open each Google Sheet you want to sync
5. Click **Share** button
6. Paste the service account email
7. Give it **Editor** or **Viewer** access (Editor if you want to write data back)
8. Click **Send**

---

## Step 2: Configure config.json

Create your `config.json` file with your specific settings:

```json
{
  "google_sheets": {
    "enabled": true,
    "sync_interval_minutes": 5,
    "credentials_file": "credentials.json",
    "spreadsheets": [
      {
        "id": "YOUR_ACTUAL_SPREADSHEET_ID_HERE",
        "sheet_name": "processed_dialects",
        "output_file": "sheets_output/processed_dialects.csv"
      },
      {
        "id": "YOUR_ACTUAL_SPREADSHEET_ID_HERE",
        "sheet_name": "digiwords_grouped",
        "output_file": "sheets_output/digiwords_grouped.csv"
      }
    ]
  },
  "file_watcher": {
    "enabled": true,
    "watch_directory": "sheets_output",
    "file_patterns": ["*.csv"]
  },
  "output": {
    "json_directory": "data/processed"
  }
}
```

### Finding Your Spreadsheet ID

Your Google Sheets URL looks like:
```
https://docs.google.com/spreadsheets/d/1AbC123XyZ456_Example_ID/edit#gid=0
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^
                                    This is your spreadsheet ID
```

Copy the ID from your URL and replace `YOUR_ACTUAL_SPREADSHEET_ID_HERE` in the config.

---

## Step 3: Deploy to Hugging Face Spaces

### 3.1 Create a New Space

1. Go to [Hugging Face](https://huggingface.co/)
2. Click your profile → **New Space**
3. Fill in the details:
   - **Space name**: `telugu-dialect-map` (or your choice)
   - **License**: Choose appropriate license
   - **Space SDK**: Select **Static** (we'll use a custom Python app)
   - **Visibility**: Public or Private
4. Click **Create Space**

### 3.2 Push Your Code to the Space

You can either:

**Option A: Use Git**
```bash
# Clone the Space repository
git clone https://huggingface.co/spaces/YOUR_USERNAME/telugu-dialect-map
cd telugu-dialect-map

# Copy your project files (excluding secrets!)
cp -r /path/to/dialect-map/* .

# Make sure .gitignore is in place
cat .gitignore  # Should include config.json and credentials.json

# Commit and push
git add .
git commit -m "Initial commit"
git push
```

**Option B: Upload via Web Interface**
1. In your Space, click **Files** tab
2. Click **Add file** → **Upload files**
3. Select all your project files (EXCEPT `config.json` and `credentials.json`)
4. Click **Commit changes**

### 3.3 Add Secrets to Your Space

This is the **critical step** - we'll add your sensitive credentials as secrets.

1. In your Space, click the **Settings** tab
2. Scroll down to **Repository secrets**
3. Add the following secrets:

#### Secret 1: HF_CONFIG_JSON

- **Name**: `HF_CONFIG_JSON`
- **Value**: Paste the **entire contents** of your `config.json` file
  
  Example (all on one line):
  ```
  {"google_sheets":{"enabled":true,"sync_interval_minutes":5,"credentials_file":"credentials.json","spreadsheets":[{"id":"1AbC123XyZ456_Example","sheet_name":"processed_dialects","output_file":"sheets_output/processed_dialects.csv"}]},"file_watcher":{"enabled":true,"watch_directory":"sheets_output","file_patterns":["*.csv"]},"output":{"json_directory":"data/processed"}}
  ```

#### Secret 2: HF_CREDENTIALS_JSON

- **Name**: `HF_CREDENTIALS_JSON`
- **Value**: Paste the **entire contents** of your `credentials.json` file

  Example (all on one line, with escaped newlines in private key):
  ```
  {"type":"service_account","project_id":"your-project","private_key_id":"abc123...","private_key":"-----BEGIN PRIVATE KEY-----\\nMIIEvQIB...\\n-----END PRIVATE KEY-----\\n","client_email":"name@project.iam.gserviceaccount.com",...}
  ```

4. Click **Add secret** for each one

**Important Notes:**
- The entire JSON must be on one line (no newlines except in the `private_key` field where `\n` should be `\\n`)
- Make sure to escape special characters if needed
- You can use a JSON minifier tool to compact your JSON

### 3.4 Rebuild Your Space

After adding secrets:
1. Your Space should automatically rebuild
2. Watch the **Logs** tab for any errors
3. Once built, click **App** tab to view your running application

---

## Step 4: Verify Deployment

### 4.1 Check Space Logs

1. Go to your Space's **Logs** tab
2. You should see:
   ```
   🔐 Loading secrets from environment variables...
   ✅ Created config.json from HF_CONFIG_JSON secret
   ✅ Created credentials.json from HF_CREDENTIALS_JSON secret
   🚀 Starting automation runner...
   ✅ Automation runner started
   🌐 Starting web server on port 7860...
   ```

3. If you see errors, check:
   - JSON formatting in your secrets
   - Spreadsheet IDs are correct
   - Service account has access to sheets

### 4.2 Access Your Application

1. Click the **App** tab
2. You should see your Telugu Dialect Map interface
3. The map should load with data from your Google Sheets

### 4.3 Verify Automation

1. Edit your Google Sheet (add/modify some dialect data)
2. Wait 5 minutes (or your configured interval)
3. Check the Space logs - you should see sync messages
4. Refresh your app - changes should appear

---

## Troubleshooting

### "HF_CONFIG_JSON not found in environment"

**Problem**: The secret wasn't added or has the wrong name.

**Solution**:
1. Go to Space Settings → Repository secrets
2. Verify the secret name is exactly `HF_CONFIG_JSON` (case-sensitive)
3. Add it if missing
4. Rebuild the Space

### "Error parsing HF_CONFIG_JSON"

**Problem**: The JSON is malformed.

**Solution**:
1. Copy your `config.json` content
2. Use a JSON validator (e.g., [jsonlint.com](https://jsonlint.com/))
3. Make sure it's valid JSON
4. Remove all newlines (except `\n` in strings should become `\\n`)
5. Update the secret with the corrected value

### "Connection refused" or "Credentials invalid"

**Problem**: Google service account credentials are wrong or not shared.

**Solution**:
1. Verify `credentials.json` content is correct
2. Check that you shared your Google Sheets with the service account email
3. Verify the Sheets API is enabled in Google Cloud Console
4. Regenerate service account key if needed

### "Automation not syncing"

**Problem**: Automation runner isn't working.

**Solution**:
1. Check Space logs for error messages
2. Verify `spreadsheets.id` values in config.json match your actual sheet IDs
3. Verify `sheet_name` values match the tab names in your spreadsheets
4. Check that service account has Editor access (not just Viewer)

### Space keeps crashing or restarting

**Problem**: HF Spaces free tier may have resource limits.

**Solution**:
1. Consider upgrading to a paid Space for guaranteed uptime
2. Reduce sync interval (e.g., 15-30 minutes instead of 5)
3. Check logs for memory/CPU issues

---

## Local Testing (Before Deploying)

Before deploying to HF Spaces, test locally:

### 1. Create actual config files

```bash
cd /home/kashikuldeep/Desktop/dialect-map

# Copy examples and fill in real values
cp config.json.example config.json
cp credentials.json.example credentials.json

# Edit with your actual values
nano config.json
nano credentials.json
```

### 2. Test the app

```bash
# Run the app locally
python app.py

# Should see:
# ✅ Created config.json from HF_CONFIG_JSON secret (if using .env)
# 🚀 Starting automation runner...
# 🌐 Starting web server on port 7860...

# Open browser to http://localhost:7860
```

### 3. Test with environment variables (simulating HF Spaces)

```bash
# Create .env file
cp .env.example .env

# Edit .env with your actual JSON (minified)
nano .env

# Load environment and run
python -c "from dotenv import load_dotenv; load_dotenv()" && python app.py
```

---

## Security Best Practices

1. **Never commit secrets to git**
   - Keep `config.json` and `credentials.json` in `.gitignore`
   - Double-check before pushing code

2. **Rotate credentials regularly**
   - Generate new service account keys periodically
   - Update HF Spaces secrets

3. **Use minimal permissions**
   - Service account should only have access to necessary sheets
   - Use Viewer access if you don't need to write back

4. **Monitor usage**
   - Check Google Cloud Console for API usage
   - Set up billing alerts
   - Review Space logs regularly

---

## Need Help?

- **Google Cloud Issues**: [Google Cloud Support](https://cloud.google.com/support)
- **Hugging Face Spaces**: [HF Documentation](https://huggingface.co/docs/hub/spaces)
- **Project Issues**: Check the Space logs first, then review this guide

---

**Happy Mapping! 🗺️**