Add GCP testing options and Python API launcher
Browse files## New Files
1. **GCP_TESTING_OPTIONS.md**: Complete guide with 4 options
- Option 1: Cloud Shell (FREE + easiest) ⭐
- Option 2: Web Console (visual)
- Option 3: gcloud CLI local
- Option 4: Python API (no CLI needed)
2. **launch_gcp_python.py**: Python-based GCP launcher
- Uses Google Cloud Compute API
- No gcloud CLI required
- Works with service account or default credentials
- Creates e2-medium spot instance (~$0.01/hr)
3. **launch_gcp_manual.md**: Detailed manual instructions
- Step-by-step for each option
- Troubleshooting
- Cost comparisons
## Recommended: Use Cloud Shell (FREE!)
```bash
# Open: https://shell.cloud.google.com/
curl -O https://huggingface.co/marcosremar2/ensemble-tts-annotation/raw/main/scripts/test/launch_gcp_spot.sh
chmod +x launch_gcp_spot.sh
./launch_gcp_spot.sh
```
Cost: ~$0.003 total (less than 1 cent!)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- GCP_TESTING_OPTIONS.md +251 -0
- scripts/test/launch_gcp_manual.md +182 -0
- scripts/test/launch_gcp_python.py +255 -0
|
@@ -0,0 +1,251 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 🚀 Opções para Testar no GCP
|
| 2 |
+
|
| 3 |
+
Você tem **4 opções** para testar o sistema no GCP. Escolha a mais conveniente:
|
| 4 |
+
|
| 5 |
+
---
|
| 6 |
+
|
| 7 |
+
## ⭐ OPÇÃO 1: Google Cloud Shell (RECOMENDADO - GRÁTIS!)
|
| 8 |
+
|
| 9 |
+
**Vantagens**:
|
| 10 |
+
- ✅ **100% Grátis** (Cloud Shell é grátis)
|
| 11 |
+
- ✅ Já tem gcloud instalado
|
| 12 |
+
- ✅ Não precisa instalar nada local
|
| 13 |
+
- ✅ Mais rápido e fácil
|
| 14 |
+
|
| 15 |
+
**Como fazer**:
|
| 16 |
+
|
| 17 |
+
1. Abra o Cloud Shell: https://shell.cloud.google.com/
|
| 18 |
+
|
| 19 |
+
2. No terminal do Cloud Shell, rode:
|
| 20 |
+
|
| 21 |
+
```bash
|
| 22 |
+
# Download do script
|
| 23 |
+
curl -O https://huggingface.co/marcosremar2/ensemble-tts-annotation/raw/main/scripts/test/launch_gcp_spot.sh
|
| 24 |
+
|
| 25 |
+
# Tornar executável
|
| 26 |
+
chmod +x launch_gcp_spot.sh
|
| 27 |
+
|
| 28 |
+
# Rodar
|
| 29 |
+
./launch_gcp_spot.sh
|
| 30 |
+
```
|
| 31 |
+
|
| 32 |
+
3. O script vai:
|
| 33 |
+
- ✅ Buscar instância mais barata (~e2-medium)
|
| 34 |
+
- ✅ Criar instância spot
|
| 35 |
+
- ✅ Instalar dependências
|
| 36 |
+
- ✅ Rodar teste
|
| 37 |
+
- ✅ Mostrar resultados
|
| 38 |
+
|
| 39 |
+
4. Quando terminar, delete a instância:
|
| 40 |
+
```bash
|
| 41 |
+
gcloud compute instances delete ensemble-test-XXX --zone=us-central1-a --quiet
|
| 42 |
+
```
|
| 43 |
+
|
| 44 |
+
**Custo**: $0 (Cloud Shell) + ~$0.003 (instância por 20min) = **~$0.003 total**
|
| 45 |
+
|
| 46 |
+
---
|
| 47 |
+
|
| 48 |
+
## 🌐 OPÇÃO 2: Console Web (Interface Gráfica)
|
| 49 |
+
|
| 50 |
+
**Vantagens**:
|
| 51 |
+
- ✅ Visual e intuitivo
|
| 52 |
+
- ✅ Não precisa de terminal
|
| 53 |
+
- ✅ Fácil para iniciantes
|
| 54 |
+
|
| 55 |
+
**Como fazer**:
|
| 56 |
+
|
| 57 |
+
1. Acesse: https://console.cloud.google.com/compute/instances
|
| 58 |
+
|
| 59 |
+
2. Click **CREATE INSTANCE**
|
| 60 |
+
|
| 61 |
+
3. Configure:
|
| 62 |
+
- Nome: `ensemble-test`
|
| 63 |
+
- Região: `us-central1 (Iowa)`
|
| 64 |
+
- Zona: `us-central1-a`
|
| 65 |
+
- Tipo: `e2-medium` (2 vCPU, 4GB RAM)
|
| 66 |
+
- ✅ **Marcar**: Spot (Preemptible)
|
| 67 |
+
- Disco: Ubuntu 22.04 LTS, 20GB
|
| 68 |
+
|
| 69 |
+
4. Em **Management → Automation**, cole este script:
|
| 70 |
+
|
| 71 |
+
```bash
|
| 72 |
+
#!/bin/bash
|
| 73 |
+
apt-get update && apt-get install -y python3-pip git
|
| 74 |
+
pip3 install --upgrade pip
|
| 75 |
+
cd /home && git clone https://huggingface.co/marcosremar2/ensemble-tts-annotation
|
| 76 |
+
cd ensemble-tts-annotation
|
| 77 |
+
pip3 install torch --index-url https://download.pytorch.org/whl/cpu
|
| 78 |
+
pip3 install transformers datasets librosa soundfile numpy pandas tqdm scikit-learn
|
| 79 |
+
python3 test_local.py > /tmp/test-results.log 2>&1
|
| 80 |
+
echo "✅ Done. Results: cat /tmp/test-results.log"
|
| 81 |
+
```
|
| 82 |
+
|
| 83 |
+
5. Click **CREATE**
|
| 84 |
+
|
| 85 |
+
6. Após 2 minutos, click **SSH** e rode:
|
| 86 |
+
```bash
|
| 87 |
+
cat /tmp/test-results.log
|
| 88 |
+
```
|
| 89 |
+
|
| 90 |
+
7. **Delete** a instância quando terminar!
|
| 91 |
+
|
| 92 |
+
**Custo**: ~$0.003 (20 minutos)
|
| 93 |
+
|
| 94 |
+
---
|
| 95 |
+
|
| 96 |
+
## 💻 OPÇÃO 3: Instalar gcloud CLI Localmente
|
| 97 |
+
|
| 98 |
+
**Vantagens**:
|
| 99 |
+
- ✅ Controle total do terminal
|
| 100 |
+
- ✅ Automação completa
|
| 101 |
+
- ✅ Reutilizável para outros projetos
|
| 102 |
+
|
| 103 |
+
**Como fazer**:
|
| 104 |
+
|
| 105 |
+
1. Instalar gcloud:
|
| 106 |
+
|
| 107 |
+
```bash
|
| 108 |
+
# macOS
|
| 109 |
+
brew install google-cloud-sdk
|
| 110 |
+
|
| 111 |
+
# Ou download direto
|
| 112 |
+
curl https://sdk.cloud.google.com | bash
|
| 113 |
+
exec -l $SHELL
|
| 114 |
+
```
|
| 115 |
+
|
| 116 |
+
2. Configurar:
|
| 117 |
+
```bash
|
| 118 |
+
gcloud init
|
| 119 |
+
```
|
| 120 |
+
|
| 121 |
+
3. Rodar o script:
|
| 122 |
+
```bash
|
| 123 |
+
cd ensemble-tts-annotation
|
| 124 |
+
bash scripts/test/launch_gcp_spot.sh
|
| 125 |
+
```
|
| 126 |
+
|
| 127 |
+
**Custo**: ~$0.003
|
| 128 |
+
|
| 129 |
+
---
|
| 130 |
+
|
| 131 |
+
## 🐍 OPÇÃO 4: Script Python (Sem gcloud CLI)
|
| 132 |
+
|
| 133 |
+
**Vantagens**:
|
| 134 |
+
- ✅ Não precisa de gcloud CLI
|
| 135 |
+
- ✅ Usa API do GCP diretamente
|
| 136 |
+
- ✅ Programático
|
| 137 |
+
|
| 138 |
+
**Como fazer**:
|
| 139 |
+
|
| 140 |
+
1. Instalar biblioteca:
|
| 141 |
+
```bash
|
| 142 |
+
pip install google-cloud-compute google-auth
|
| 143 |
+
```
|
| 144 |
+
|
| 145 |
+
2. Autenticar:
|
| 146 |
+
|
| 147 |
+
**Opção A** - Application Default Credentials (se tem gcloud):
|
| 148 |
+
```bash
|
| 149 |
+
gcloud auth application-default login
|
| 150 |
+
```
|
| 151 |
+
|
| 152 |
+
**Opção B** - Service Account (sem gcloud):
|
| 153 |
+
1. Console: https://console.cloud.google.com/iam-admin/serviceaccounts
|
| 154 |
+
2. Create Service Account
|
| 155 |
+
3. Grant role: **Compute Instance Admin**
|
| 156 |
+
4. Create key → Download JSON
|
| 157 |
+
5. Set environment:
|
| 158 |
+
```bash
|
| 159 |
+
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/key.json"
|
| 160 |
+
export GCP_PROJECT_ID="seu-projeto-id"
|
| 161 |
+
```
|
| 162 |
+
|
| 163 |
+
3. Rodar script:
|
| 164 |
+
```bash
|
| 165 |
+
python scripts/test/launch_gcp_python.py
|
| 166 |
+
```
|
| 167 |
+
|
| 168 |
+
**Custo**: ~$0.003
|
| 169 |
+
|
| 170 |
+
---
|
| 171 |
+
|
| 172 |
+
## 📊 Comparação das Opções
|
| 173 |
+
|
| 174 |
+
| Opção | Setup | Facilidade | Custo | Tempo |
|
| 175 |
+
|-------|-------|------------|-------|-------|
|
| 176 |
+
| **1. Cloud Shell** | 0 min | ⭐⭐⭐⭐⭐ | $0.003 | 5 min |
|
| 177 |
+
| **2. Console Web** | 0 min | ⭐⭐⭐⭐ | $0.003 | 10 min |
|
| 178 |
+
| **3. gcloud CLI** | 5 min | ⭐⭐⭐ | $0.003 | 5 min |
|
| 179 |
+
| **4. Python API** | 10 min | ⭐⭐ | $0.003 | 5 min |
|
| 180 |
+
|
| 181 |
+
---
|
| 182 |
+
|
| 183 |
+
## 🎯 Minha Recomendação
|
| 184 |
+
|
| 185 |
+
**Use a OPÇÃO 1 (Cloud Shell)**:
|
| 186 |
+
|
| 187 |
+
1. É **grátis** (Cloud Shell não cobra)
|
| 188 |
+
2. É **mais rápido** (já tem tudo instalado)
|
| 189 |
+
3. É **mais fácil** (só copiar e colar)
|
| 190 |
+
|
| 191 |
+
```bash
|
| 192 |
+
# Literalmente 3 comandos:
|
| 193 |
+
curl -O https://huggingface.co/marcosremar2/ensemble-tts-annotation/raw/main/scripts/test/launch_gcp_spot.sh
|
| 194 |
+
chmod +x launch_gcp_spot.sh
|
| 195 |
+
./launch_gcp_spot.sh
|
| 196 |
+
```
|
| 197 |
+
|
| 198 |
+
---
|
| 199 |
+
|
| 200 |
+
## ⚠️ IMPORTANTE: Deletar Instância
|
| 201 |
+
|
| 202 |
+
**Não esqueça de deletar** quando terminar!
|
| 203 |
+
|
| 204 |
+
```bash
|
| 205 |
+
# Via gcloud
|
| 206 |
+
gcloud compute instances delete ensemble-test-XXX --zone=us-central1-a --quiet
|
| 207 |
+
|
| 208 |
+
# Via console
|
| 209 |
+
https://console.cloud.google.com/compute/instances
|
| 210 |
+
→ Selecionar instância → DELETE
|
| 211 |
+
```
|
| 212 |
+
|
| 213 |
+
Se esquecer rodando, custo máximo:
|
| 214 |
+
- $0.01/hora × 24 horas = **$0.24/dia**
|
| 215 |
+
- GCP vai enviar alerta de cobrança
|
| 216 |
+
|
| 217 |
+
---
|
| 218 |
+
|
| 219 |
+
## 📝 Checklist
|
| 220 |
+
|
| 221 |
+
Antes de começar, certifique-se:
|
| 222 |
+
|
| 223 |
+
- [ ] Tem conta GCP (gratuita serve: https://cloud.google.com/free)
|
| 224 |
+
- [ ] Tem projeto criado
|
| 225 |
+
- [ ] Compute Engine API está habilitada
|
| 226 |
+
- [ ] Tem créditos ou cartão configurado (custo: ~$0.003)
|
| 227 |
+
|
| 228 |
+
Se primeira vez usando GCP:
|
| 229 |
+
1. https://console.cloud.google.com/
|
| 230 |
+
2. Create new project
|
| 231 |
+
3. Enable Compute Engine API
|
| 232 |
+
4. Seguir OPÇÃO 1 (Cloud Shell)
|
| 233 |
+
|
| 234 |
+
---
|
| 235 |
+
|
| 236 |
+
## 🚀 Quick Start
|
| 237 |
+
|
| 238 |
+
**Para começar AGORA (5 minutos)**:
|
| 239 |
+
|
| 240 |
+
1. Abrir: https://shell.cloud.google.com/
|
| 241 |
+
2. Colar:
|
| 242 |
+
```bash
|
| 243 |
+
curl -O https://huggingface.co/marcosremar2/ensemble-tts-annotation/raw/main/scripts/test/launch_gcp_spot.sh && chmod +x launch_gcp_spot.sh && ./launch_gcp_spot.sh
|
| 244 |
+
```
|
| 245 |
+
3. Seguir instruções
|
| 246 |
+
4. Ver resultados
|
| 247 |
+
5. Deletar instância
|
| 248 |
+
|
| 249 |
+
**Pronto!** 🎉
|
| 250 |
+
|
| 251 |
+
Custo total: **menos de 1 centavo** 💰
|
|
@@ -0,0 +1,182 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Manual GCP Spot Instance Launch
|
| 2 |
+
|
| 3 |
+
Como você não tem gcloud instalado localmente, aqui estão as opções:
|
| 4 |
+
|
| 5 |
+
## Opção 1: Instalar gcloud CLI (Recomendado)
|
| 6 |
+
|
| 7 |
+
```bash
|
| 8 |
+
# macOS
|
| 9 |
+
brew install google-cloud-sdk
|
| 10 |
+
|
| 11 |
+
# Ou download direto
|
| 12 |
+
curl https://sdk.cloud.google.com | bash
|
| 13 |
+
exec -l $SHELL
|
| 14 |
+
gcloud init
|
| 15 |
+
|
| 16 |
+
# Depois rodar o script
|
| 17 |
+
bash scripts/test/launch_gcp_spot.sh
|
| 18 |
+
```
|
| 19 |
+
|
| 20 |
+
## Opção 2: Usar GCP Console (Interface Web)
|
| 21 |
+
|
| 22 |
+
### Passo 1: Acesse o Console
|
| 23 |
+
https://console.cloud.google.com/compute/instances
|
| 24 |
+
|
| 25 |
+
### Passo 2: Criar Instância Spot
|
| 26 |
+
1. Click "CREATE INSTANCE"
|
| 27 |
+
2. Configure:
|
| 28 |
+
- **Nome**: `ensemble-test`
|
| 29 |
+
- **Região**: `us-central1 (Iowa)` - Mais barato
|
| 30 |
+
- **Zona**: `us-central1-a`
|
| 31 |
+
- **Tipo de máquina**: `e2-medium` (2 vCPU, 4GB RAM)
|
| 32 |
+
- **Modelo de provisionamento**: ✅ **Spot** (Preemptible)
|
| 33 |
+
- **Disco de inicialização**:
|
| 34 |
+
- OS: `Ubuntu 22.04 LTS`
|
| 35 |
+
- Tamanho: `20 GB`
|
| 36 |
+
- **Firewall**: ✅ Permitir tráfego HTTP/HTTPS (opcional)
|
| 37 |
+
|
| 38 |
+
### Passo 3: Script de Inicialização
|
| 39 |
+
|
| 40 |
+
Na seção "Management, security, disks, networking, sole tenancy", vá em **Automation** e cole este script:
|
| 41 |
+
|
| 42 |
+
```bash
|
| 43 |
+
#!/bin/bash
|
| 44 |
+
|
| 45 |
+
# Update system
|
| 46 |
+
apt-get update
|
| 47 |
+
apt-get install -y python3-pip git
|
| 48 |
+
|
| 49 |
+
# Install dependencies
|
| 50 |
+
pip3 install --upgrade pip
|
| 51 |
+
|
| 52 |
+
# Clone repository
|
| 53 |
+
cd /home
|
| 54 |
+
git clone https://huggingface.co/marcosremar2/ensemble-tts-annotation
|
| 55 |
+
cd ensemble-tts-annotation
|
| 56 |
+
|
| 57 |
+
# Install requirements (CPU-only torch)
|
| 58 |
+
pip3 install -q torch --index-url https://download.pytorch.org/whl/cpu
|
| 59 |
+
pip3 install -q transformers datasets librosa soundfile numpy pandas tqdm scikit-learn
|
| 60 |
+
|
| 61 |
+
# Run test
|
| 62 |
+
python3 test_local.py > /tmp/test-results.log 2>&1
|
| 63 |
+
|
| 64 |
+
echo "✅ Test completed. Results in /tmp/test-results.log"
|
| 65 |
+
echo "To see results: cat /tmp/test-results.log"
|
| 66 |
+
```
|
| 67 |
+
|
| 68 |
+
### Passo 4: Criar e Aguardar
|
| 69 |
+
|
| 70 |
+
1. Click **CREATE**
|
| 71 |
+
2. Aguarde ~2 minutos para inicialização
|
| 72 |
+
3. Click no botão **SSH** na lista de instâncias
|
| 73 |
+
4. No terminal SSH, rode:
|
| 74 |
+
|
| 75 |
+
```bash
|
| 76 |
+
# Ver status do setup
|
| 77 |
+
tail -f /var/log/syslog | grep startup-script
|
| 78 |
+
|
| 79 |
+
# Ver resultado do teste
|
| 80 |
+
cat /tmp/test-results.log
|
| 81 |
+
|
| 82 |
+
# Ou rodar teste manualmente
|
| 83 |
+
cd /home/ensemble-tts-annotation
|
| 84 |
+
python3 test_local.py
|
| 85 |
+
```
|
| 86 |
+
|
| 87 |
+
### Passo 5: Deletar Instância
|
| 88 |
+
|
| 89 |
+
⚠️ **IMPORTANTE**: Não esqueça de deletar para não gerar custos!
|
| 90 |
+
|
| 91 |
+
1. Volte para Console → Compute Engine → VM Instances
|
| 92 |
+
2. Selecione `ensemble-test`
|
| 93 |
+
3. Click **DELETE**
|
| 94 |
+
|
| 95 |
+
**Custo estimado**: ~$0.003 se deletar em 20 minutos
|
| 96 |
+
|
| 97 |
+
---
|
| 98 |
+
|
| 99 |
+
## Opção 3: Usar Google Cloud Shell (Grátis!)
|
| 100 |
+
|
| 101 |
+
Google Cloud Shell já tem gcloud instalado e é **GRÁTIS**!
|
| 102 |
+
|
| 103 |
+
### Passo 1: Abrir Cloud Shell
|
| 104 |
+
https://shell.cloud.google.com/
|
| 105 |
+
|
| 106 |
+
### Passo 2: Rodar comandos
|
| 107 |
+
|
| 108 |
+
```bash
|
| 109 |
+
# Clone o script
|
| 110 |
+
curl -O https://huggingface.co/marcosremar2/ensemble-tts-annotation/raw/main/scripts/test/launch_gcp_spot.sh
|
| 111 |
+
|
| 112 |
+
# Tornar executável
|
| 113 |
+
chmod +x launch_gcp_spot.sh
|
| 114 |
+
|
| 115 |
+
# Rodar
|
| 116 |
+
./launch_gcp_spot.sh
|
| 117 |
+
```
|
| 118 |
+
|
| 119 |
+
O script vai:
|
| 120 |
+
1. ✅ Buscar a instância mais barata
|
| 121 |
+
2. ✅ Criar instância spot (e2-medium)
|
| 122 |
+
3. ✅ Instalar tudo automaticamente
|
| 123 |
+
4. ✅ Rodar os testes
|
| 124 |
+
5. ✅ Mostrar comandos para acessar e deletar
|
| 125 |
+
|
| 126 |
+
---
|
| 127 |
+
|
| 128 |
+
## Opção 4: Usar Script Python (Sem gcloud CLI)
|
| 129 |
+
|
| 130 |
+
Vou criar um script Python que usa a API do GCP diretamente...
|
| 131 |
+
|
| 132 |
+
### Passo 1: Instalar biblioteca
|
| 133 |
+
|
| 134 |
+
```bash
|
| 135 |
+
pip install google-cloud-compute
|
| 136 |
+
```
|
| 137 |
+
|
| 138 |
+
### Passo 2: Autenticar
|
| 139 |
+
|
| 140 |
+
```bash
|
| 141 |
+
# Download credentials JSON do console
|
| 142 |
+
# https://console.cloud.google.com/apis/credentials
|
| 143 |
+
|
| 144 |
+
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/credentials.json"
|
| 145 |
+
```
|
| 146 |
+
|
| 147 |
+
### Passo 3: Rodar script
|
| 148 |
+
|
| 149 |
+
```bash
|
| 150 |
+
python scripts/test/launch_gcp_python.py
|
| 151 |
+
```
|
| 152 |
+
|
| 153 |
+
---
|
| 154 |
+
|
| 155 |
+
## Resumo de Custos
|
| 156 |
+
|
| 157 |
+
| Opção | Custo | Facilidade |
|
| 158 |
+
|-------|-------|------------|
|
| 159 |
+
| Cloud Shell | $0 (grátis!) | ⭐⭐⭐⭐⭐ Mais fácil |
|
| 160 |
+
| Console Web | ~$0.003 | ⭐⭐⭐⭐ Fácil |
|
| 161 |
+
| gcloud CLI local | ~$0.003 | ⭐⭐⭐ Médio |
|
| 162 |
+
| Python API | ~$0.003 | ⭐⭐ Avançado |
|
| 163 |
+
|
| 164 |
+
---
|
| 165 |
+
|
| 166 |
+
## Recomendação
|
| 167 |
+
|
| 168 |
+
**Melhor opção**: Use **Google Cloud Shell** (Opção 3)
|
| 169 |
+
- ✅ Grátis
|
| 170 |
+
- ✅ Já tem gcloud instalado
|
| 171 |
+
- ✅ Não precisa instalar nada local
|
| 172 |
+
- ✅ Script funciona direto
|
| 173 |
+
|
| 174 |
+
```bash
|
| 175 |
+
# 1. Abrir https://shell.cloud.google.com/
|
| 176 |
+
# 2. Rodar:
|
| 177 |
+
curl -O https://huggingface.co/marcosremar2/ensemble-tts-annotation/raw/main/scripts/test/launch_gcp_spot.sh
|
| 178 |
+
chmod +x launch_gcp_spot.sh
|
| 179 |
+
./launch_gcp_spot.sh
|
| 180 |
+
```
|
| 181 |
+
|
| 182 |
+
Qual opção você prefere? Posso criar o script Python (Opção 4) se preferir não instalar gcloud.
|
|
@@ -0,0 +1,255 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""
|
| 3 |
+
Launch GCP spot instance using Python API (no gcloud CLI needed).
|
| 4 |
+
|
| 5 |
+
Requirements:
|
| 6 |
+
pip install google-cloud-compute google-auth
|
| 7 |
+
|
| 8 |
+
Setup:
|
| 9 |
+
1. Create service account: https://console.cloud.google.com/iam-admin/serviceaccounts
|
| 10 |
+
2. Download JSON key
|
| 11 |
+
3. Set environment variable:
|
| 12 |
+
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/key.json"
|
| 13 |
+
|
| 14 |
+
Or use: gcloud auth application-default login
|
| 15 |
+
"""
|
| 16 |
+
|
| 17 |
+
import os
|
| 18 |
+
import sys
|
| 19 |
+
import time
|
| 20 |
+
from google.cloud import compute_v1
|
| 21 |
+
from google.oauth2 import service_account
|
| 22 |
+
import logging
|
| 23 |
+
|
| 24 |
+
logging.basicConfig(level=logging.INFO, format='%(message)s')
|
| 25 |
+
logger = logging.getLogger(__name__)
|
| 26 |
+
|
| 27 |
+
# Configuration
|
| 28 |
+
PROJECT_ID = os.getenv('GCP_PROJECT_ID', None)
|
| 29 |
+
ZONE = 'us-central1-a'
|
| 30 |
+
INSTANCE_NAME = f'ensemble-test-{int(time.time())}'
|
| 31 |
+
MACHINE_TYPE = 'e2-medium'
|
| 32 |
+
IMAGE_FAMILY = 'ubuntu-2204-lts'
|
| 33 |
+
IMAGE_PROJECT = 'ubuntu-os-cloud'
|
| 34 |
+
|
| 35 |
+
STARTUP_SCRIPT = """#!/bin/bash
|
| 36 |
+
apt-get update
|
| 37 |
+
apt-get install -y python3-pip git
|
| 38 |
+
|
| 39 |
+
pip3 install --upgrade pip
|
| 40 |
+
|
| 41 |
+
cd /home
|
| 42 |
+
git clone https://huggingface.co/marcosremar2/ensemble-tts-annotation
|
| 43 |
+
cd ensemble-tts-annotation
|
| 44 |
+
|
| 45 |
+
pip3 install -q torch --index-url https://download.pytorch.org/whl/cpu
|
| 46 |
+
pip3 install -q transformers datasets librosa soundfile numpy pandas tqdm scikit-learn
|
| 47 |
+
|
| 48 |
+
python3 test_local.py > /tmp/test-results.log 2>&1
|
| 49 |
+
|
| 50 |
+
echo "✅ Test completed" >> /tmp/test-results.log
|
| 51 |
+
"""
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
def get_credentials():
|
| 55 |
+
"""Get GCP credentials."""
|
| 56 |
+
creds_path = os.getenv('GOOGLE_APPLICATION_CREDENTIALS')
|
| 57 |
+
|
| 58 |
+
if creds_path and os.path.exists(creds_path):
|
| 59 |
+
logger.info(f"Using credentials from: {creds_path}")
|
| 60 |
+
return service_account.Credentials.from_service_account_file(creds_path)
|
| 61 |
+
else:
|
| 62 |
+
logger.info("Using default credentials (gcloud auth application-default login)")
|
| 63 |
+
from google.auth import default
|
| 64 |
+
credentials, project = default()
|
| 65 |
+
return credentials
|
| 66 |
+
|
| 67 |
+
|
| 68 |
+
def get_project_id():
|
| 69 |
+
"""Get GCP project ID."""
|
| 70 |
+
if PROJECT_ID:
|
| 71 |
+
return PROJECT_ID
|
| 72 |
+
|
| 73 |
+
# Try to get from gcloud config
|
| 74 |
+
try:
|
| 75 |
+
import subprocess
|
| 76 |
+
result = subprocess.run(
|
| 77 |
+
['gcloud', 'config', 'get-value', 'project'],
|
| 78 |
+
capture_output=True,
|
| 79 |
+
text=True
|
| 80 |
+
)
|
| 81 |
+
if result.returncode == 0:
|
| 82 |
+
project = result.stdout.strip()
|
| 83 |
+
if project:
|
| 84 |
+
return project
|
| 85 |
+
except:
|
| 86 |
+
pass
|
| 87 |
+
|
| 88 |
+
# Ask user
|
| 89 |
+
project = input("Enter GCP Project ID: ").strip()
|
| 90 |
+
return project
|
| 91 |
+
|
| 92 |
+
|
| 93 |
+
def create_instance(project_id, zone, instance_name, machine_type):
|
| 94 |
+
"""Create spot instance."""
|
| 95 |
+
logger.info("=" * 60)
|
| 96 |
+
logger.info("Creating GCP Spot Instance")
|
| 97 |
+
logger.info("=" * 60)
|
| 98 |
+
logger.info(f"Project: {project_id}")
|
| 99 |
+
logger.info(f"Zone: {zone}")
|
| 100 |
+
logger.info(f"Instance: {instance_name}")
|
| 101 |
+
logger.info(f"Machine Type: {machine_type}")
|
| 102 |
+
logger.info(f"Estimated cost: ~$0.01/hr")
|
| 103 |
+
logger.info("")
|
| 104 |
+
|
| 105 |
+
# Get credentials
|
| 106 |
+
credentials = get_credentials()
|
| 107 |
+
|
| 108 |
+
# Create compute client
|
| 109 |
+
instance_client = compute_v1.InstancesClient(credentials=credentials)
|
| 110 |
+
|
| 111 |
+
# Get image
|
| 112 |
+
logger.info("Getting Ubuntu image...")
|
| 113 |
+
image_client = compute_v1.ImagesClient(credentials=credentials)
|
| 114 |
+
image = image_client.get_from_family(
|
| 115 |
+
project=IMAGE_PROJECT,
|
| 116 |
+
family=IMAGE_FAMILY
|
| 117 |
+
)
|
| 118 |
+
|
| 119 |
+
# Configure instance
|
| 120 |
+
logger.info("Configuring instance...")
|
| 121 |
+
|
| 122 |
+
disk = compute_v1.AttachedDisk()
|
| 123 |
+
disk.boot = True
|
| 124 |
+
disk.auto_delete = True
|
| 125 |
+
disk.initialize_params = compute_v1.AttachedDiskInitializeParams()
|
| 126 |
+
disk.initialize_params.source_image = image.self_link
|
| 127 |
+
disk.initialize_params.disk_size_gb = 20
|
| 128 |
+
|
| 129 |
+
network_interface = compute_v1.NetworkInterface()
|
| 130 |
+
network_interface.name = "global/networks/default"
|
| 131 |
+
access_config = compute_v1.AccessConfig()
|
| 132 |
+
access_config.name = "External NAT"
|
| 133 |
+
access_config.type_ = "ONE_TO_ONE_NAT"
|
| 134 |
+
network_interface.access_configs = [access_config]
|
| 135 |
+
|
| 136 |
+
metadata = compute_v1.Metadata()
|
| 137 |
+
metadata.items = [
|
| 138 |
+
compute_v1.Items(key="startup-script", value=STARTUP_SCRIPT)
|
| 139 |
+
]
|
| 140 |
+
|
| 141 |
+
scheduling = compute_v1.Scheduling()
|
| 142 |
+
scheduling.preemptible = True
|
| 143 |
+
scheduling.automatic_restart = False
|
| 144 |
+
scheduling.on_host_maintenance = "TERMINATE"
|
| 145 |
+
|
| 146 |
+
instance = compute_v1.Instance()
|
| 147 |
+
instance.name = instance_name
|
| 148 |
+
instance.machine_type = f"zones/{zone}/machineTypes/{machine_type}"
|
| 149 |
+
instance.disks = [disk]
|
| 150 |
+
instance.network_interfaces = [network_interface]
|
| 151 |
+
instance.metadata = metadata
|
| 152 |
+
instance.scheduling = scheduling
|
| 153 |
+
|
| 154 |
+
# Create instance
|
| 155 |
+
logger.info("Launching instance...")
|
| 156 |
+
operation = instance_client.insert(
|
| 157 |
+
project=project_id,
|
| 158 |
+
zone=zone,
|
| 159 |
+
instance_resource=instance
|
| 160 |
+
)
|
| 161 |
+
|
| 162 |
+
# Wait for operation to complete
|
| 163 |
+
logger.info("Waiting for instance to be created...")
|
| 164 |
+
while operation.status != compute_v1.Operation.Status.DONE:
|
| 165 |
+
time.sleep(2)
|
| 166 |
+
operation = compute_v1.ZoneOperationsClient(credentials=credentials).get(
|
| 167 |
+
project=project_id,
|
| 168 |
+
zone=zone,
|
| 169 |
+
operation=operation.name
|
| 170 |
+
)
|
| 171 |
+
|
| 172 |
+
if operation.error:
|
| 173 |
+
logger.error(f"Error creating instance: {operation.error}")
|
| 174 |
+
return None
|
| 175 |
+
|
| 176 |
+
# Get instance details
|
| 177 |
+
instance = instance_client.get(
|
| 178 |
+
project=project_id,
|
| 179 |
+
zone=zone,
|
| 180 |
+
instance=instance_name
|
| 181 |
+
)
|
| 182 |
+
|
| 183 |
+
external_ip = instance.network_interfaces[0].access_configs[0].nat_i_p
|
| 184 |
+
|
| 185 |
+
logger.info("")
|
| 186 |
+
logger.info("=" * 60)
|
| 187 |
+
logger.info("✅ Instance created successfully!")
|
| 188 |
+
logger.info("=" * 60)
|
| 189 |
+
logger.info(f"Instance Name: {instance_name}")
|
| 190 |
+
logger.info(f"External IP: {external_ip}")
|
| 191 |
+
logger.info(f"Zone: {zone}")
|
| 192 |
+
logger.info("")
|
| 193 |
+
logger.info("SSH Command:")
|
| 194 |
+
logger.info(f" gcloud compute ssh {instance_name} --zone={zone} --project={project_id}")
|
| 195 |
+
logger.info("")
|
| 196 |
+
logger.info("Or from console:")
|
| 197 |
+
logger.info(f" https://console.cloud.google.com/compute/instances?project={project_id}")
|
| 198 |
+
logger.info("")
|
| 199 |
+
logger.info("Check test results (wait ~2 min for setup):")
|
| 200 |
+
logger.info(f" gcloud compute ssh {instance_name} --zone={zone} --command='cat /tmp/test-results.log'")
|
| 201 |
+
logger.info("")
|
| 202 |
+
logger.info("Delete instance:")
|
| 203 |
+
logger.info(f" gcloud compute instances delete {instance_name} --zone={zone} --project={project_id} --quiet")
|
| 204 |
+
logger.info("")
|
| 205 |
+
logger.info("=" * 60)
|
| 206 |
+
logger.info("⚠️ Remember to delete the instance to avoid charges!")
|
| 207 |
+
logger.info("=" * 60)
|
| 208 |
+
|
| 209 |
+
return instance_name
|
| 210 |
+
|
| 211 |
+
|
| 212 |
+
def main():
|
| 213 |
+
"""Main function."""
|
| 214 |
+
logger.info("GCP Spot Instance Launcher (Python API)")
|
| 215 |
+
logger.info("")
|
| 216 |
+
|
| 217 |
+
# Get project
|
| 218 |
+
project_id = get_project_id()
|
| 219 |
+
if not project_id:
|
| 220 |
+
logger.error("No project ID provided")
|
| 221 |
+
return 1
|
| 222 |
+
|
| 223 |
+
# Create instance
|
| 224 |
+
try:
|
| 225 |
+
instance_name = create_instance(
|
| 226 |
+
project_id=project_id,
|
| 227 |
+
zone=ZONE,
|
| 228 |
+
instance_name=INSTANCE_NAME,
|
| 229 |
+
machine_type=MACHINE_TYPE
|
| 230 |
+
)
|
| 231 |
+
|
| 232 |
+
if instance_name:
|
| 233 |
+
logger.info("\n✅ Success! Instance is launching.")
|
| 234 |
+
logger.info("\nNext steps:")
|
| 235 |
+
logger.info("1. Wait ~2 minutes for setup to complete")
|
| 236 |
+
logger.info("2. SSH to instance and check results")
|
| 237 |
+
logger.info("3. Delete instance when done")
|
| 238 |
+
return 0
|
| 239 |
+
else:
|
| 240 |
+
logger.error("\n❌ Failed to create instance")
|
| 241 |
+
return 1
|
| 242 |
+
|
| 243 |
+
except Exception as e:
|
| 244 |
+
logger.error(f"\n❌ Error: {e}")
|
| 245 |
+
logger.error("\nMake sure you have:")
|
| 246 |
+
logger.error("1. Installed: pip install google-cloud-compute")
|
| 247 |
+
logger.error("2. Set credentials:")
|
| 248 |
+
logger.error(" - Option A: export GOOGLE_APPLICATION_CREDENTIALS='/path/to/key.json'")
|
| 249 |
+
logger.error(" - Option B: gcloud auth application-default login")
|
| 250 |
+
logger.error("3. Enabled Compute Engine API in your project")
|
| 251 |
+
return 1
|
| 252 |
+
|
| 253 |
+
|
| 254 |
+
if __name__ == "__main__":
|
| 255 |
+
sys.exit(main())
|