---
title: Jarvis Mobile Agent
emoji: 🤖
colorFrom: blue
colorTo: purple
sdk: docker
pinned: false
---

# 🤖 Jarvis — AI Automation Planner for Mobile Devices

Jarvis is an intelligent AI automation planner that converts user voice/text commands into structured JSON task plans for mobile device automation.

## Architecture

```
Mobile Application
      ↓
HTTP POST → /v1/chat/completions
      ↓
Jarvis AI (Reason → Plan → Output)
      ↓
JSON automation plan returned
      ↓
Mobile automation engine executes locally
```

## Project Structure

```
AI-Agent/
├── app.py                  # Main app — Gradio UI + FastAPI endpoints
├── jarvis/
│   ├── __init__.py
│   ├── agent.py            # Core agent (Reason → Plan → Execute)
│   ├── tools.py            # Dynamic tool registry
│   ├── security.py         # Restricted-action enforcement
│   └── prompts.py          # System prompt builder
├── tool_registry.json      # Tool definitions (editable without code changes)
├── requirements.txt
├── Dockerfile
└── README.md
```

## API Endpoint

### `POST /v1/chat/completions`

OpenAI-compatible chat completions endpoint.

**Request:**
```json
{
  "messages": [
    { "role": "user", "content": "Download a galaxy image and send it to Arun" }
  ]
}
```

**Response:**
```json
{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "model": "Qwen/Qwen2-1.5B-Instruct",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "{\"steps\":[{\"tool\":\"download_image\",\"query\":\"galaxy\"},{\"tool\":\"send_message\",\"contact\":\"Arun\",\"message\":\"Sending the galaxy image\"}]}"
      },
      "finish_reason": "stop"
    }
  ]
}
```

### `GET /health`

Returns service status and available tools.

### `GET /v1/models`

Lists the currently loaded model.

## Supported Tools

| Tool | Description | Parameters |
|------|-------------|------------|
| `open_app` | Open a device application | `app_name` |
| `search_google` | Internet search | `query` |
| `download_image` | Download an image | `query` |
| `download_video` | Download a video | `url` |
| `send_message` | Send a message to a contact | `contact`, `message` |
| `product_search` | Search e-commerce platforms | `query` |
| `set_alarm` | Create alarm/reminder | `time` |
| `save_file` | Download and store a file | `url` |

New tools can be added by editing `tool_registry.json` — no code changes needed.

## Security Restrictions

Jarvis refuses to automate:
- Banking / payment apps (GPay, PhonePe, Paytm, etc.)
- Financial transactions
- Password managers
- Personal identity data (Aadhaar, PAN, SSN, etc.)

Restricted requests return:
```json
{ "error": "This action is restricted for security reasons." }
```

## Deployment to Hugging Face Spaces

### Option A — Gradio SDK (recommended)

1. Create a new Space at [huggingface.co/new-space](https://huggingface.co/new-space)
2. Choose **SDK: Gradio**
3. Push this repository to the Space
4. Add your `HF_TOKEN` as a Space secret (Settings → Secrets)
5. The Space will be available at:
   ```
   POST https://<your-space>.hf.space/v1/chat/completions
   ```

### Option B — Docker SDK

1. Create a new Space with **SDK: Docker**
2. Push this repository (the `Dockerfile` is included)
3. Add `HF_TOKEN` as a Space secret

## Environment Variables

| Variable | Default | Description |
|----------|---------|-------------|
| `JARVIS_MODEL` | `Qwen/Qwen2-1.5B-Instruct` | HF model ID for the planner LLM |
| `HF_TOKEN` | *(none)* | Hugging Face API token for gated models |
| `PORT` | `7860` | Server port |

## Local Development

```bash
pip install -r requirements.txt
export HF_TOKEN="hf_..."
python app.py
```

The Gradio UI will be available at `http://localhost:7860` and the API at `http://localhost:7860/v1/chat/completions`.

## Mobile Integration (Android Example)

```kotlin
val client = OkHttpClient()
val json = """{"messages":[{"role":"user","content":"$userCommand"}]}"""
val body = json.toRequestBody("application/json".toMediaType())

val request = Request.Builder()
    .url("https://Valtry-mobile-agent.hf.space/v1/chat/completions")
    .post(body)
    .build()

client.newCall(request).enqueue(object : Callback {
    override fun onResponse(call: Call, response: Response) {
        val plan = JSONObject(response.body!!.string())
            .getJSONArray("choices")
            .getJSONObject(0)
            .getJSONObject("message")
            .getString("content")
        // Parse and execute the automation plan
    }
    override fun onFailure(call: Call, e: IOException) { /* handle */ }
})
```