Fix: Handle function-call style

README.md

@@ -10,11 +10,14 @@ pinned: false
 
 # Bielik App Service
 
-Multi-model LLM service for description enhancement and A/B testing.
+Multi-model LLM service for description enhancement, batch gap-filling, and A/B testing.
 
 ## Overview
 
-This service provides an API for generating enhanced descriptions using multiple open-source LLMs. It supports comparing outputs across different models to evaluate quality, speed, and Polish language support.
+This service provides an API for generating enhanced descriptions using multiple open-source LLMs. It supports:
+- **Description Enhancement**: Generate marketing descriptions from structured data
+- **Batch Infill**: Fill gaps (`[GAP:n]` or `___`) in ad texts with natural words
+- **Multi-Model Comparison**: Compare outputs across different models for A/B testing
 
 ## Models
 
@@ -42,13 +45,20 @@ This service provides an API for generating enhanced descriptions using multiple
 | `POST` | `/models/{name}/load` | Load a model into memory |
 | `POST` | `/models/{name}/unload` | Unload a model from memory |
 
-### Generation
+### Description Generation
 
 | Method | Endpoint | Description |
 |--------|----------|-------------|
 | `POST` | `/enhance-description` | Generate description with single model |
 | `POST` | `/compare` | Compare outputs from multiple models |
 
+### Batch Infill (Gap-Filling)
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| `POST` | `/infill` | Batch gap-filling with single model |
+| `POST` | `/compare-infill` | Compare gap-filling across multiple models |
+
 ---
 
 ## Lazy Loading
@@ -227,6 +237,137 @@ Compare outputs from multiple models for the same input.
 
 ---
 
+### `POST /infill`
+
+Batch gap-filling for ads using a single model. Accepts texts with `[GAP:n]` markers or `___` and returns filled text with per-gap choices and alternatives.
+
+**Gap Notation:**
+- `[GAP:1]`, `[GAP:2]`, ... → Explicit numbered gaps (preferred)
+- `___` → Auto-numbered in scan order
+
+**Request:**
+```json
+{
+  "domain": "cars",
+  "items": [
+    {
+      "id": "ad1",
+      "text_with_gaps": "Sprzedam [GAP:1] BMW w [GAP:2] stanie technicznym"
+    },
+    {
+      "id": "ad2", 
+      "text_with_gaps": "Auto ma ___ km przebiegu i ___ lakier"
+    }
+  ],
+  "model": "bielik-1.5b",
+  "options": {
+    "top_n_per_gap": 3,
+    "language": "pl",
+    "temperature": 0.6
+  }
+}
+```
+
+**Response:**
+```json
+{
+  "model": "bielik-1.5b",
+  "results": [
+    {
+      "id": "ad1",
+      "status": "ok",
+      "filled_text": "Sprzedam eleganckie BMW w doskonałym stanie technicznym",
+      "gaps": [
+        {
+          "index": 1,
+          "marker": "[GAP:1]",
+          "choice": "eleganckie",
+          "alternatives": ["piękne", "zadbane"]
+        },
+        {
+          "index": 2,
+          "marker": "[GAP:2]",
+          "choice": "doskonałym",
+          "alternatives": ["bardzo dobrym", "idealnym"]
+        }
+      ],
+      "error": null
+    }
+  ],
+  "total_time": 3.45,
+  "processed_count": 2,
+  "error_count": 0
+}
+```
+
+**Options:**
+| Field | Type | Default | Description |
+|-------|------|---------|-------------|
+| `gap_notation` | string | `"auto"` | `"auto"`, `"[GAP:n]"`, or `"___"` |
+| `top_n_per_gap` | int | `3` | Alternatives per gap (1-5) |
+| `language` | string | `"pl"` | Output language |
+| `temperature` | float | `0.6` | Generation temperature (0-1) |
+| `max_new_tokens` | int | `256` | Max tokens to generate |
+
+---
+
+### `POST /compare-infill`
+
+Multi-model batch gap-filling comparison for A/B testing.
+
+**Request:**
+```json
+{
+  "domain": "cars",
+  "items": [
+    {
+      "id": "ad1",
+      "text_with_gaps": "Sprzedam [GAP:1] BMW w [GAP:2] stanie"
+    }
+  ],
+  "models": ["bielik-1.5b", "qwen2.5-3b", "pllum-12b"],
+  "options": {
+    "top_n_per_gap": 3
+  }
+}
+```
+
+**Response:**
+```json
+{
+  "domain": "cars",
+  "models": [
+    {
+      "model": "bielik-1.5b",
+      "type": "local",
+      "results": [...],
+      "time": 2.1,
+      "error_count": 0
+    },
+    {
+      "model": "qwen2.5-3b",
+      "type": "local",
+      "results": [...],
+      "time": 1.8,
+      "error_count": 0
+    }
+  ],
+  "total_time": 5.2
+}
+```
+
+---
+
+## Domains
+
+Currently supported domains:
+
+| Domain | Schema Fields |
+|--------|---------------|
+| `cars` | `make`, `model`, `year`, `mileage`, `features[]`, `condition` |
+
+---
+
 ## Environment Variables
 
 | Variable | Description | Required |
@@ -255,3 +396,18 @@ uvicorn app.main:app --reload --port 8000
 API available at `http://localhost:8000`
 
 Docs at `http://localhost:8000/docs`
+
+## Live Demo
+
+Deployed on HuggingFace Spaces:
+
+**URL:** `https://studzinsky-bielik-app-service.hf.space`
+
+**Quick Test:**
+```bash
+# Health check
+curl https://studzinsky-bielik-app-service.hf.space/health
+
+# List models
+curl https://studzinsky-bielik-app-service.hf.space/models
+```

app/logic/infill_utils.py

@@ -96,6 +96,7 @@ def parse_infill_json(raw_output: str) -> Optional[dict]:
     Handles common LLM quirks:
     - JSON wrapped in markdown code blocks
     - Leading/trailing text before/after JSON
+    - Function-call style wrapper ({"name": "...", "arguments": {...}})
     - Minor formatting issues
     
     Args:
@@ -147,6 +148,15 @@ def parse_infill_json(raw_output: str) -> Optional[dict]:
     try:
         parsed = json.loads(json_str)
         
+        # Handle function-call style wrapper:
+        # {"name": "filled_text", "arguments": {"filled_text": "...", "gaps": [...]}}
+        if 'arguments' in parsed and isinstance(parsed['arguments'], dict):
+            parsed = parsed['arguments']
+        
+        # Also handle: {"name": "...", "parameters": {...}}
+        if 'parameters' in parsed and isinstance(parsed['parameters'], dict):
+            parsed = parsed['parameters']
+        
         # Validate required fields
         if 'filled_text' not in parsed and 'gaps' not in parsed:
             return None