Sync backend with LN-TTS: fix Chapter 1 scraper bug, add sentence offsets, improve error handling, update README

README.md

@@ -1,76 +1,49 @@
 ---
 title: CoreReader
-emoji: 📉
-colorFrom: yellow
-colorTo: yellow
+emoji: 📖
+colorFrom: blue
+colorTo: purple
 sdk: docker
 pinned: false
 ---
 
-# CoreReader Backend (Docker)
+# CoreReader Backend
 
-This repository is a backend-only deployment target for CoreReader / LN-TTS.
+A FastAPI backend for [CoreReader](https://github.com/user/LN-TTS) — a light-novel reader with AI text-to-speech.
 
-It runs a FastAPI server that:
-- Scrapes NovelCool chapters + chapter index
-- Runs Kokoro ONNX TTS (CPU)
-- Streams **sentence-atomic** PCM16 mono audio over WebSocket
-	- one binary WebSocket message per sentence (with a short trailing pause)
-	- sentence audio includes a tiny fade-in/out to avoid boundary clicks
+This Space runs the backend server that:
+- Scrapes novel chapters from NovelCool
+- Synthesizes speech using [Kokoro ONNX](https://github.com/thewh1teagle/kokoro-onnx) TTS (50+ English voices, CPU)
+- Streams sentence-atomic PCM16 audio over WebSocket
 
-## Endpoints
+## Use from the Flutter App
 
-- GET /health
-- GET /voices
-- GET /novel_index?url=...
-- GET /novel_details?url=... (best-effort cover URL)
-- GET /novel_meta?url=...
-- GET /novel_chapter?url=...&n=...
-- WS /ws
+1. Open the CoreReader app.
+2. Go to **Settings** → **WebSocket base URL**.
+3. Paste your Space URL:
 
-## Use from the Flutter app
+   ```
+   wss://<your-space>.hf.space
+   ```
 
-In Settings → WebSocket base URL:
+The app connects to `/ws` automatically.
 
-- Hugging Face Spaces URL: wss://<space-subdomain>.hf.space
+## API Endpoints
 
-The app connects to: wss://<space-subdomain>.hf.space/ws
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/health` | Server status |
+| GET | `/voices` | Available TTS voices |
+| GET | `/novel_index?url=...` | Chapter list for a novel |
+| GET | `/novel_details?url=...` | Novel cover URL (best-effort) |
+| GET | `/novel_meta?url=...` | Chapter count |
+| GET | `/novel_chapter?url=...&n=...` | Resolve chapter by number |
+| WS | `/ws` | Audio streaming WebSocket |
+| GET | `/docs` | Interactive API documentation |
+| GET | `/info` | Runtime status (JSON) |
 
 ## Notes
 
-- The container downloads models on startup via download_models.py.
-- Offline downloads in the app use WS play with realtime=false so synthesis runs faster than real-time.
-
-Protocol note:
-- WS `chapter_info` includes `sentence_total` (best-effort) so the client can render accurate download progress rings.
-- `/novel_index` chapter numbers are parsed from title or URL (more robust for Chapter 1 / Prologue edge-cases).
-
-## Deploy to Azure (Container Apps)
-
-This Docker image can be deployed to Azure Container Apps.
-
-1) Create a resource group + registry:
-
-- az group create -n corereader-rg -l westeurope
-- az acr create -n <acrName> -g corereader-rg --sku Basic
-
-2) Build + push to ACR:
-
-- az acr build -r <acrName> -t corereader-backend:v1 .
-
-3) Deploy a public Container App (binds to PORT, default 7860):
-
-- az extension add --name containerapp --upgrade
-- az containerapp env create -g corereader-rg -n corereader-env -l westeurope
-- loginServer=$(az acr show -n <acrName> -g corereader-rg --query loginServer -o tsv)
-- az containerapp create -g corereader-rg -n corereader-backend --environment corereader-env \
-	--image "$loginServer/corereader-backend:v1" \
-	--ingress external --target-port 7860 --registry-server "$loginServer"
-
-4) Get the URL:
-
-- fqdn=$(az containerapp show -g corereader-rg -n corereader-backend --query properties.configuration.ingress.fqdn -o tsv)
-
-Paste into the Flutter app Settings:
-
-- wss://$fqdn
+- Models are downloaded on first boot (~30s).
+- Free Spaces sleep after ~15 min of inactivity. First request after sleeping takes 30–60s.
+- The WS protocol supports `realtime: false` for offline chapter downloads.

backend/scraper.py

@@ -111,6 +111,7 @@ class NovelCoolScraper:
 
         def parse_chapter_number(title: str, url: str) -> int | None:
             t = (title or '').strip()
+            # Best-effort chapter number parsing from visible text.
             m = re.search(r"(?:\bChapter\b|\bCh\.?\b|\bC\b)\s*(\d+)", t, flags=re.IGNORECASE)
             if m:
                 try:
@@ -119,6 +120,8 @@ class NovelCoolScraper:
                 except Exception:
                     pass
 
+            # Fallback: parse from URL, e.g.
+            # /chapter/<Novel>-Chapter-15/<id>/ or .../Chapter_15/... etc.
             u = (url or '')
             m = re.search(r"(?:chapter|ch)[^0-9]{0,12}(\d+)", u, flags=re.IGNORECASE)
             if m:
@@ -138,15 +141,17 @@ class NovelCoolScraper:
             abs_url = urljoin(novel_url, href)
             if abs_url in seen:
                 continue
-            seen.add(abs_url)
             title = a.get_text(' ', strip=True)
             if not title:
-                # Some chapter links have empty text (icons). Skip.
+                # Some chapter links have empty text (icons). Skip but do NOT
+                # mark as seen — the real link with text may appear later.
                 continue
+            seen.add(abs_url)
             n = parse_chapter_number(title, abs_url)
             links.append({"n": n, "title": title, "url": abs_url})
 
-        # Sort by chapter number when possible, but preserve stable ordering for unknowns.
+        # Sort by chapter number when possible, but preserve stable ordering
+        # for unknowns (avoid pushing an unparsed Chapter 1 to the end).
         def chapter_key(item):
             n = item.get('n')
             if isinstance(n, int):
@@ -159,7 +164,7 @@ class NovelCoolScraper:
     async def scrape_novel_details(self, novel_url: str):
         """Scrape a NovelCool novel page and return lightweight metadata.
 
-        Returns:
+        Currently returns:
         - title: best-effort title
         - cover_url: absolute URL to the cover image, when detectable
         """

backend/server.py

@@ -52,6 +52,10 @@ app.add_middleware(
     allow_headers=["*"],
 )
 
+# ---------------------------------------------------------------------------
+# HuggingFace Space landing page & info endpoint
+# ---------------------------------------------------------------------------
+
 def _space_runtime_info(request: Request) -> dict:
     tts = getattr(app.state, "tts", None)
     voices = []
@@ -125,7 +129,7 @@ async def root(request: Request):
             <li><code>/health</code></li>
             <li><code>/voices</code></li>
             <li><code>/novel_index?url=&lt;novel_url&gt;</code></li>
-                        <li><code>/novel_details?url=&lt;novel_url&gt;</code></li>
+            <li><code>/novel_details?url=&lt;novel_url&gt;</code></li>
             <li><code>/novel_meta?url=&lt;novel_url&gt;</code></li>
             <li><code>/novel_chapter?url=&lt;novel_url&gt;&amp;n=&lt;chapter_number&gt;</code></li>
             <li><code>/ws</code> (WebSocket)</li>
@@ -366,6 +370,8 @@ async def websocket_endpoint(websocket: WebSocket):
                                 "encoding": "pcm_s16le",
                                 "sample_rate": app.state.tts.sample_rate,
                                 "channels": 1,
+                                # For backward-compatibility, keep frame_ms but note that
+                                # the stream is now sentence-chunked.
                                 "frame_ms": frame_ms,
                                 "chunking": "sentence",
                             },
@@ -373,9 +379,6 @@ async def websocket_endpoint(websocket: WebSocket):
                     )
 
                     last_key = None
-                    # Cumulative samples sent so far — used to stamp ms_start on each
-                    # sentence event so the client can fire highlights at the right
-                    # playback position rather than at message-arrival time.
                     cumulative_samples = 0
                     sample_rate = app.state.tts.sample_rate
                     try:
@@ -397,7 +400,7 @@ async def websocket_endpoint(websocket: WebSocket):
                             elif cmd == "stop":
                                 cancel_event.set()
 
-                        async for p_idx, s_idx, sentence, audio_chunk in app.state.tts.generate_audio_stream_paragraphs_sentence_chunks(
+                        async for p_idx, s_idx, sentence, audio_chunk, cs, ce in app.state.tts.generate_audio_stream_paragraphs_sentence_chunks(
                             paragraphs_slice,
                             voice=voice,
                             speed=speed,
@@ -434,8 +437,6 @@ async def websocket_endpoint(websocket: WebSocket):
                             key = (p_idx + start_paragraph, s_idx, sentence)
                             if key != last_key:
                                 last_key = key
-                                # ms_start lets the client fire this highlight exactly when
-                                # the audio reaches this sentence, regardless of buffering.
                                 ms_start = (cumulative_samples * 1000) // sample_rate
                                 await websocket.send_json(
                                     {
@@ -444,16 +445,25 @@ async def websocket_endpoint(websocket: WebSocket):
                                         "paragraph_index": int(p_idx + start_paragraph),
                                         "sentence_index": int(s_idx),
                                         "ms_start": ms_start,
+                                        "char_start": int(cs),
+                                        "char_end": int(ce),
+                                        # Size of the *next* binary message for this sentence in samples/bytes.
+                                        # Helps clients associate metadata with audio even if transport splits chunks.
+                                        "chunk_samples": int(len(audio_chunk) // 2),
+                                        "chunk_bytes": int(len(audio_chunk)),
                                     }
                                 )
                             await websocket.send_bytes(audio_chunk)
-                            # Track cumulative audio sent (int16 = 2 bytes per sample).
                             cumulative_samples += len(audio_chunk) // 2
 
                             # Optional realtime pacing.
+                            # - streaming: send roughly in-time to reduce client buffer bloat.
+                            # - downloads: realtime=false sends as fast as synthesis allows.
                             if realtime:
                                 expected_s = cumulative_samples / float(sample_rate)
                                 elapsed_s = time.monotonic() - stream_t0
+                                # Let the stream run slightly ahead to avoid stutter from
+                                # small scheduling/network jitter.
                                 ahead_s = 0.10
                                 sleep_s = (expected_s - elapsed_s) - ahead_s
                                 if sleep_s > 0:
@@ -471,17 +481,26 @@ async def websocket_endpoint(websocket: WebSocket):
                         )
                     except Exception as e:
                         logger.error(f"Play stream error: {e}")
-                        await websocket.send_json({"type": "error", "message": str(e)})
+                        try:
+                            await websocket.send_json({"type": "error", "message": str(e)})
+                        except Exception:
+                            pass  # Client already disconnected
                 
                 else:
                     await websocket.send_json({"error": "Unknown command"})
             
             except json.JSONDecodeError:
-                await websocket.send_json({"error": "Invalid JSON"})
+                try:
+                    await websocket.send_json({"error": "Invalid JSON"})
+                except Exception:
+                    pass
             except Exception as e:
                 logger.error(f"Error processing message: {e}")
                 traceback.print_exc()
-                await websocket.send_json({"error": "Internal server error"})
+                try:
+                    await websocket.send_json({"error": "Internal server error"})
+                except Exception:
+                    pass
 
     except WebSocketDisconnect:
         logger.info("Client disconnected")

backend/tts.py

@@ -17,6 +17,20 @@ class TTSEngine:
         model_path: str = "models/kokoro-v1.0.onnx",
         voices_path: str = "models/voices-v1.0.bin",
     ):
+        # Resolve relative paths against this backend module directory, not the
+        # process working directory (important for serverless/ASGI hosts).
+        base_dir = Path(__file__).resolve().parent
+        mp = Path(model_path)
+        if not mp.is_absolute():
+            candidate = (base_dir / mp).resolve()
+            if candidate.exists():
+                model_path = str(candidate)
+        vp = Path(voices_path)
+        if not vp.is_absolute():
+            candidate = (base_dir / vp).resolve()
+            if candidate.exists():
+                voices_path = str(candidate)
+
         # Ensure models exist
         if not os.path.exists(model_path):
             raise FileNotFoundError(f"Model not found at {model_path}. Run download_models.py first.")
@@ -34,10 +48,38 @@ class TTSEngine:
         # CPU-only mode for maximum compatibility.
         self.providers = ["CPUExecutionProvider"]
 
+        # ONNX Runtime performance tuning (CPU).
+        # Keep defaults conservative; allow override via env for deployments.
+        sess_options = None
+        try:
+            sess_options = ort.SessionOptions()
+            sess_options.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL
+            # Thread counts: 0 means ORT will choose (often = physical cores).
+            intra = int(os.getenv("ORT_INTRA_OP_THREADS", "0") or "0")
+            inter = int(os.getenv("ORT_INTER_OP_THREADS", "1") or "1")
+            if intra >= 0:
+                sess_options.intra_op_num_threads = intra
+            if inter >= 0:
+                sess_options.inter_op_num_threads = inter
+            sess_options.execution_mode = ort.ExecutionMode.ORT_SEQUENTIAL
+            sess_options.add_session_config_entry("session.intra_op.allow_spinning", os.getenv("ORT_ALLOW_SPINNING", "1"))
+        except Exception:
+            sess_options = None
+
         # kokoro_onnx API varies by version; try passing providers if supported.
         kokoro_sig = inspect.signature(Kokoro)
+        kokoro_kwargs = {}
         if "providers" in kokoro_sig.parameters:
-            self.kokoro = Kokoro(self.model_path, self.voices_path, providers=self.providers)
+            kokoro_kwargs["providers"] = self.providers
+        # Newer versions may support passing ORT session options.
+        if sess_options is not None:
+            for k in ("sess_options", "session_options", "ort_session_options"):
+                if k in kokoro_sig.parameters:
+                    kokoro_kwargs[k] = sess_options
+                    break
+
+        if kokoro_kwargs:
+            self.kokoro = Kokoro(self.model_path, self.voices_path, **kokoro_kwargs)
         else:
             self.kokoro = Kokoro(self.model_path, self.voices_path)
 
@@ -100,6 +142,56 @@ class TTSEngine:
         sentences = re.split(r"(?<!\w\.\w.)(?<![A-Z][a-z]\.)(?<=\.|\?|\!)\s+", text)
         return [s.strip() for s in sentences if s and s.strip()]
 
+    def split_sentences_with_offsets(self, text: str) -> List[tuple[str, int, int]]:
+        """Split `text` into sentences and return (sentence, char_start, char_end).
+
+        Offsets are relative to the provided `text` (typically a paragraph).
+        The returned span is trimmed for leading/trailing whitespace so clients
+        can highlight the exact sentence substring without `indexOf`.
+        """
+        if not text:
+            return []
+
+        # Match the whitespace boundary *after* sentence punctuation.
+        boundary = re.compile(r"(?<!\w\.\w.)(?<![A-Z][a-z]\.)(?<=\.|\?|\!)\s+")
+        out: List[tuple[str, int, int]] = []
+        start = 0
+        for m in boundary.finditer(text):
+            end = m.start()
+            if end <= start:
+                start = m.end()
+                continue
+            seg_start, seg_end = start, end
+            # Trim whitespace within the segment and adjust offsets.
+            while seg_start < seg_end and text[seg_start].isspace():
+                seg_start += 1
+            while seg_end > seg_start and text[seg_end - 1].isspace():
+                seg_end -= 1
+            if seg_end > seg_start:
+                out.append((text[seg_start:seg_end], seg_start, seg_end))
+            start = m.end()
+
+        # Tail segment.
+        if start < len(text):
+            seg_start, seg_end = start, len(text)
+            while seg_start < seg_end and text[seg_start].isspace():
+                seg_start += 1
+            while seg_end > seg_start and text[seg_end - 1].isspace():
+                seg_end -= 1
+            if seg_end > seg_start:
+                out.append((text[seg_start:seg_end], seg_start, seg_end))
+
+        # Fallback: if boundary regex didn't match but text has content.
+        if not out:
+            seg_start, seg_end = 0, len(text)
+            while seg_start < seg_end and text[seg_start].isspace():
+                seg_start += 1
+            while seg_end > seg_start and text[seg_end - 1].isspace():
+                seg_end -= 1
+            if seg_end > seg_start:
+                out.append((text[seg_start:seg_end], seg_start, seg_end))
+        return out
+
     def split_paragraphs(self, paragraphs: List[str]) -> List[tuple[int, int, str, bool]]:
         """Flatten paragraphs into (paragraph_index, sentence_index, sentence_text, is_last_in_paragraph)."""
         out: List[tuple[int, int, str, bool]] = []
@@ -114,6 +206,31 @@ class TTSEngine:
                 out.append((p_idx, s_idx, s, s_idx == (len(sentences) - 1)))
         return out
 
+    def split_paragraphs_with_offsets(self, paragraphs: List[str]) -> List[tuple[int, int, str, bool, int, int]]:
+        """Flatten paragraphs into (p_idx, s_idx, sentence, is_last, char_start, char_end)."""
+        out: List[tuple[int, int, str, bool, int, int]] = []
+        for p_idx, raw in enumerate(paragraphs):
+            p = raw or ""
+            if not p.strip():
+                continue
+            parts = self.split_sentences_with_offsets(p)
+            if not parts:
+                # Whole paragraph as one sentence.
+                seg = p
+                # Trim offsets to first/last non-space.
+                seg_start, seg_end = 0, len(seg)
+                while seg_start < seg_end and seg[seg_start].isspace():
+                    seg_start += 1
+                while seg_end > seg_start and seg[seg_end - 1].isspace():
+                    seg_end -= 1
+                if seg_end > seg_start:
+                    out.append((p_idx, 0, seg[seg_start:seg_end], True, seg_start, seg_end))
+                continue
+
+            for s_idx, (s, cs, ce) in enumerate(parts):
+                out.append((p_idx, s_idx, s, s_idx == (len(parts) - 1), cs, ce))
+        return out
+
     def _iter_pcm_frames(self, pcm16: bytes, frame_bytes: int) -> Iterable[bytes]:
         if frame_bytes <= 0:
             yield pcm16
@@ -122,7 +239,11 @@ class TTSEngine:
             yield pcm16[i : i + frame_bytes]
 
     def _apply_edge_fade_pcm16(self, pcm16: bytes, *, fade_ms: int = 6) -> bytes:
-        """Apply a short fade-in/out to reduce boundary clicks."""
+        """Apply a short fade-in/out to reduce boundary clicks.
+
+        Kokoro is synthesized per sentence, so concatenation (or appending silence)
+        can produce discontinuities. A tiny edge fade is a minimal, cheap fix.
+        """
         if not pcm16 or fade_ms <= 0:
             return pcm16
 
@@ -136,6 +257,7 @@ class TTSEngine:
         if fade_samples < 2:
             return pcm16
 
+        # Work in float for clean scaling then back to int16.
         x = samples.astype(np.float32)
         ramp = np.linspace(0.0, 1.0, fade_samples, endpoint=False, dtype=np.float32)
         x[:fade_samples] *= ramp
@@ -294,14 +416,20 @@ class TTSEngine:
         pause_question_ms: int = 260,
         pause_paragraph_extra_ms: int = 240,
         fade_ms: int = 6,
-    ) -> AsyncIterator[tuple[int, int, str, bytes]]:
+    ) -> AsyncIterator[tuple[int, int, str, bytes, int, int]]:
         """Yield sentence-atomic PCM chunks.
 
-        Each yielded binary chunk is full-sentence PCM16 plus a trailing pause.
+        Returns (paragraph_index, sentence_index, sentence_text, pcm16_bytes).
+
+        Each yielded `pcm16_bytes` contains the full sentence audio (smoothed by
+        a short fade-in/out) *plus* a short silence pause appended.
+
+        This is designed so that if buffering is needed, playback can only pause
+        between sentences (at the end of the current chunk), not mid-sentence.
         """
 
-        segments = self.split_paragraphs(paragraphs)
-        queue: asyncio.Queue[Optional[tuple[int, int, str, bytes, int]]] = asyncio.Queue(
+        segments = self.split_paragraphs_with_offsets(paragraphs)
+        queue: asyncio.Queue[Optional[tuple[int, int, str, bytes, int, int, int]]] = asyncio.Queue(
             maxsize=max(1, prefetch_sentences)
         )
 
@@ -320,7 +448,7 @@ class TTSEngine:
 
         async def producer() -> None:
             try:
-                for p_idx, s_idx, s, is_last in segments:
+                for p_idx, s_idx, s, is_last, cs, ce in segments:
                     if cancel_event is not None and cancel_event.is_set():
                         break
                     if not s:
@@ -329,7 +457,7 @@ class TTSEngine:
                     if fade_ms and fade_ms > 0:
                         pcm16 = self._apply_edge_fade_pcm16(pcm16, fade_ms=int(fade_ms))
                     pause_ms = pause_ms_for(s, is_last)
-                    await queue.put((p_idx, s_idx, s, pcm16, pause_ms))
+                    await queue.put((p_idx, s_idx, s, pcm16, pause_ms, int(cs), int(ce)))
             finally:
                 await queue.put(None)
 
@@ -339,7 +467,7 @@ class TTSEngine:
                 item = await queue.get()
                 if item is None:
                     break
-                p_idx, s_idx, sentence, pcm16, pause_ms = item
+                p_idx, s_idx, sentence, pcm16, pause_ms, cs, ce = item
                 if cancel_event is not None and cancel_event.is_set():
                     return
 
@@ -349,7 +477,7 @@ class TTSEngine:
                     chunk = pcm16 + (b"\x00" * silence_bytes)
                 else:
                     chunk = pcm16
-                yield (p_idx, s_idx, sentence, chunk)
+                yield (p_idx, s_idx, sentence, chunk, cs, ce)
         finally:
             producer_task.cancel()
             with contextlib.suppress(Exception):