docs/findings.md

Dogfooding findings

Things contimp-app surfaced about the contimp/oumi pipeline. Newest first.

2026-06-10 — api importer drops vanilla (no-tools) LangFuse chat traces

What: api/shared/src/shared/inference_logs/external/langfuse.py parses a generation's input only when it is a JSON-encoded dict ({"model": ..., "messages": [...]}). But the LangFuse OpenAI integration (langfuse.openai, verified on SDK 4.7.1, _extract_chat_prompt) logs the bare message array whenever the request carries no tools/functions — the most common customer shape. Result: every vanilla single-turn chat trace is silently dropped on import (parse() yields nothing). Traces from tool-using apps import fine.

Repro: trace any plain chat.completions.create(messages=...) call through langfuse.openai, export observations JSONL, run parse() → 0 conversations. contimp-app's pr-area task hits this; config-copilot (tools) does not.

Verified fix (tested against real exports of both shapes), in _parse_input_dict:

    if isinstance(parsed, list):
        # Vanilla chat completions: the LangFuse OpenAI integration logs the
        # bare message array when the request carries no tools/functions.
        return {"messages": parsed}
    return parsed if isinstance(parsed, dict) else None

Relevant existing TODO: LOU-1829 ("simplify once real customer exports are available") — these exports now exist, generated by this app.