Update language and prompt

.github/README.md

@@ -7,7 +7,7 @@ Hosted on Hugging Face Spaces: [xq-dokumentassistent.hf.space](https://xq-dokume
 
 ## Dansk
 
-En produktionsklar RAG-applikation, der gør det muligt at stille spørgsmål til dokumenter på dansk og få svar med kildehenvisninger. Systemet er bygget på open source-komponenter (LangChain, LangGraph, Qdrant, Ollama) og kan køre helt lokalt uden eksterne API-kald. Det implementerer hybrid søgning med reranking, en Plan-and-Execute agent med samtalehukommelse, og RAGAS-baseret evaluering af svarkvaliteten.
+En produktionsklar RAG-applikation, der gør det muligt at stille spørgsmål til dokumenter på et hvilket som helst sprog og få svar med kildehenvisninger. Systemet er bygget på open source-komponenter (LangChain, LangGraph, Qdrant, Ollama) og kan køre helt lokalt uden eksterne API-kald. Det implementerer hybrid søgning med reranking, en Plan-and-Execute agent med samtalehukommelse, og RAGAS-baseret evaluering af svarkvaliteten.
 
 ### Funktioner
 
@@ -79,7 +79,7 @@ Se `.env.example` for konfiguration pr. provider.
 
 Demoen ligger på [xq-dokumentassistent.hf.space](https://xq-dokumentassistent.hf.space).
 
-Prøv for eksempel disse spørgsmål på dansk.
+Prøv for eksempel disse spørgsmål på et hvilket som helst sprog.
 
 - "Hvad er KU's politik for brug af AI-værktøjer?"
 - "Hvilke regler gælder for brug af generativ AI i eksamen?"
@@ -177,7 +177,7 @@ docs/                      # eksempel-PDF'er eller tekster (KU AI-dokumenter)
 
 ## English
 
-A production-ready RAG application that lets users ask questions about documents in Danish and receive answers with source citations. The system is built on open source components (LangChain, LangGraph, Qdrant, Ollama) and can run fully local without any external API calls. It implements hybrid search with reranking, a Plan-and-Execute agent with conversation memory, and RAGAS-based evaluation of answer quality.
+A production-ready RAG application that lets users ask questions about documents in any language and receive answers with source citations. The system is built on open source components (LangChain, LangGraph, Qdrant, Ollama) and can run fully local without any external API calls. It implements hybrid search with reranking, a Plan-and-Execute agent with conversation memory, and RAGAS-based evaluation of answer quality.
 
 ### Capabilities
 
@@ -249,7 +249,7 @@ See `.env.example` for per-provider configuration.
 
 The demo lives at [xq-dokumentassistent.hf.space](https://xq-dokumentassistent.hf.space).
 
-Try asking these questions in Danish.
+Try asking these questions, or in your language.
 
 - "Hvad er KU's politik for brug af AI-værktøjer?"
 - "Hvilke regler gælder for brug af generativ AI i eksamen?"

README.md

@@ -17,7 +17,7 @@ Hosted on Hugging Face Spaces: [xq-dokumentassistent.hf.space](https://xq-dokume
 
 ## Dansk
 
-En produktionsklar RAG-applikation, der gør det muligt at stille spørgsmål til dokumenter på dansk og få svar med kildehenvisninger. Systemet er bygget på open source-komponenter (LangChain, LangGraph, Qdrant, Ollama) og kan køre helt lokalt uden eksterne API-kald. Det implementerer hybrid søgning med reranking, en Plan-and-Execute agent med samtalehukommelse, og RAGAS-baseret evaluering af svarkvaliteten.
+En produktionsklar RAG-applikation, der gør det muligt at stille spørgsmål til dokumenter på et hvilket som helst sprog og få svar med kildehenvisninger. Systemet er bygget på open source-komponenter (LangChain, LangGraph, Qdrant, Ollama) og kan køre helt lokalt uden eksterne API-kald. Det implementerer hybrid søgning med reranking, en Plan-and-Execute agent med samtalehukommelse, og RAGAS-baseret evaluering af svarkvaliteten.
 
 ### Funktioner
 
@@ -89,7 +89,7 @@ Se `.env.example` for konfiguration pr. provider.
 
 Demoen ligger på [xq-dokumentassistent.hf.space](https://xq-dokumentassistent.hf.space).
 
-Prøv for eksempel disse spørgsmål på dansk.
+Prøv for eksempel disse spørgsmål på et hvilket som helst sprog.
 
 - "Hvad er KU's politik for brug af AI-værktøjer?"
 - "Hvilke regler gælder for brug af generativ AI i eksamen?"
@@ -187,7 +187,7 @@ docs/                      # eksempel-PDF'er eller tekster (KU AI-dokumenter)
 
 ## English
 
-A production-ready RAG application that lets users ask questions about documents in Danish and receive answers with source citations. The system is built on open source components (LangChain, LangGraph, Qdrant, Ollama) and can run fully local without any external API calls. It implements hybrid search with reranking, a Plan-and-Execute agent with conversation memory, and RAGAS-based evaluation of answer quality.
+A production-ready RAG application that lets users ask questions about Danish documents in any language and receive answers with source citations. The system is built on open source components (LangChain, LangGraph, Qdrant, Ollama) and can run fully local without any external API calls. It implements hybrid search with reranking, a Plan-and-Execute agent with conversation memory, and RAGAS-based evaluation of answer quality.
 
 ### Capabilities
 
@@ -259,7 +259,7 @@ See `.env.example` for per-provider configuration.
 
 The demo lives at [xq-dokumentassistent.hf.space](https://xq-dokumentassistent.hf.space).
 
-Try asking these questions in Danish.
+Try asking these questions, or in your language.
 
 - "Hvad er KU's politik for brug af AI-værktøjer?"
 - "Hvilke regler gælder for brug af generativ AI i eksamen?"

src/agent/plan_and_execute.py

@@ -26,7 +26,7 @@ from langgraph.graph import END, StateGraph
 from langgraph.prebuilt import create_react_agent
 
 from src.agent.memory import ConversationMemory
-from src.agent.tools import ToolResultStore, make_retrieval_tools
+from src.agent.tools import ToolResultStore, detect_document_languages, make_retrieval_tools
 from src.models import GenerationResponse, IntentType, PipelineDetails, QueryResult
 from src.retrieval.hybrid import HybridRetriever
 from src.retrieval.reranker import Reranker
@@ -145,6 +145,7 @@ class PlanAndExecuteRouter:
         vector_store: VectorStore,
         default_top_k: int = 5,
         memory: ConversationMemory | None = None,
+        document_languages: list[str] | None = None,
     ) -> None:
         """Initialise the Plan-and-Execute router.
 
@@ -158,6 +159,9 @@ class PlanAndExecuteRouter:
                 When provided, prior conversation history is injected into
                 planner and synthesizer prompts, and each completed turn
                 is automatically recorded.
+            document_languages: Optional pre-detected list of corpus
+                languages. When omitted, the router lazily detects them
+                from the vector store on first use via the LLM.
         """
         self._llm = llm
         self._hybrid_retriever = hybrid_retriever
@@ -165,6 +169,24 @@ class PlanAndExecuteRouter:
         self._vector_store = vector_store
         self._default_top_k = default_top_k
         self._memory = memory or ConversationMemory()
+        self._document_languages: list[str] | None = (
+            list(document_languages) if document_languages else None
+        )
+
+    def _ensure_document_languages(self) -> list[str]:
+        """Lazily detect and cache the document corpus languages via the LLM.
+
+        Returns:
+            List of detected language names (e.g. ``["Danish"]`` or
+            ``["Danish", "English"]``). Empty list when the corpus is empty
+            or no readable text could be sampled.
+        """
+        if self._document_languages is not None:
+            return self._document_languages
+        self._document_languages = detect_document_languages(self._vector_store, self._llm)
+        if self._document_languages:
+            logger.info("Detected document corpus languages: %s", self._document_languages)
+        return self._document_languages
 
     # ------------------------------------------------------------------
     # Node functions
@@ -217,6 +239,7 @@ class PlanAndExecuteRouter:
                 store,
                 self._default_top_k,
                 llm_chain=self._llm,
+                document_languages=self._ensure_document_languages(),
             )
             sub_agent = create_react_agent(self._llm, tools)
 

src/agent/router.py

@@ -20,6 +20,7 @@ from langgraph.graph import END, StateGraph
 
 from src.models import IntentType, GenerationResponse, PipelineDetails, QueryResult
 from src.agent.intent_classifier import IntentClassifier
+from src.agent.tools import detect_document_languages
 from src.retrieval.hybrid import HybridRetriever
 from src.retrieval.reranker import Reranker
 
@@ -138,6 +139,7 @@ class QueryRouter:
         llm_chain: Runnable,
         *,
         translate_query: bool = True,
+        document_languages: list[str] | None = None,
     ) -> None:
         """Initialize the query router.
 
@@ -147,17 +149,42 @@ class QueryRouter:
             reranker: Reranker instance.
             llm_chain: LLM chain (llm | StrOutputParser) for generation,
                 translation, and language detection.
-            translate_query: Whether to translate non-Danish queries to Danish
-                before retrieval. When False, language detection still runs for
-                the answer-language rule but no translation is performed.
+            translate_query: Whether to translate the user query into a
+                corpus language before BM25 retrieval when the query
+                language does not already match one of the corpus languages.
+                When False, no translation is performed.
+            document_languages: Optional pre-detected list of corpus
+                languages. When omitted, the router lazily detects them
+                from the vector store on first translation/generation via
+                the LLM.
         """
         self._intent_classifier = intent_classifier
         self._hybrid_retriever = hybrid_retriever
         self._reranker = reranker
         self._llm_chain = llm_chain
         self._translate_query_enabled = translate_query
+        self._document_languages: list[str] | None = (
+            list(document_languages) if document_languages else None
+        )
         self._graph = self._build_graph()
 
+    def _ensure_document_languages(self) -> list[str]:
+        """Lazily detect and cache the document corpus languages via the LLM.
+
+        Returns:
+            List of detected language names (e.g. ``["Danish"]`` or
+            ``["Danish", "English"]``). Empty list when the corpus is empty
+            or no readable text could be sampled.
+        """
+        if self._document_languages is not None:
+            return self._document_languages
+        self._document_languages = detect_document_languages(
+            self._hybrid_retriever.vector_store, self._llm_chain
+        )
+        if self._document_languages:
+            logger.info("Detected document corpus languages: %s", self._document_languages)
+        return self._document_languages
+
     def _detect_language_and_intent(self, query: str) -> tuple[str, IntentType]:
         """Detect the query language and classify intent in a single LLM call.
 
@@ -203,29 +230,49 @@ class QueryRouter:
         return detected, intent
 
     def _translate_query(self, query: str, detected_language: str) -> str:
-        """Translate the query to Danish if needed.
+        """Translate the query into a corpus language when needed.
+
+        BM25 needs token-level matches against the corpus, so when the user's
+        query language is not present in the corpus we translate it to the
+        primary corpus language. When the corpus contains the user's
+        language already (single- or multi-language corpus), no translation
+        is performed — the original query is used as-is.
 
         Args:
             query: The user's original query.
             detected_language: Detected language of the query.
 
         Returns:
-            The Danish retrieval query, or the original if already Danish.
+            The retrieval query, translated when necessary.
         """
-        if detected_language.lower() in ("danish", "dansk"):
+        doc_langs = self._ensure_document_languages()
+
+        # Without a known corpus language we cannot pick a translation target.
+        if not doc_langs:
+            return query
+
+        user_lang = detected_language.lower().strip()
+        doc_lang_set = {lang.lower() for lang in doc_langs}
+        # Accept the Danish autonym so legacy "dansk" detection still matches.
+        if user_lang == "dansk":
+            user_lang = "danish"
+
+        # Query already in one of the corpus languages → BM25 will work as-is.
+        if user_lang in doc_lang_set:
             return query
 
         if not self._translate_query_enabled:
             logger.info("Query translation disabled; using original query for retrieval")
             return query
 
+        target = doc_langs[0]
         translate_prompt = (
-            "Translate the following text to Danish. "
+            f"Translate the following text to {target}. "
             "Reply with ONLY the translated text, nothing else.\n\n"
             f"Text: {query}"
         )
         translated = _extract_content(self._llm_chain.invoke(translate_prompt))
-        logger.info("Translated query to Danish: %s", translated)
+        logger.info("Translated query to %s: %s", target, translated)
         return translated
 
     # ------------------------------------------------------------------
@@ -552,10 +599,21 @@ class QueryRouter:
 
         instruction = intent_instructions[intent]
 
+        doc_langs = self._ensure_document_languages()
+        if doc_langs:
+            corpus_clause = (
+                f"The context documents may be in {' or '.join(doc_langs)} — "
+                f"use them as reference but always reply in {user_language}."
+            )
+        else:
+            corpus_clause = (
+                f"The context documents may be in a different language — "
+                f"use them as reference but always reply in {user_language}."
+            )
         language_rule = (
             f"IMPORTANT: You MUST answer in {user_language}. "
             f"The user asked in {user_language}, so your entire response must be in {user_language}. "
-            f"The context documents may be in Danish — use them as reference but always reply in {user_language}."
+            f"{corpus_clause}"
         )
 
         return (

src/agent/tools.py

@@ -69,6 +69,81 @@ class ToolResultStore:
     fused_results: list[QueryResult] = field(default_factory=list)
 
 
+def detect_document_languages(
+    vector_store: VectorStore,
+    llm: Runnable,
+    *,
+    max_documents: int = 5,
+    chunks_per_document: int = 2,
+    sample_chars: int = 2000,
+) -> list[str]:
+    """Detect all languages present in the document corpus via the LLM.
+
+    Samples chunks from up to ``max_documents`` distinct documents and asks the
+    LLM in a single call to identify every language present. Used by routers
+    so that intermediate retrieval queries can be phrased in the corpus
+    language(s) without hardcoding any specific language.
+
+    Args:
+        vector_store: VectorStore to sample chunks from.
+        llm: LLM runnable used for the single detection call.
+        max_documents: Maximum number of documents to sample from.
+        chunks_per_document: Chunks taken from each sampled document.
+        sample_chars: Cap on total sample text length sent to the LLM.
+
+    Returns:
+        List of detected language names in English (e.g. ``["Danish"]`` or
+        ``["Danish", "English"]``), preserving the order returned by the LLM.
+        Returns an empty list when the corpus is empty or no readable text
+        could be sampled (e.g. when the vector store is mocked in tests).
+    """
+    try:
+        ids = vector_store.list_document_ids()
+    except Exception:
+        return []
+    if not isinstance(ids, list) or not ids:
+        return []
+
+    samples: list[str] = []
+    for doc_id in ids[:max_documents]:
+        try:
+            chunks = vector_store.get_chunks_by_document_id(doc_id)
+        except Exception:
+            continue
+        if not isinstance(chunks, list):
+            continue
+        for c in chunks[:chunks_per_document]:
+            text = (getattr(c, "text", "") or "").strip()
+            if text:
+                samples.append(text)
+
+    sample_text = "\n---\n".join(samples)[:sample_chars].strip()
+    if not sample_text:
+        return []
+
+    prompt = (
+        "You are a language detector. The text samples below come from "
+        "different documents in a knowledge base. Identify ALL distinct "
+        "languages present across the samples (do not list a language more "
+        "than once). Reply with ONLY the language names in English, one per "
+        "line, no explanation.\n\n"
+        f"Samples:\n{sample_text}"
+    )
+    raw = _extract_content(llm.invoke(prompt))
+
+    seen: set[str] = set()
+    detected: list[str] = []
+    for line in raw.strip().splitlines():
+        name = line.strip().lstrip("-•*0123456789.) ").rstrip(".").strip()
+        if not name:
+            continue
+        name = name.capitalize()
+        if name.lower() not in seen:
+            seen.add(name.lower())
+            detected.append(name)
+    return detected
+
+
 def _merge_results(existing: list[QueryResult], new: list[QueryResult]) -> list[QueryResult]:
     """Merge two QueryResult lists by chunk_id, keeping the highest score.
 
@@ -117,6 +192,7 @@ def make_retrieval_tools(
     store: ToolResultStore,
     default_top_k: int = 5,
     llm_chain: Runnable | None = None,
+    document_languages: list[str] | None = None,
 ) -> list:
     """Create retrieval tools bound to the given components and result store.
 
@@ -133,10 +209,34 @@ def make_retrieval_tools(
         llm_chain: Optional LLM chain for tools that need generation
             (summarize_document, multi_query_search). When None, those
             tools are excluded from the returned list.
+        document_languages: Detected languages of the document corpus
+            (e.g. ``["Danish"]`` or ``["Danish", "English"]``). Used by
+            multi_query_search to phrase sub-queries in the corpus
+            language(s) for best BM25 recall. When None or empty, the
+            sub-query language is left unconstrained.
 
     Returns:
         List of LangChain tool callables ready for bind_tools / ToolNode.
     """
+    if document_languages:
+        if len(document_languages) == 1:
+            _lang_clause = (
+                f"The queries should be in {document_languages[0]} "
+                f"(the document base is {document_languages[0]})."
+            )
+        else:
+            _lang_list = ", ".join(document_languages)
+            _lang_clause = (
+                f"The document base contains multiple languages: {_lang_list}. "
+                f"For each sub-query, write it in whichever of these languages "
+                f"best matches the topic; mix languages across sub-queries if "
+                f"the topic is likely covered by documents in different languages."
+            )
+    else:
+        _lang_clause = (
+            "Write each sub-query in the language most likely used by the "
+            "underlying documents."
+        )
 
     # ------------------------------------------------------------------
     # Core search tool
@@ -317,8 +417,7 @@ def make_retrieval_tools(
             decompose_prompt = (
                 "You are a search query planner. Given a complex question, "
                 "decompose it into 2-4 simple, independent search queries that "
-                "together cover all aspects of the question. The queries should "
-                "be in Danish (since the document base is Danish).\n\n"
+                f"together cover all aspects of the question. {_lang_clause}\n\n"
                 "Reply with ONLY the queries, one per line, nothing else.\n\n"
                 f"Question: {question}"
             )

src/retrieval/hybrid.py

@@ -52,6 +52,11 @@ class HybridRetriever:
         self._dense_weight = dense_weight
         self._bm25_weight = bm25_weight
 
+    @property
+    def vector_store(self) -> VectorStore:
+        """Underlying vector store, exposed for callers that need corpus-level access."""
+        return self._vector_store
+
     def search(self, query: str, top_k: int) -> list[QueryResult]:
         """Execute hybrid search combining dense and sparse results.
 

tests/test_router.py

@@ -244,7 +244,7 @@ class TestQueryTranslation:
         retriever.search_detailed.assert_called_once_with("Hvad er reglerne?", top_k=3)
 
     def test_english_query_translated_for_retrieval(self, mock_components) -> None:
-        """English queries should be translated to Danish for retrieval."""
+        """English queries should be translated into the corpus language for retrieval."""
         classifier, retriever, reranker, llm_chain = mock_components
 
         results = [_make_query_result("ctx", 0.5)]
@@ -252,7 +252,10 @@ class TestQueryTranslation:
         reranker.rerank.return_value = results
         _setup_llm_chain_english(llm_chain, "Hvad er reglerne?", "The rules are...", intent="rag")
 
-        router = QueryRouter(classifier, retriever, reranker, llm_chain, translate_query=True)
+        router = QueryRouter(
+            classifier, retriever, reranker, llm_chain,
+            translate_query=True, document_languages=["Danish"],
+        )
         response = router.route("What are the rules?", top_k=3)
 
         # 3 invoke calls: combined detection + translation + generation