update field definitions for clarity and usage instructions in metadata and also prompt for better field_usage while facet generation.

app.py

@@ -101,15 +101,15 @@ field_metadata = [
     },
     {
         "field_name": "company_name",
-        "type": "string (exact match)",
+        "type": "string (exact match, for faceting)",
         "example_values": ["pfizer inc.", "astrazeneca plc", "roche"],
-        "definition": "The canonical, standardized name of a company. Good for comparison. For searching, prefer 'company_name_s' to catch all variations."
+        "definition": "The canonical, standardized name of a company. **Crucially, you MUST use this field for `terms` faceting** to group results by a unique company. Do NOT use this for searching."
     },
     {
         "field_name": "company_name_s",
         "type": "string (multi-valued, for searching)",
         "example_values": ["pfizer inc.", "roche", "f. hoffmann-la roche ag", "nih"],
-        "definition": "A field containing all known names, synonyms, and abbreviations for a company. **Use this field for all queries involving a company name** to ensure comprehensive results (e.g., searching for 'roche' will also find 'f. hoffmann-la roche ag')."
+        "definition": "A field containing all known names and synonyms for a company. **You MUST use this field for all `query` parameter searches involving a company name** to ensure comprehensive results. Do NOT use for `terms` faceting."
     },
     {
         "field_name": "territory_hq_s",
@@ -121,61 +121,61 @@ field_metadata = [
         "field_name": "therapeutic_category",
         "type": "string (specific)",
         "example_values": ["cancer, other", "cancer, nsclc metastatic", "alzheimer's"],
-        "definition": "The specific disease or therapeutic area being targeted, often in a detailed, hierarchical format. Use for very specific disease queries."
+        "definition": "The specific disease or therapeutic area being targeted. Use for very specific disease queries."
     },
     {
         "field_name": "therapeutic_category_s",
         "type": "string (multi-valued, for searching)",
         "example_values": ["cancer", "oncology", "infections", "cns"],
-        "definition": "Broader, multi-valued therapeutic categories and their synonyms. **Use this field for broad category searches** (e.g., 'cancer', 'cns', 'immune') as it is designed for easier searching."
+        "definition": "Broader, multi-valued therapeutic categories and their synonyms. **Use this field for broad category searches** in the `query` parameter."
     },
     {
         "field_name": "compound_name",
-        "type": "string (exact match)",
+        "type": "string (exact match, for faceting)",
         "example_values": ["opdivo injection solution", "keytruda injection solution"],
-        "definition": "The specific, full trade or development name of a drug/compound, including its formulation. For searching, prefer 'compound_name_s'."
+        "definition": "The specific, full trade name of a drug. **Use this field for `terms` faceting** on compounds."
     },
     {
         "field_name": "compound_name_s",
         "type": "string (multi-valued, for searching)",
         "example_values": ["nivolumab injection solution", "opdivo injection solution", "ono-4538 injection solution"],
-        "definition": "A field containing all known trade names, development codes, and synonyms for a drug/compound. **Use this field for all queries involving a compound name**."
+        "definition": "A field with all known trade names and synonyms for a drug. **Use this field for all `query` parameter searches** involving a compound name."
     },
     {
         "field_name": "molecule_name",
-        "type": "string (exact match)",
+        "type": "string (exact match, for faceting)",
         "example_values": ["cannabidiol", "paclitaxel", "pembrolizumab"],
-        "definition": "The generic, non-proprietary name of the active molecule. For searching, prefer 'molecule_name_s'."
+        "definition": "The generic, non-proprietary name of the active molecule. **Use this field for `terms` faceting** on molecules."
     },
     {
         "field_name": "molecule_name_s",
         "type": "string (multi-valued, for searching)",
         "example_values": ["cbd", "s1-220", "a1002n5s"],
-        "definition": "A field containing all known generic names, development codes, and synonyms for a molecule. **Use this field for all queries involving a molecule name**."
+        "definition": "A field with all known generic names and synonyms for a molecule. **Use this field for all `query` parameter searches** involving a molecule name."
     },
     {
         "field_name": "highest_phase",
         "type": "string (categorical)",
         "example_values": ["marketed", "phase 2", "phase 1"],
-        "definition": "The highest stage of development a drug has ever reached. Distinct from 'event_type' which is about the event in the news."
+        "definition": "The highest stage of development a drug has ever reached."
     },
     {
         "field_name": "drug_delivery_branch_s",
         "type": "string (multi-valued, for searching)",
         "example_values": ["injection", "parenteral", "oral", "injection, other", "oral, other"],
-        "definition": "The method of drug administration (e.g., injection, oral). **Use this for queries about route of administration** as it contains broader, search-friendly terms. contains synonyms."
+        "definition": "The method of drug administration. **Use this for `query` parameter searches about route of administration** as it contains broader, search-friendly terms."
     },
     {
         "field_name": "drug_delivery_branch",
-        "type": "string (categorical, specific)",
+        "type": "string (categorical, specific, for faceting)",
         "example_values": ["injection, other", "prefilled syringes", "np liposome", "oral enteric/delayed release"],
-        "definition": "The most specific category of drug delivery technology or formulation. This provides a detailed, granular breakdown (e.g., 'prefilled syringes', 'np liposome'). Use this field only when a user asks for very specific technologies or for detailed faceting on the technology itself."
+        "definition": "The most specific category of drug delivery technology. **Use this field for `terms` faceting** on specific delivery technologies."
     },
     {
         "field_name": "route_branch",
         "type": "string (categorical)",
         "example_values": ["injection", "oral", "topical", "inhalation"],
-        "definition": "The specific, primary route of drug administration. This field is good for faceting on exact routes, for example comparing values for injection vs oral. it is usually better to use the broader 'drug_delivery_branch_s' field which contains synonyms and parent categories for search operations."
+        "definition": "The primary route of drug administration. Good for faceting on exact routes."
     },
     {
         "field_name": "molecule_api_group",
@@ -193,13 +193,13 @@ field_metadata = [
         "field_name": "date",
         "type": "date",
         "example_values": ["2020-10-22T00:00:00Z"],
-        "definition": "The full publication date and time in ISO 8601 format. Use for precise date range queries (e.g., 'last 30 days', 'between date A and date B')."
+        "definition": "The full publication date and time in ISO 8601 format. Use for precise date range queries."
     },
     {
         "field_name": "date_year",
         "type": "number (year)",
         "example_values": [2020, 2021, 2022],
-        "definition": "The 4-digit year of publication. **Use this for queries involving whole years** (e.g., 'in 2023', 'last year', 'since 2020') as it is more efficient."
+        "definition": "The 4-digit year of publication. **Use this for queries involving whole years** (e.g., 'in 2023', 'last year', 'since 2020')."
     },
     {
         "field_name": "total_deal_value_in_million",
@@ -244,11 +244,14 @@ You are an expert Solr query engineer who converts natural language questions in
 ### CONTEXT & RULES
 
 1.  **Today's Date for Calculations**: {datetime.datetime.now().date().strftime("%Y-%m-%d")}
-2.  **Field Usage**: You MUST use the fields described in the 'Field Definitions' section. Pay close attention to the definitions to select the correct field. For instance, use `company_name_s` for searching companies and `date_year` for year-based queries.
-3.  **No `count(*)`**: Do NOT use functions like `count(*)`. The default facet bucket count is sufficient for counting documents.
-4.  **Allowed Aggregations**: For statistical facets (`stats` or `stat` type), only use these functions: `sum`, `avg`, `min`, `max`, `unique`. The primary metric field is `total_deal_value_in_million`.
-5.  **Term Facet Limits**: Every `terms` facet MUST include a `limit` key. Default to `limit: 10` unless the user specifies a different number of top results.
-6.  **Output Format**: Your final output must be a single, raw JSON object and nothing else. Do not add comments, explanations, or markdown formatting like ```json.
+2.  **Field Usage**: You MUST use the fields described in the 'Field Definitions' section. Pay close attention to the definitions to select the correct field.
+3.  **Facet vs. Query Field Distinction**: This is critical.
+    *   For searching in the main `query` parameter, ALWAYS use the multi-valued search fields (ending in `_s`, like `company_name_s`) to get comprehensive results.
+    *   For grouping in a `terms` facet, ALWAYS use the canonical, single-value field (e.g., `company_name`, `molecule_name`) to ensure unique and accurate grouping.
+4.  **No `count(*)`**: Do NOT use functions like `count(*)`. The default facet bucket count is sufficient for counting documents.
+5.  **Allowed Aggregations**: For statistical facets (`stats` or `stat` type), only use these functions: `sum`, `avg`, `min`, `max`, `unique`. The primary metric field is `total_deal_value_in_million`.
+6.  **Term Facet Limits**: Every `terms` facet MUST include a `limit` key. Default to `limit: 10` unless the user specifies a different number of top results.
+7.  **Output Format**: Your final output must be a single, raw JSON object and nothing else. 
 
 ---
 ### FIELD DEFINITIONS (Your Source of Truth)