added agent gender

plan.md

@@ -1,91 +0,0 @@
-# Project Scan & Improvement Plan
-
-## 1. Overview
-A static analysis of the **`c:/voice/the-brain/python-services`** package was performed. The goal is to surface:
-- Critical bugs and runtime errors.
-- Security concerns (PII leakage, missing environment validation).
-- Performance bottlenecks.
-- Code‑quality and maintainability issues.
-- Recommendations for testing, documentation, and CI.
-
-All file references are clickable for easy navigation.
-
----
-
-## 2. Critical Bugs & Security Issues
-| Severity | File | Line(s) | Issue | Recommendation |
-|---|---|---|---|---|
-
-
-| **High** | [caller_info_extractor.py](file:///c:/voice/the-brain/python-services/caller_info_extractor.py) | 46‑47 | Phone numbers are logged at `INFO` level, exposing PII. | Remove raw numbers from logs or mask them (e.g., `****1234`) and downgrade to `DEBUG`. |
-| **High** | [utils.py](file:///c:/voice/the-brain/python-services/utils.py) | 135‑141 | `strip_formatting` removes all parenthetical text, potentially discarding spoken content. | Refine regex to only strip markdown parentheses (`[text](url)`) while preserving plain language parentheses. |
-| **Medium** | [conversation_manager.py](file:///c:/voice/the-brain/python-services/conversation_manager.py) | 84‑86 | Hindi (`hi`) is forced to Urdu (`ur`), which may mis‑route genuine Hindi callers. | Document the rationale; make the mapping configurable. |
-| **Medium** | [llm_server.py](file:///c:/voice/the-brain/python-services/llm_server.py) | 174‑176 | Token‑governor uses `req.sessionStart` which can be `None`, leading to misleading duration calculations. | Guard against `None` and log a warning when the timestamp is missing. |
-
----
-
-## 3. High‑Priority Improvement Roadmap
-
-1. **PII‑Safe Logging**
-   - Add a helper `mask_phone(phone: str) -> str` returning a masked version (e.g., `****1234`).
-   - Use it wherever phone numbers are logged and lower the log level to `DEBUG`.
-2. **Typed Code & Static Checks**
-   - Add missing type hints across the code base.
-   - Integrate `mypy --strict` and `ruff`/`flake8` into CI.
-3. **Test Suite**
-   - Add `tests/` with pytest covering:
-     - Regex extraction (`extract_phone_from_text`, `extract_name_from_text`).
-     - Language detection edge cases.
-     - Conversation session lifecycle (creation, expiration, cleanup).
-     - API endpoint sanity using FastAPI’s `TestClient`.
-4. **Dependency Pinning**
-   - Update `requirements.txt` with explicit version numbers.
-5. **Documentation Upgrade**
-   - Expand `README.md` with setup, run instructions, API reference, and contribution guide.
-6. **Docker / Deploy Hygiene**
-   - Add a health‑check in the `Dockerfile` that hits `/health`.
-   - Externalise hard‑coded thresholds and model names to env vars or a config file.
-
----
-
-## 4. Medium‑Priority Enhancements
-- Lazy‑load Groq/Fireworks clients after configuration validation.
-- Use a module‑level `ThreadPoolExecutor` for all `run_in_executor` calls to avoid per‑request thread creation.
-- Replace duplicate `arabic_regex` definition in `utils.py` with a single compiled regex.
-- Introduce structured JSON logging for easier aggregation.
-- Implement exponential back‑off before falling back to Fireworks on 429 errors.
-
----
-
-## 5. Low‑Priority / Nice‑to‑Have
-- Add GitHub Actions workflow running lint, type‑check, and tests on each PR.
-- Provide pre‑commit hooks (`black`, `ruff`).
-- Expose OpenAPI schema (FastAPI does this automatically; just add the URL to docs).
-- Add Prometheus metrics exporter for request latency, token usage, and error rates.
-- Store language‑specific prompt templates in separate files for easier updates.
-
----
-
-## 6. Verification Plan
-- **Manual Review** – Walk through each changed file; confirm no raw PII appears in logs.
-- **Automated Tests** – Run `pytest`; ensure ≥80 % coverage for core utilities.
-- **Static Analysis** – Run `ruff` and `mypy --strict`; fix all reported issues.
-- **Runtime Smoke Test** – Start the server (`uvicorn llm_server:app --port 7860`) and hit `/health` and `/session/start` endpoints.
-- **Security Scan** – Simulate a request that extracts a phone number and verify logs contain only masked values.
-
----
-
-## 7. Immediate Action Items (Task List)
-- `[ ]` Create `config.py` with env‑var validation.
-- `[ ]` Refactor `client.py` to lazily instantiate providers after config validation.
-- `[ ]` Implement `mask_phone` and replace raw phone logs.
-- `[ ]` Add missing type hints and run `mypy`.
-- `[ ]` Write unit tests for regex utilities.
-- `[ ]` Pin dependency versions in `requirements.txt`.
-- `[ ]` Expand `README.md`.
-- `[ ]` Add Docker health‑check.
-- `[ ]` Externalise thresholds and model names.
-
----
-
-*Prepared by Antigravity – your AI coding assistant.*

python-services/conversation_manager.py

@@ -34,12 +34,14 @@ class ConversationSession:
         session_id: str,
         company_name: str,
         agent_name: str,
+        agent_gender: str = "female",       # Added parameter
         custom_rules: List[str] = None,
         document_context: str = "",  # RAG payload vector string placeholder
     ):
         self.session_id = session_id
         self.company_name = company_name
         self.agent_name = agent_name
+        self.agent_gender = agent_gender    # Store parameter
         self.custom_rules = custom_rules or []
         self.document_context = document_context
 
@@ -151,6 +153,7 @@ class ConversationSession:
             "session_id": self.session_id,
             "company": self.company_name,
             "agent": self.agent_name,
+            "agent_gender": self.agent_gender,  # Added to summary dict
             "language": self.get_current_language(),
             "turns": self.turn_count,
             "caller_info": self.caller_info.to_dict(),
@@ -172,6 +175,7 @@ class ConversationManager:
         self,
         company_name: str,
         agent_name: str,
+        agent_gender: str = "female",       # Added parameter
         custom_rules: List[str] = None,
         document_context: str = "",
         session_id: Optional[str] = None,
@@ -183,6 +187,7 @@ class ConversationManager:
             session_id=sid,
             company_name=company_name,
             agent_name=agent_name,
+            agent_gender=agent_gender,      # Pass parameter
             custom_rules=custom_rules,
             document_context=document_context,
         )

python-services/llm_server.py

@@ -66,6 +66,7 @@ manager = ConversationManager()
 class StartSessionRequest(BaseModel):
     company_name: str = "Our Company"
     agent_name: str = "Sara"
+    agent_gender: str = "female"         # Added field
     custom_rules: List[str] = []
     document_context: str = ""          # RAG text block container
     session_id: Optional[str] = None    # Generated if missing
@@ -93,6 +94,7 @@ async def start_session(req: StartSessionRequest):
     session = manager.create_session(
         company_name=req.company_name,
         agent_name=req.agent_name,
+        agent_gender=req.agent_gender,   # Pass parameter
         custom_rules=req.custom_rules,
         document_context=req.document_context,
         session_id=req.session_id,
@@ -125,6 +127,7 @@ async def chat(req: ChatRequest):
         session = manager.create_session(
             company_name=req.company_name or "Our Company",
             agent_name=req.agent_name or "Sara",
+            agent_gender="female",      # Default backup gender
             session_id=req.session_id
         )
         logger.warning(f"Target memory space missing. Initialized emergency backup stack instance: {session.session_id}")
@@ -152,11 +155,12 @@ async def chat(req: ChatRequest):
     system_prompt = build_system_prompt(
         company_name=session.company_name,
         agent_name=session.agent_name,
+        agent_gender=session.agent_gender, # Pass parameter from session
         language=detected_language,
         document_context=session.document_context,
         custom_rules=session.custom_rules,
         collected_caller_info=session.caller_info.to_dict(),
-        input_text=req.message  # FIXED: Directly passed user input string to match scripts accurately
+        input_text=req.message
     )
 
     # Append structural input tokens into history logs

python-services/prompt_builder.py

@@ -34,6 +34,7 @@ LANGUAGE_SCRIPT_MAP = {
 def build_system_prompt(
     company_name: str,
     agent_name: str,
+    agent_gender: str,                 # Added parameter
     language: str,
     document_context: str = "",         # RAG placeholder channel
     custom_rules: List[str] = None,     # Company-specific behavioral rules
@@ -54,6 +55,7 @@ def build_system_prompt(
     caller_context = _get_caller_context(collected_caller_info or {})
     document_section = _get_document_section(document_context)
     rules_section = _get_rules_section(custom_rules or [])
+    agent_gender_instruction = _get_agent_gender_instruction(agent_gender, lang)
     current_time = datetime.now().strftime("%Y-%m-%d %H:%M")
 
     prompt = f"""You are {agent_name}, a professional customer service representative working for {company_name}.
@@ -64,6 +66,9 @@ def build_system_prompt(
 - Current time: {current_time}.
 - You operate entirely as an elite human client coordinator on a live telephone connection — NEVER state, hint, or imply that you are an "AI", "Robot", "Bot", "Language Model", or a software script. 
 
+## Agent Gender & Grammatical Self-Reference Rules
+{agent_gender_instruction}
+
 ## Language & Script Enforcement Constraints
 {language_instructions}
 
@@ -78,6 +83,7 @@ def build_system_prompt(
 8. VOICE SYNTHESIS CRITICAL RULE: Never use bullet points, asterisks, numbered listings, dashes, or markdown typography formatting (** or __) in your output phrases. Speak purely in plain, flowing text sentences.
 9. If you do not have data to answer a query, state that you will verify the logs or arrange a specialist follow-up call. Never hypothesize details.
 10. Avoid mentioning competitors of {company_name} or drafting relative industry charts.
+11. Avoid referencing your training, software design parameters, or guidelines explicitly during runtime. Maintain the human coordinator persona at all times.
 
 ## Caller Information Collection
 Collect the caller's name and contact number naturally throughout the dialogue.
@@ -100,7 +106,7 @@ Collect the caller's name and contact number naturally throughout the dialogue.
     return prompt.strip()
 
 
-def build_system_prompts(lang_code: str, user_name_inner: str) -> List[Dict[str, str]]:
+def build_system_prompts(lang_code: str, user_name_inner: str, agent_gender: str = "female") -> List[Dict[str, str]]:
     """
     Compiles system configuration parameters into an array format compatible 
     with standard OpenAI and Groq chat execution completions.
@@ -109,6 +115,7 @@ def build_system_prompts(lang_code: str, user_name_inner: str) -> List[Dict[str,
     raw_prompt = build_system_prompt(
         company_name="our world-class organization",
         agent_name="the customer support team coordinator",
+        agent_gender=agent_gender,
         language=lang_code,
         collected_caller_info={"name": user_name_inner} if user_name_inner else {}
     )
@@ -116,6 +123,33 @@ def build_system_prompts(lang_code: str, user_name_inner: str) -> List[Dict[str,
     return [{"role": "system", "content": raw_prompt}]
 
 
+def _get_agent_gender_instruction(agent_gender: str, language: str) -> str:
+    """Generates standalone gender constraints section string based on agent gender and active language."""
+    lang = (language or "en").lower().strip()
+    
+    if agent_gender == "female":
+        return """- Your gender is FEMALE. You MUST speak and refer to yourself using proper feminine grammatical forms, verbs, adjectives, and pronouns.
+- In Urdu (اردو): Always use feminine first-person verb endings and adjectives (e.g., use 'کرتی ہوں', 'رہی ہوں', 'ہوں گی', 'تیار ہوں' [f] and NEVER 'کرتا ہوں', 'رہا ہوں', 'ہوں گا', 'تیار ہوں' [m]).
+- In Arabic (العربية): Always use feminine adjectives, nouns, and participial forms (e.g., use 'أنا سعيدة', 'أنا مسؤولة', 'أنا متحدثة', 'مستعدة' [f] and NEVER 'سعيد', 'مسؤول', 'متحدث', 'مستعد' [m]). This rule applies across all Arabic dialects (including Gulf, Egyptian, Levantine); mirror the caller's dialect using correct feminine forms.
+- In Persian (فارسی): Verbs and adjectives are grammatically genderless in the first person, but use feminine self-identifiers if speaking of your role (e.g., using 'خانم' [Ms./Lady/Coordinator] as a self-identifier if applicable).
+- In English (en): Refer to yourself using female pronouns (she/her) if the context requires pronouns, and standard first-person pronouns (I/me) otherwise."""
+    
+    elif agent_gender == "male":
+        return """- Your gender is MALE. You MUST speak and refer to yourself using proper masculine grammatical forms, verbs, adjectives, and pronouns.
+- In Urdu (اردو): Always use masculine first-person verb endings and adjectives (e.g., use 'کرتا ہوں', 'رہا ہوں', 'ہوں گا', 'تیار ہوں' [m] and NEVER 'کرتی ہوں', 'رہی ہوں', 'ہوں گی', 'تیار ہوں' [f]).
+- In Arabic (العربية): Always use masculine adjectives, nouns, and participial forms (e.g., use 'أنا سعيد', 'أنا مسؤول', 'أنا متحدث', 'مستعد' [m] and NEVER 'سعيدة', 'مسؤولة', 'متحدثة', 'مستعدة' [f]). This rule applies across all Arabic dialects (including Gulf, Egyptian, Levantine); mirror the caller's dialect using correct masculine forms.
+- In Persian (فارسی): Verbs and adjectives are grammatically genderless in the first person, but use masculine self-identifiers if speaking of your role (e.g., using 'آقای' [Mr./Coordinator] as a self-identifier if applicable).
+- In English (en): Refer to yourself using male pronouns (he/him) if the context requires pronouns, and standard first-person pronouns (I/me) otherwise."""
+    
+    else:  # unspecified / neutral
+        return """- Your gender is UNSPECIFIED/NEUTRAL. You MUST use gender-neutral self-referencing language and honorifics where possible.
+- Avoid committing to strict male or female verb endings or adjectives.
+- In Urdu (اردو): Urdu grammar strictly requires gendered verb endings for first-person references. You MUST restructure sentences to avoid first-person gendered verb endings where possible. When unavoidable, default to formal masculine verb endings (e.g., 'کرتے ہیں') as the standard formal professional default in Pakistani contexts.
+- In Arabic (العربية): Use formal neutral phrasing and avoid adjectives, nouns, or participial forms that require gender agreement about yourself. Default to plural or balanced structures if gender is unavoidable. Dialect rendering should also maintain this neutral tone.
+- In Persian (فارسی): Verbs and adjectives are naturally gender-neutral, so communicate normally without gendered role titles.
+- In English (en): Use standard gender-neutral pronouns (they/them) or restructure sentences to avoid third-person self-references."""
+
+
 def _get_language_instructions(language: str, input_text: str) -> str:
     """Generates explicit orthographical rule blocks based on targeted language codes."""
     script_info = LANGUAGE_SCRIPT_MAP.get(language)