| # Implementation Issues and Resolutions |
|
|
| This document captures the major issues encountered while implementing and operationalizing the revenue-assurance agent workflow, including local execution and Azure ML submission. |
|
|
| ## 1) No initial implementation in starter repo |
| - Symptom: The workspace started as a minimal agent scaffold and had no revenue-assurance domain workflow. |
| - Root cause: Domain features were not yet built. |
| - Resolution: Added domain models, reconciliation logic, case management, skills, deterministic investigator, and model-backed agentic investigator. |
| - Prevention: Keep a milestone checklist and ship in vertical slices (data model -> rule engine -> case -> recommendation). |
|
|
| ## 2) Missing Python dependencies |
| - Symptom: Runtime/import failures while enabling model-backed workflow and AML submit path. |
| - Root cause: Required SDKs were not yet installed. |
| - Resolution: Added and used dependencies for OpenAI and Azure ML stack. |
| - Prevention: Keep dependency declarations in pyproject and verify with CI test runs. |
|
|
| ## 3) Placeholder values treated as real model settings |
| - Symptom: Model path attempted using invalid placeholder endpoint/key values. |
| - Root cause: Config accepted placeholder-like values as usable credentials. |
| - Resolution: Added placeholder detection in config loader and automatic fallback to non-agentic behavior. |
| - Prevention: Validate config at startup and fail fast or gracefully downgrade by policy. |
|
|
| ## 4) Unsupported model request parameter |
| - Symptom: Upstream model call rejected a hard-coded generation parameter. |
| - Root cause: Parameter compatibility mismatch with deployed model API. |
| - Resolution: Removed unsupported parameter and aligned request payload with deployment behavior. |
| - Prevention: Keep per-model capability checks and a compatibility matrix. |
|
|
| ## 5) Non-numeric confidence handling |
| - Symptom: Agent output parsing was brittle when confidence came as text. |
| - Root cause: Parser assumed numeric confidence. |
| - Resolution: Added tolerant parsing to accept textual confidence labels. |
| - Prevention: Normalize model outputs and validate schema before policy routing. |
|
|
| ## 6) AML submit runtime NameError |
| - Symptom: Submission crashed before Azure calls with NameError on helper usage. |
| - Root cause: Helper function order caused reference before definition during module execution path. |
| - Resolution: Reintroduced helper in valid scope before use. |
| - Prevention: Add unit tests that execute CLI code path, not only isolated functions. |
|
|
| ## 7) Azure credential chain auth failures |
| - Symptom: DefaultAzureCredential could not obtain token; expired refresh token in chain. |
| - Root cause: Local Azure auth session had expired. |
| - Resolution: Reauthenticated with azd auth login scope for ARM. |
| - Prevention: Reauth as part of deployment checklist and include token health check. |
|
|
| ## 8) Invalid subscription identifier in submit config |
| - Symptom: Azure rejected malformed subscription ID. |
| - Root cause: Placeholder/config mismatch remained in active config source. |
| - Resolution: Replaced with valid GUID subscription and verified workspace tuple. |
| - Prevention: Validate config values using UUID/resource format checks before submit. |
|
|
| ## 9) Invalid AML environment image URI |
| - Symptom: Job creation failed with Invalid dockerImageUri. |
| - Root cause: Generic image reference not accepted by AML environment validation path. |
| - Resolution: Switched to a valid AML-compatible base image. |
| - Prevention: Maintain approved base images list per workspace policy. |
|
|
| ## 10) Azure ML run failed with ModuleNotFoundError for agent_app |
| - Symptom: Job launched but failed at runtime: No module named agent_app. |
| - Root cause: Uploaded code was present but package dependencies/module not installed in remote run environment. |
| - Resolution: Updated job command bootstrap to install package before running module. |
| - Prevention: For code-first jobs, always include pip install step or curated environment image with prebuilt wheel. |
|
|
| ## 11) No component graph visible in AML run |
| - Symptom: User expected node/component graph but saw a single run tile. |
| - Root cause: Submitted workload was a Command job, not a Pipeline job. |
| - Resolution: Added optional pipeline submit mode while retaining command mode. |
| - Prevention: Use command mode for simple execution and pipeline mode for lineage/graph needs. |
|
|
| ## 12) Secret leakage risk in example configs |
| - Symptom: Example files contained real endpoint/key/workspace values. |
| - Root cause: Real runtime values were written into shareable template files. |
| - Resolution: Sanitized examples to placeholders and added local-only patterns to gitignore. |
| - Prevention: Keep sample files sanitized; use *.local.json and environment variables for real values. |
| |
| ## 13) Push readiness blocker for GitHub |
| - Symptom: Cannot push immediately from this workspace. |
| - Root cause: No git remote configured in local repository. |
| - Resolution: Repository prepared for safe push; remote setup and push command still required. |
| - Prevention: Standardize repo bootstrap with origin remote and branch protection setup. |
| |
| ## Recommended production hardening next |
| - Add CI pipeline for unit tests and lint checks. |
| - Add integration smoke test for AML submit in dry-run and optional live mode. |
| - Add structured logging and telemetry correlation IDs. |
| - Add config schema validation and explicit startup diagnostics. |
| - Add secure secret-loading path (env vars/Key Vault) and key rotation runbook. |
| - Add evaluation set with expected recommendations and approval-policy assertions. |
| |
