# Implementation Issues and Resolutions

This document captures the major issues encountered while implementing and operationalizing the revenue-assurance agent workflow, including local execution and Azure ML submission.

## 1) No initial implementation in starter repo
- Symptom: The workspace started as a minimal agent scaffold and had no revenue-assurance domain workflow.
- Root cause: Domain features were not yet built.
- Resolution: Added domain models, reconciliation logic, case management, skills, deterministic investigator, and model-backed agentic investigator.
- Prevention: Keep a milestone checklist and ship in vertical slices (data model -> rule engine -> case -> recommendation).

## 2) Missing Python dependencies
- Symptom: Runtime/import failures while enabling model-backed workflow and AML submit path.
- Root cause: Required SDKs were not yet installed.
- Resolution: Added and used dependencies for OpenAI and Azure ML stack.
- Prevention: Keep dependency declarations in pyproject and verify with CI test runs.

## 3) Placeholder values treated as real model settings
- Symptom: Model path attempted using invalid placeholder endpoint/key values.
- Root cause: Config accepted placeholder-like values as usable credentials.
- Resolution: Added placeholder detection in config loader and automatic fallback to non-agentic behavior.
- Prevention: Validate config at startup and fail fast or gracefully downgrade by policy.

## 4) Unsupported model request parameter
- Symptom: Upstream model call rejected a hard-coded generation parameter.
- Root cause: Parameter compatibility mismatch with deployed model API.
- Resolution: Removed unsupported parameter and aligned request payload with deployment behavior.
- Prevention: Keep per-model capability checks and a compatibility matrix.

## 5) Non-numeric confidence handling
- Symptom: Agent output parsing was brittle when confidence came as text.
- Root cause: Parser assumed numeric confidence.
- Resolution: Added tolerant parsing to accept textual confidence labels.
- Prevention: Normalize model outputs and validate schema before policy routing.

## 6) AML submit runtime NameError
- Symptom: Submission crashed before Azure calls with NameError on helper usage.
- Root cause: Helper function order caused reference before definition during module execution path.
- Resolution: Reintroduced helper in valid scope before use.
- Prevention: Add unit tests that execute CLI code path, not only isolated functions.

## 7) Azure credential chain auth failures
- Symptom: DefaultAzureCredential could not obtain token; expired refresh token in chain.
- Root cause: Local Azure auth session had expired.
- Resolution: Reauthenticated with azd auth login scope for ARM.
- Prevention: Reauth as part of deployment checklist and include token health check.

## 8) Invalid subscription identifier in submit config
- Symptom: Azure rejected malformed subscription ID.
- Root cause: Placeholder/config mismatch remained in active config source.
- Resolution: Replaced with valid GUID subscription and verified workspace tuple.
- Prevention: Validate config values using UUID/resource format checks before submit.

## 9) Invalid AML environment image URI
- Symptom: Job creation failed with Invalid dockerImageUri.
- Root cause: Generic image reference not accepted by AML environment validation path.
- Resolution: Switched to a valid AML-compatible base image.
- Prevention: Maintain approved base images list per workspace policy.

## 10) Azure ML run failed with ModuleNotFoundError for agent_app
- Symptom: Job launched but failed at runtime: No module named agent_app.
- Root cause: Uploaded code was present but package dependencies/module not installed in remote run environment.
- Resolution: Updated job command bootstrap to install package before running module.
- Prevention: For code-first jobs, always include pip install step or curated environment image with prebuilt wheel.

## 11) No component graph visible in AML run
- Symptom: User expected node/component graph but saw a single run tile.
- Root cause: Submitted workload was a Command job, not a Pipeline job.
- Resolution: Added optional pipeline submit mode while retaining command mode.
- Prevention: Use command mode for simple execution and pipeline mode for lineage/graph needs.

## 12) Secret leakage risk in example configs
- Symptom: Example files contained real endpoint/key/workspace values.
- Root cause: Real runtime values were written into shareable template files.
- Resolution: Sanitized examples to placeholders and added local-only patterns to gitignore.
- Prevention: Keep sample files sanitized; use *.local.json and environment variables for real values.

## 13) Push readiness blocker for GitHub
- Symptom: Cannot push immediately from this workspace.
- Root cause: No git remote configured in local repository.
- Resolution: Repository prepared for safe push; remote setup and push command still required.
- Prevention: Standardize repo bootstrap with origin remote and branch protection setup.

## Recommended production hardening next
- Add CI pipeline for unit tests and lint checks.
- Add integration smoke test for AML submit in dry-run and optional live mode.
- Add structured logging and telemetry correlation IDs.
- Add config schema validation and explicit startup diagnostics.
- Add secure secret-loading path (env vars/Key Vault) and key rotation runbook.
- Add evaluation set with expected recommendations and approval-policy assertions.