+
+
AI Code Maintainability Scoring & Refactoring Engine
+
Revision Notes — Light-touch interview prep (2–3 sentences + one technical detail per topic)
+
+
+Scope reminder: this project was deliberately scoped as light-touch, not a deep-dive like Urdu Sentiment or Medical Image. Use the "One-Line Summary" and "Quick Answers" sections as your primary prep. The deeper sections exist so you're not caught flat-footed on a follow-up, not so you memorize all of it.
+
+
+
What Is This Project? (Elevator Pitch)
+
+"I built a two-phase AI system that scores the structural quality of Python code and then autonomously refactors risky code to improve it. Phase 1 uses AST parsing + a Random Forest classifier to predict a maintainability risk score. Phase 2 uses a CodeT5 deep learning model to generate and iteratively select better versions of risky code."
+
+
+
The Problem It Solves
+
+Traditional linters (PyLint, Flake8) use rigid, hand-written rules. This engine instead learns what "risky" structure looks like from data — deep nesting, high complexity, large functions — and then goes a step further than any linter by actually generating improved code, not just flagging problems.
+
+
+
Full Pipeline Flow
+
Input Code
+ │
+ ▼
+PHASE 1 — Evaluator
+ AST Parsing → Feature Extraction → ML Scoring → Explanation
+ │
+ ▼
+Risk Score (0–100) + Top 3 Reasons
+ │
+ ▼
+PHASE 2 — Refactorer (only if risky)
+ Generate Candidates (CodeT5) → Validate → Re-score via Phase 1 → Select Best
+ │ (loop until target score or max iterations)
+ ▼
+Improved Code
+
+
Phase 1 — The Evaluator
+
+
1. AST Analyzer — 11 Structural Features
+
+Parses code into an Abstract Syntax Tree (not line-by-line text reading — actual structural traversal via
ast.walk()) and extracts 11 signals:
+
+| Feature | What it measures |
|---|
+| max_nesting_depth | Strongest predictor — deepest if/for/while/try nesting |
+| cyclomatic_complexity | 1 + every decision point (if/for/while/except/with/assert/bool-op) |
+| avg_function_length | Mean lines per function |
+| num_functions, num_loops, num_if, num_try_except, num_return | Raw structural counts |
+| line_count | Total file size |
+| recursion_flag | 1 if any function calls itself, else 0 |
+| global_variable_count | Count of global declarations — hidden state / tight coupling |
+
+
One technical detail worth knowing cold: nesting depth is computed via a recursive traversal that increments depth only when it enters a defined set of "nesting nodes" (
If, For, While, With, Try, FunctionDef, AsyncFunctionDef, ClassDef) — everything else keeps the current depth.
+
+
+
2. Dataset Generation
+
+Synthetic dataset generated from code templates, not scraped real-world code. 220 total samples, perfectly balanced: 110 labeled Clean (0), 110 labeled Risky (1). Reproducible via random.seed(42) — same dataset every run.
+
+
+
3. Feature Pipeline
+
+Converts the feature dictionary into a fixed-order numeric vector (FEATURE_SCHEMA defines the order — this order must stay identical between training and inference or predictions break silently). Scales with StandardScaler (mean=0, std=1) so no single feature like line_count dominates just because its raw numbers are bigger. Scaler is pickled and reused at inference time — never re-fit on new data.
+
+
+
4. Model Training
+
+Two models trained side by side on an 80/20 stratified split (176 train / 44 test):
+
+| Model | Key hyperparameters |
|---|
+| Random Forest primary / production | 200 trees, max_depth=10, min_samples_split=4, class_weight="balanced" |
+| XGBoost comparison only | 200 estimators, max_depth=6, learning_rate=0.1, subsample=0.8 |
+
+
Random Forest is the one actually used in the live Scoring API — XGBoost is trained and evaluated for comparison but not deployed.
+
+
+No saved accuracy/F1 number exists in the repo — the evaluation function prints it live but doesn't persist it to a file. Run python model_trainer.py before your interview and note the actual number rather than guessing one.
+
+
+
5. Explanation Engine
+
+Answers "why is this risky" without recomputing anything new. Logic: for each of the 11 features, check if the code's actual value exceeds a fixed threshold (e.g. max_nesting_depth > 3, cyclomatic_complexity > 5, global_variable_count > 1). Of the features that exceed threshold, rank by the model's feature importance and return the top 3 as plain-English sentences (e.g. "Deep nesting detected (depth: 7)").
+
+
+
6. Scoring API — Final Output
+
+Single entry point:
evaluate(code). Runs the model's predicted probability of the "risky" class × 100 as the risk score.
+
+| Risk Score | Level |
|---|
+| 0–30 | Low |
+| 31–60 | Medium |
+| 61–100 | High |
+
+
{
+ "risk_score": 82,
+ "risk_level": "High",
+ "confidence": 0.82,
+ "top_risk_factors": ["Deep nesting detected (depth: 7)", ...]
+}
+Properties worth naming if asked: deterministic, stateless, fast (no training happens at inference time).
+
+
+
Phase 2 — The Refactorer
+
+
+Only runs on code Phase 1 flagged as risky. Uses CodeT5 (a deep learning code-generation model) to produce multiple refactored candidates, then uses Phase 1's own evaluate() as a reward/ranking function to pick the best one.
+
+
+
1. Candidate Generation — 3 Strategies
+
+Generates one candidate per strategy, each with a different prompt and temperature:
+
+| Strategy | Temperature |
|---|
+| Improve readability and clarity | 0.5 (conservative) |
+| Reduce nesting and simplify logic | 0.7 |
+| Refactor for strict maintainability best practices | 0.85 (more creative) |
+
+
One technical detail: temperature controls how much the model deviates from the "safe" rewrite — low temperature stays close to minimal edits, high temperature takes bigger structural risks.
+
+
+
2. Validation → Selection → Iteration
+
+Each candidate is checked to actually compile (no syntax errors) before it's even considered. Valid candidates are re-scored by feeding them back through the Phase 1 evaluate() API, and the lowest risk score wins that round.
+
+Iterative loop defaults: target_score=20, max_iterations=3. Stops early if the target is hit, or after 3 rounds regardless. Starts by initializing the best-known score to float('inf') — a simple trick meaning "anything found is automatically an improvement over nothing."
+
+
+
Tech Stack
+
+
+| Component | Technology |
|---|
+| Structural Analysis | Python ast module |
+| Risk Prediction | Random Forest (primary), XGBoost (comparison) — scikit-learn |
+| Refactoring | CodeT5 (Transformers / PyTorch) |
+| API | Unified main.py entry point + api_server.py |
+| Deployment | Live at ai-code-maintainability.hmuhammadusman.com |
+
+
+
+
Numbers to Remember
+
+
+| Structural features extracted | 11 |
+| Total dataset samples | 220 (110 Clean / 110 Risky — balanced) |
+| Train / test split | 176 / 44 (80/20, stratified) |
+| Random Forest trees | 200 (max_depth=10) |
+| Risk score range | 0–100 |
+| Top risk factors surfaced | 3 |
+| Refactor candidate strategies | 3 (temps 0.5 / 0.7 / 0.85) |
+| Default optimizer target / max iterations | 20 / 3 |
+| Reproducibility seed | 42 |
+
+
+
+
Quick Answers — Likely Questions
+
+
"Why Random Forest over a simpler rule-based linter?"
+
Rule-based linters need every threshold hand-tuned per rule. A trained model learns which combinations of features actually correlate with risk from data, and can weigh 11 signals together instead of checking them independently.
+
+
"Why two models (RF + XGBoost) if only one is deployed?"
+
Comparison during development — training both and comparing accuracy tells you whether the extra complexity of boosting is worth it on this dataset size before committing to one in production.
+
+
"Isn't 220 samples very small for ML?"
+
Yes — be upfront about this if asked. It's a synthetic, template-generated dataset, not real-world code, which is both the honest limitation and a natural "what I'd improve next" answer (train on real open-source repos labeled by actual maintainability metrics).
+
+
"How does Phase 2 know a candidate is actually better, not just different?"
+
It doesn't trust the language model's own judgment — every candidate gets re-scored through the exact same Phase 1 evaluate() pipeline used on the original code, so "better" is measured by the same objective risk score, not by how the refactor looks.
+
+
"What would you improve?"
+
Real-world training data instead of synthetic templates; persisting evaluation metrics instead of only printing them; and multi-file / cross-function analysis, since right now every file is scored in isolation.
+
+
One-Line Summary to Open With
+
+"I built a two-phase AI system — Phase 1 uses AST parsing and a Random Forest model to score Python code's maintainability risk out of 100 and explain why, and Phase 2 uses a CodeT5 deep learning model to iteratively generate and select better refactored versions of risky code, using the Phase 1 score itself as the judge of improvement."
+
+
+