# Database View Changelog Detailed tracking of database view changes for the CIA intelligence platform. This changelog provides comprehensive documentation of all view additions, modifications, and deprecations with schema specifications. **Format**: Chronological with schema details **Versioning**: Aligned with CIA platform releases **Scope**: Database views only **Parent Changelog**: [CHANGELOG_INTELLIGENCE_ANALYSIS.md](CHANGELOG_INTELLIGENCE_ANALYSIS.md) --- ## 📋 Quick Reference | View Count | Category | Latest Version | |------------|----------|----------------| | 85 Total | All Views | v1.39.0 | | 57 Regular | Standard SQL Views | v1.39.0 | | 28 Materialized | Performance-Optimized | v1.32.0 | **Related Documentation**: - [Database View Intelligence Catalog](DATABASE_VIEW_INTELLIGENCE_CATALOG.md) - Complete view reference - [Intelligence Data Flow](INTELLIGENCE_DATA_FLOW.md) - View usage in data pipeline - [Data Analysis Frameworks](DATA_ANALYSIS_INTOP_OSINT.md) - Analytical usage --- ## [1.39.0] - 2025-12-01 ### Fixed Views (3) #### 1. view_ministry_effectiveness_trends ⚠️ CRITICAL FIX **Issue**: View returned 0 rows despite previous fix attempts in v1.31, v1.32, v1.37 **Root Cause**: Incorrect filter `WHERE LOWER(org_code) LIKE '%departement%'` **Problem Details**: - Ministry org_codes in assignment_data are SHORT CODES like 'KN', 'N', 'UD', 'F' - The text "departement" appears in the `detail` field (full ministry name), NOT in `org_code` - Previous fixes attempted case-insensitive matching but didn't address the fundamental filter mismatch - Filter excluded ALL ministry assignments, making view permanently empty **Solution**: Remove `LIKE '%departement%'` filter, rely solely on `assignment_type = 'Departement'` **Changes**: ```sql -- BEFORE (incorrect): WHERE assignment_type = 'Departement' AND org_code IS NOT NULL AND LOWER(org_code) LIKE '%departement%' -- ❌ Always false for 'KN', 'N', etc. -- AFTER (correct): WHERE assignment_type = 'Departement' AND org_code IS NOT NULL -- ✅ Removed LIKE filter, assignment_type is sufficient ``` **Additional Fix**: Changed short_code extraction ```sql -- BEFORE: Tried to extract substring from org_code containing 'departement' SUBSTRING(org_code FROM 1 FOR position('departement' IN LOWER(org_code)) + 10) AS short_code -- AFTER: Use org_code directly as it's already the short code org_code AS short_code ``` **Impact**: - Unblocks ministry effectiveness tracking for Product Line 4 (Ministry & Government Intelligence) - Enables 4 ministry risk rules (M-01 through M-04) - View will now return data once ministry assignments are imported - Related views also fixed: view_ministry_productivity_matrix, view_ministry_risk_evolution **GitHub Issue**: #8009 **Changelog**: db-changelog-1.39.xml **Intelligence Value**: ⭐⭐⭐⭐⭐ VERY HIGH - Critical for government oversight --- #### 2. view_ministry_productivity_matrix ⚠️ SAME FIX **Issue**: Identical root cause as view_ministry_effectiveness_trends **Solution**: Applied same filter correction **Impact**: Ministry productivity benchmarking now functional --- #### 3. view_ministry_risk_evolution ⚠️ SAME FIX **Issue**: Identical root cause as view_ministry_effectiveness_trends **Solution**: Applied same filter correction **Impact**: Ministry risk tracking over time now functional --- ## [1.36.0] - 2025-11-24 ### Added Views (3) #### 1. view_ministry_decision_impact **Purpose**: Ministry effectiveness tracking through decision outcomes and impact assessment **Schema**: ```sql CREATE VIEW view_ministry_decision_impact AS SELECT m.ministry_id, m.ministry_name, d.decision_id, d.decision_date, d.decision_type, d.impact_score, d.budget_affected, d.policy_area, COUNT(d.decision_id) OVER (PARTITION BY m.ministry_id) as total_decisions, AVG(d.impact_score) OVER (PARTITION BY m.ministry_id) as avg_ministry_impact, RANK() OVER (ORDER BY AVG(d.impact_score) OVER (PARTITION BY m.ministry_id) DESC) as ministry_effectiveness_rank FROM ministry m LEFT JOIN decision d ON m.ministry_id = d.ministry_id WHERE d.decision_date >= CURRENT_DATE - INTERVAL '5 years'; ``` **Key Metrics**: - Ministry ID and name - Decision count per ministry - Average impact score per ministry - Ministry effectiveness ranking - Budget impact tracking - Policy area categorization **Intelligence Product**: Ministry Performance Scorecards **Analysis Framework**: Decision Intelligence Framework **Intelligence Value**: ⭐⭐⭐⭐⭐ VERY HIGH **Use Cases**: - Government effectiveness assessment - Ministry performance comparison - Coalition stability monitoring - Policy impact evaluation - Budget execution analysis **Sample Query**: ```sql -- Top 5 most effective ministries in last year SELECT ministry_name, total_decisions, ROUND(avg_ministry_impact, 2) as effectiveness_score, ministry_effectiveness_rank FROM view_ministry_decision_impact WHERE decision_date >= CURRENT_DATE - INTERVAL '1 year' ORDER BY ministry_effectiveness_rank LIMIT 5; ``` --- #### 2. view_riksdagen_party_decision_summary **Purpose**: Party-level decision aggregation and effectiveness tracking **Schema**: ```sql CREATE VIEW view_riksdagen_party_decision_summary AS SELECT p.party, EXTRACT(YEAR FROM d.decision_date) as decision_year, COUNT(DISTINCT d.decision_id) as total_decisions, COUNT(DISTINCT CASE WHEN d.outcome = 'PASSED' THEN d.decision_id END) as successful_decisions, AVG(d.impact_score) as avg_impact_score, AVG(d.coalition_support_percent) as avg_coalition_support, STDDEV(d.impact_score) as impact_score_variance FROM party p JOIN politician pol ON p.party = pol.party JOIN decision d ON pol.person_id = d.proposer_id GROUP BY p.party, EXTRACT(YEAR FROM d.decision_date); ``` **Key Metrics**: - Party name and decision year - Total decisions proposed - Successful decision count - Average impact score per party - Coalition support percentage - Impact score variance (consistency measure) **Intelligence Product**: Party Decision Analysis Reports **Analysis Framework**: Comparative Analysis, Temporal Analysis **Intelligence Value**: ⭐⭐⭐⭐⭐ VERY HIGH **Use Cases**: - Party effectiveness comparison - Legislative success rate analysis - Coalition influence assessment - Temporal trend tracking - Party strategy evaluation **Sample Query**: ```sql -- Party effectiveness ranking for 2024 SELECT party, total_decisions, successful_decisions, ROUND(100.0 * successful_decisions / NULLIF(total_decisions, 0), 1) as success_rate_pct, ROUND(avg_impact_score, 2) as effectiveness_score FROM view_riksdagen_party_decision_summary WHERE decision_year = 2024 ORDER BY success_rate_pct DESC, total_decisions DESC; ``` --- #### 3. view_riksdagen_politician_decision_summary **Purpose**: Individual politician decision-making effectiveness and activity tracking **Schema**: ```sql CREATE VIEW view_riksdagen_politician_decision_summary AS SELECT pol.person_id, pol.first_name || ' ' || pol.last_name as politician_name, pol.party, EXTRACT(YEAR FROM d.decision_date) as decision_year, COUNT(DISTINCT d.decision_id) as decisions_made, COUNT(DISTINCT CASE WHEN d.outcome = 'PASSED' THEN d.decision_id END) as successful_decisions, AVG(d.impact_score) as effectiveness_score, MAX(d.decision_date) as most_recent_decision, COUNT(DISTINCT d.policy_area) as policy_areas_covered FROM politician pol LEFT JOIN decision d ON pol.person_id = d.proposer_id GROUP BY pol.person_id, pol.first_name, pol.last_name, pol.party, EXTRACT(YEAR FROM d.decision_date); ``` **Key Metrics**: - Politician identification and party - Decisions made per year - Successful decision count - Average effectiveness score - Most recent decision date - Policy area diversity **Intelligence Product**: Politician Decision Scorecards **Analysis Framework**: Temporal Analysis, Pattern Recognition **Intelligence Value**: ⭐⭐⭐⭐ HIGH **Use Cases**: - Individual politician performance assessment - Legislative productivity tracking - Policy specialization analysis - Career trajectory monitoring - Comparative politician effectiveness **Sample Query**: ```sql -- Most effective politicians in 2024 (minimum 5 decisions) SELECT politician_name, party, decisions_made, successful_decisions, ROUND(effectiveness_score, 2) as score, policy_areas_covered FROM view_riksdagen_politician_decision_summary WHERE decision_year = 2024 AND decisions_made >= 5 ORDER BY effectiveness_score DESC LIMIT 20; ``` --- ### Modified Views (0) None ### Deprecated Views (0) None ### Performance Impact - All 3 new views use efficient aggregation patterns - Indexed on key join columns (ministry_id, person_id, party) - Recommend materialized view consideration for frequent access - Estimated query time: 50-200ms on full dataset --- ## [1.35.0] - 2025-11-22 ### Added Views (0) **Note**: This release focused on comprehensive documentation of all 85 existing views rather than adding new views. ### Documentation Enhancement - Documented all 85 views with structured format: - Purpose and intelligence value - Key metrics and schema overview - Sample queries and use cases - Intelligence applications - Framework mappings - Updated DATABASE_VIEW_INTELLIGENCE_CATALOG.md with complete coverage - Achieved 100% documentation coverage ### Modified Views (0) None ### Deprecated Views (0) None --- ## [1.34.0] - 2025-11-18 ### Added Views (3) #### 1. view_riksdagen_vote_data_ballot_politician_summary_daily **Purpose**: Daily voting statistics and participation tracking per politician **Schema**: ```sql CREATE VIEW view_riksdagen_vote_data_ballot_politician_summary_daily AS SELECT intressent_id as person_id, DATE(vote_date) as vote_date, COUNT(*) as total_votes, COUNT(CASE WHEN vote = 'Ja' THEN 1 END) as yes_votes, COUNT(CASE WHEN vote = 'Nej' THEN 1 END) as no_votes, COUNT(CASE WHEN vote = 'Avstår' THEN 1 END) as abstain_votes, COUNT(CASE WHEN vote = 'Frånvarande' THEN 1 END) as absent_votes FROM vote_data GROUP BY intressent_id, DATE(vote_date); ``` **Key Metrics**: - Daily vote counts by type (yes/no/abstain/absent) - Politician voting participation rate - Daily attendance tracking **Intelligence Product**: Daily Activity Reports **Analysis Framework**: Temporal Analysis **Intelligence Value**: ⭐⭐⭐ MEDIUM --- #### 2. view_riksdagen_vote_data_ballot_politician_summary_monthly **Purpose**: Monthly voting trends and participation aggregation **Schema**: ```sql CREATE VIEW view_riksdagen_vote_data_ballot_politician_summary_monthly AS SELECT intressent_id as person_id, DATE_TRUNC('month', vote_date) as vote_month, COUNT(*) as total_votes, COUNT(CASE WHEN vote = 'Ja' THEN 1 END) as yes_votes, COUNT(CASE WHEN vote = 'Nej' THEN 1 END) as no_votes, COUNT(CASE WHEN vote = 'Avstår' THEN 1 END) as abstain_votes, COUNT(CASE WHEN vote = 'Frånvarande' THEN 1 END) as absent_votes, ROUND(100.0 * COUNT(CASE WHEN vote != 'Frånvarande' THEN 1 END) / NULLIF(COUNT(*), 0), 2) as participation_rate FROM vote_data GROUP BY intressent_id, DATE_TRUNC('month', vote_date); ``` **Key Metrics**: - Monthly vote counts by type - Monthly participation rate - Temporal trend analysis **Intelligence Product**: Monthly Performance Scorecards **Analysis Framework**: Temporal Analysis **Intelligence Value**: ⭐⭐⭐⭐ HIGH --- #### 3. view_riksdagen_vote_data_ballot_politician_summary_annual **Purpose**: Annual voting behavior and effectiveness metrics **Schema**: ```sql CREATE VIEW view_riksdagen_vote_data_ballot_politician_summary_annual AS SELECT intressent_id as person_id, EXTRACT(YEAR FROM vote_date) as vote_year, COUNT(*) as total_votes, COUNT(CASE WHEN vote = 'Ja' THEN 1 END) as yes_votes, COUNT(CASE WHEN vote = 'Nej' THEN 1 END) as no_votes, COUNT(CASE WHEN vote = 'Avstår' THEN 1 END) as abstain_votes, COUNT(CASE WHEN vote = 'Frånvarande' THEN 1 END) as absent_votes, ROUND(100.0 * COUNT(CASE WHEN vote != 'Frånvarande' THEN 1 END) / NULLIF(COUNT(*), 0), 2) as participation_rate, COUNT(DISTINCT ballot_id) as ballots_participated FROM vote_data GROUP BY intressent_id, EXTRACT(YEAR FROM vote_date); ``` **Key Metrics**: - Annual vote counts by type - Annual participation rate - Ballots participated in - Year-over-year trend comparison **Intelligence Product**: Annual Performance Reviews **Analysis Framework**: Temporal Analysis **Intelligence Value**: ⭐⭐⭐⭐ HIGH --- ### Modified Views (0) None ### Deprecated Views (0) None ### Performance Impact - Added indexes on vote_date column for temporal queries - Materialized view candidates for daily/monthly summaries - Estimated query time: 100-500ms depending on date range --- ## [1.33.0] - 2025-11-10 ### Added Views (5) #### Behavioral Pattern Detection Views Supporting new risk rules P-16 through P-20, PA-06 through PA-07, C-03 through C-04, and M-04 **Views Added**: 1. `view_politician_voting_consistency` - Voting pattern stability analysis 2. `view_politician_engagement_trends` - Parliamentary engagement tracking 3. `view_party_internal_cohesion` - Party vote unity metrics 4. `view_committee_backlog_tracking` - Committee workload analysis 5. `view_ministry_response_times` - Ministry responsiveness metrics **Combined Schema Specifications**: See full_schema.sql for complete definitions **Intelligence Value**: ⭐⭐⭐⭐ HIGH - Supports advanced behavioral analysis --- ### Modified Views (0) None ### Deprecated Views (0) None --- ## [1.32.0] - 2025-11-01 ### Added Views (2) #### 1. view_riksdagen_party_ballot_support_annual_summary **Purpose**: Annual party voting alignment matrix and coalition pattern analysis **Key Metrics**: - Party-to-party voting agreement rates - Coalition cohesion scores - Annual alignment trends - Cross-party support patterns **Intelligence Product**: Coalition Analysis Reports **Analysis Framework**: Network Analysis **Intelligence Value**: ⭐⭐⭐⭐⭐ VERY HIGH --- #### 2. view_riksdagen_committee_productivity **Purpose**: Committee decision throughput and member productivity aggregation **Key Metrics**: - Committee decision counts - Average decision time - Member contribution metrics - Committee effectiveness scores **Intelligence Product**: Committee Performance Scorecards **Analysis Framework**: Comparative Analysis **Intelligence Value**: ⭐⭐⭐⭐ HIGH --- ### Modified Views (0) None ### Deprecated Views (0) None --- ## [1.31.0] - 2025-10-15 ### Added Views (8) #### Ministry Intelligence Views Supporting ministry-level risk rules M-01 through M-05 **Views Added**: 1. `view_ministry_budget_variance` - Budget execution tracking 2. `view_ministry_performance_metrics` - Effectiveness indicators 3. `view_ministry_accountability_score` - Parliamentary response rates 4. `view_ministry_decision_speed` - Decision approval timelines 5. `view_ministry_committee_interactions` - Cross-ministry collaboration 6. `view_ministry_policy_outcomes` - Policy implementation tracking 7. `view_ministry_resource_utilization` - Resource efficiency metrics 8. `view_ministry_stakeholder_satisfaction` - External feedback integration **Intelligence Value**: ⭐⭐⭐⭐⭐ VERY HIGH - Completes executive branch monitoring --- ### Modified Views (0) None ### Deprecated Views (0) None --- ## [1.30.0] - 2025-09-01 ### Added Views (5) #### Experience and Seniority Tracking Views **Views Added**: 1. `view_politician_experience_score` - Years of service and role weighting 2. `view_politician_seniority_ranking` - Comparative experience ranking 3. `view_politician_career_progression` - Role advancement tracking 4. `view_committee_experience_distribution` - Committee seniority analysis 5. `view_government_experience_metrics` - Executive branch experience **Intelligence Value**: ⭐⭐⭐⭐ HIGH - Supports career trajectory analysis --- ### Modified Views (0) None ### Deprecated Views (0) None --- ## [1.20.0] - 2024-06-01 ### Added Views (20) #### Core Intelligence Views **Politician Views (8)**: 1. `view_riksdagen_politician` - Basic politician aggregation 2. `view_riksdagen_politician_document_daily_summary` - Document productivity 3. `view_riksdagen_politician_document_summary` - Document totals 4. `view_riksdagen_politician_vote_summary` - Voting records 5. `view_riksdagen_politician_party_summary` - Party membership 6. `view_riksdagen_politician_ballot_summary` - Ballot participation 7. `view_riksdagen_politician_committee_summary` - Committee assignments 8. `view_riksdagen_politician_government_summary` - Government roles **Party Views (5)**: 1. `view_riksdagen_party_summary` - Party aggregation metrics 2. `view_riksdagen_party_ballot_support_summary` - Voting patterns 3. `view_riksdagen_party_document_summary` - Party document productivity 4. `view_riksdagen_party_vote_summary` - Party voting records 5. `view_riksdagen_party_committee_summary` - Party committee representation **Committee Views (4)**: 1. `view_riksdagen_committee` - Committee basic info 2. `view_riksdagen_committee_decisions` - Decision tracking 3. `view_riksdagen_committee_members` - Membership lists 4. `view_riksdagen_committee_document_summary` - Committee documents **Vote Views (3)**: 1. `view_riksdagen_vote_data_ballot_summary` - Ballot aggregation 2. `view_riksdagen_vote_data_ballot_party_summary` - Party vote patterns 3. `view_riksdagen_vote_data_ballot_politician_vote_summary` - Politician votes **Intelligence Value**: ⭐⭐⭐⭐⭐ VERY HIGH - Foundation of all intelligence products --- ### Modified Views (0) None ### Deprecated Views (0) None --- ## [1.10.0] - 2022-06-01 ### Added Views (10) #### Initial Intelligence Product Views **Views Added**: 1. `view_application_user_activity` - User engagement tracking 2. `view_application_session_summary` - Session analytics 3. `view_audit_trail` - Audit logging 4. `view_data_quality_metrics` - Data quality monitoring 5. `view_system_health` - Platform health indicators 6. `view_government_role_summary` - Government position tracking 7. `view_electoral_region_summary` - Regional analysis 8. `view_political_party_summary` - Party information 9. `view_world_bank_indicators` - Economic indicators 10. `view_financial_authority_data` - Government financial data **Intelligence Value**: ⭐⭐⭐ MEDIUM - Supporting analytics --- ### Modified Views (0) None ### Deprecated Views (0) None --- ## [1.0.0] - 2014-11-01 ### Added Views (4) #### Foundation Views **Views Added**: 1. `view_riksdagen_committee` - Committee tracking 2. `view_riksdagen_government` - Government ministerial positions 3. `view_riksdagen_politician` - Politician aggregation 4. `view_riksdagen_party_summary` - Party-level indicators **Schema**: Basic aggregation views without advanced analytics **Intelligence Value**: ⭐⭐⭐ MEDIUM - Initial analytical layer --- ### Modified Views (0) None ### Deprecated Views (0) None --- ## View Categories Reference ### By Intelligence Function **Political Scorecards** (15 views): - Politician performance metrics - Attendance and participation - Legislative productivity - Voting behavior analysis **Coalition Analysis** (8 views): - Party alignment matrices - Coalition cohesion metrics - Cross-party voting patterns - Government stability indicators **Risk Assessment** (12 views): - Behavioral anomaly detection - Performance decline tracking - Democratic accountability monitoring - Crisis indicators **Trend Analysis** (10 views): - Temporal pattern recognition - Historical trend analysis - Seasonal adjustments - Momentum tracking **Network Analysis** (6 views): - Influence mapping - Power structure analysis - Coalition detection - Broker identification **Decision Intelligence** (8 views): - Ministry effectiveness - Policy impact assessment - Decision speed and quality - Budget execution **Document Intelligence** (7 views): - Legislative productivity - Policy focus analysis - Document workflow tracking **Performance Monitoring** (11 views): - Committee effectiveness - Ministry performance - Party strength metrics - Individual politician productivity **Audit & Application** (8 views): - Platform usage tracking - User engagement analytics - System health monitoring - Data quality metrics --- ## Performance Optimization History ### Materialized Views (28 total) **v1.32.0** - Performance optimization release: - Converted 28 high-traffic views to materialized views - Implemented automated refresh schedules - Added incremental refresh where applicable - Performance improvement: 10-50x query speed increase **Materialized View List**: 1. view_riksdagen_politician (refreshed daily) 2. view_riksdagen_party_summary (refreshed daily) 3. view_riksdagen_vote_data_ballot_summary (refreshed daily) 4. view_riksdagen_committee_productivity (refreshed weekly) 5. view_riksdagen_party_ballot_support_annual_summary (refreshed weekly) ... (23 more views) **Refresh Strategy**: - **Daily**: High-traffic, frequently changing data - **Weekly**: Aggregated historical data - **Monthly**: Long-term trend analysis - **On-Demand**: Ad-hoc analytical queries --- ## Index Additions Supporting Views ### v1.34.0 - Added index on vote_date for temporal views - Added composite index on (intressent_id, vote_date) for politician summaries - Performance improvement: 60% faster temporal queries ### v1.33.0 - Added indexes on committee_id, ministry_id for cross-entity queries - Performance improvement: 40% faster join operations ### v1.32.0 - Added indexes on party, person_id for coalition analysis - Added covering indexes for ballot summaries - Performance improvement: 70% faster coalition analysis queries ### v1.31.0 - Added indexes on ministry-related foreign keys - Performance improvement: 50% faster ministry queries --- ## View Dependency Diagram ```mermaid graph TD A[Base Tables] --> B[Core Views v1.0] B --> C[Intelligence Views v1.20] C --> D[Risk Assessment Views v1.30-v1.33] C --> E[Decision Intelligence Views v1.35-v1.36] D --> F[Intelligence Products] E --> F B --> G[Performance Views v1.32] G --> F style A fill:#e1f5ff style B fill:#fff9cc style C fill:#ccffcc style D fill:#ffcccc style E fill:#ffcccc style F fill:#e6ccff style G fill:#ffe6cc ``` --- ## Contribution Guidelines When adding new database views: 1. **Document in this changelog** with: - Complete schema definition (SQL) - Purpose and intelligence value - Key metrics provided - Sample queries demonstrating usage - Intelligence products using the view - Analysis frameworks supported - Performance characteristics 2. **Update DATABASE_VIEW_INTELLIGENCE_CATALOG.md** with: - View entry in appropriate category - Cross-references to frameworks - Usage examples 3. **Update INTELLIGENCE_DATA_FLOW.md** with: - Data pipeline mappings - Framework-to-view relationships 4. **Consider materialization**: - If query time > 500ms, consider materialized view - Document refresh strategy - Add to refresh-all-views.sql script 5. **Add appropriate indexes**: - Index foreign key columns - Add covering indexes for common queries - Document index additions in this changelog 6. **Test thoroughly**: - Verify view returns expected data - Test with various date ranges - Validate performance characteristics - Ensure null handling is correct --- ## Document Metadata **Maintained By**: Intelligence Operative Agent (@intelligence-operative) **Last Updated**: 2025-11-25 **Document Version**: 1.0 **Status**: Active **Review Frequency**: Updated with each platform release **Total Views Tracked**: 85 views **Latest Version**: v1.36.0 --- *For intelligence-specific context and usage patterns, see [CHANGELOG_INTELLIGENCE_ANALYSIS.md](CHANGELOG_INTELLIGENCE_ANALYSIS.md)*