Spaces:
Running on CPU Upgrade
Running on CPU Upgrade
| <html lang=en dir=ltr class="docs-wrapper plugin-docs plugin-id-default docs-version-current docs-doc-page docs-doc-id-guides/search-patterns" data-has-hydrated=false><head><meta charset=UTF-8><meta name=generator content="Docusaurus v3.10.0"><title data-rh=true>Scale and Search Patterns: End-to-End Civic Tech Projects | Open Navigator</title><meta data-rh=true name=viewport content="width=device-width, initial-scale=1.0"/><meta data-rh=true property=og:image content=https://www.communityone.com/img/docusaurus-social-card.jpg /><meta data-rh=true name=twitter:image content=https://www.communityone.com/img/docusaurus-social-card.jpg /><meta data-rh=true property=og:url content=https://www.communityone.com/docs/guides/search-patterns /><meta data-rh=true property=og:locale content=en /><meta data-rh=true name=docusaurus_locale content=en /><meta data-rh=true name=docsearch:language content=en /><meta data-rh=true name=keywords content="civic engagement, policy tracking, meeting minutes, nonprofit tracking, municipal government, advocacy, open data, local government"/><meta data-rh=true property=og:type content=website /><meta data-rh=true property=og:site_name content="Open Navigator"/><meta data-rh=true name=twitter:card content=summary_large_image /><meta data-rh=true name=docusaurus_version content=current /><meta data-rh=true name=docusaurus_tag content=docs-default-current /><meta data-rh=true name=docsearch:version content=current /><meta data-rh=true name=docsearch:docusaurus_tag content=docs-default-current /><meta data-rh=true property=og:title content="Scale and Search Patterns: End-to-End Civic Tech Projects | Open Navigator"/><meta data-rh=true name=description content="This guide analyzes 6 additional civic tech projects focused on full-stack deployments, large-scale data aggregation, and public search portals. These complement our existing integration (Civic Scraper, City Scrapers, CDP, Engagic, Councilmatic) with new patterns for:"/><meta data-rh=true property=og:description content="This guide analyzes 6 additional civic tech projects focused on full-stack deployments, large-scale data aggregation, and public search portals. These complement our existing integration (Civic Scraper, City Scrapers, CDP, Engagic, Councilmatic) with new patterns for:"/><link data-rh=true rel=icon href=/img/favicon.ico /><link data-rh=true rel=canonical href=https://www.communityone.com/docs/guides/search-patterns /><link data-rh=true rel=alternate href=https://www.communityone.com/docs/guides/search-patterns hreflang=en /><link data-rh=true rel=alternate href=https://www.communityone.com/docs/guides/search-patterns hreflang=x-default /><script data-rh=true type=application/ld+json>{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","item":"https://www.communityone.com/docs/for-developers","name":"Developers & Technical Users","position":1},{"@type":"ListItem","item":"https://www.communityone.com/docs/guides/search-patterns","name":"Scale and Search Patterns: End-to-End Civic Tech Projects","position":2}]}</script><link rel=alternate type=application/rss+xml href=/blog/rss.xml title="Open Navigator RSS Feed"><link rel=alternate type=application/atom+xml href=/blog/atom.xml title="Open Navigator Atom Feed"><link rel=preconnect href=https://www.google-analytics.com><link rel=preconnect href=https://www.googletagmanager.com><script async src="https://www.googletagmanager.com/gtag/js?id=G-5EQV815915"></script><script>function gtag(){dataLayer.push(arguments)}window.dataLayer=window.dataLayer||[],gtag("js",new Date),gtag("config","G-5EQV815915",{anonymize_ip:!0})</script><link rel=stylesheet href=/assets/css/styles.c89d6b2d.css /><script src=/assets/js/runtime~main.c8fa085e.js defer></script><script src=/assets/js/main.6e24e536.js defer></script></head><body><svg style="display: none;"><defs> | |
| <symbol id=theme-svg-external-link viewBox="0 0 24 24"><path fill=currentColor d="M21 13v10h-21v-19h12v2h-10v15h17v-8h2zm3-12h-10.988l4.035 4-6.977 7.07 2.828 2.828 6.977-7.07 4.125 4.172v-11z"/></symbol> | |
| </defs></svg> | |
| <script>!function(){var t=function(){try{return new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}}()||function(){try{return window.localStorage.getItem("theme-7e9")}catch(t){}}();document.documentElement.setAttribute("data-theme",t||(window.matchMedia("(prefers-color-scheme: dark)").matches?"dark":"light")),document.documentElement.setAttribute("data-theme-choice",t||"system")}(),function(){try{for(var[t,e]of new URLSearchParams(window.location.search).entries())if(t.startsWith("docusaurus-data-")){var a=t.replace("docusaurus-data-","data-");document.documentElement.setAttribute(a,e)}}catch(t){}}()</script><div id=__docusaurus><link rel=preload as=image href=/img/communityone_logo.svg /><script type=application/ld+json>{"@context":"https://schema.org","@type":"Organization","address":{"@type":"PostalAddress","addressCountry":"US","addressLocality":"Tuscaloosa","addressRegion":"AL","postalCode":"35406","streetAddress":"5617 Lakeridge Court"},"contactPoint":{"@type":"ContactPoint","availableLanguage":["English"],"contactType":"Customer Service","email":"johnbowyer@communityone.com"},"description":"Track 90,000+ jurisdictions, 1.8M nonprofits, and analyze meeting minutes with AI. The open path to everything local.","email":"johnbowyer@communityone.com","legalName":"CommunityOne","logo":"https://www.communityone.com/img/communityone_logo.svg","name":"CommunityOne","sameAs":["https://www.facebook.com/communityone","https://www.instagram.com/communityone","https://twitter.com/communityone","https://www.linkedin.com/company/communityone","https://www.youtube.com/@communityone","https://discord.gg/communityone","https://github.com/getcommunityone/open-navigator"],"url":"https://www.communityone.com"}</script><script type=application/ld+json>{"@context":"https://schema.org","@type":"WebSite","alternateName":"CommunityOne Open Navigator","description":"AI-powered civic engagement platform tracking jurisdictions, nonprofits, and government meetings","name":"Open Navigator","potentialAction":{"@type":"SearchAction","query-input":"required name=search_term_string","target":{"@type":"EntryPoint","urlTemplate":"https://www.communityone.com/search?q={search_term_string}"}},"url":"https://www.communityone.com"}</script><script type=application/ld+json>{"@context":"https://schema.org","@type":"SoftwareApplication","aggregateRating":{"@type":"AggregateRating","ratingCount":"1","ratingValue":"5"},"applicationCategory":"BusinessApplication","description":"Track 90,000+ jurisdictions, 1.8M nonprofits, and analyze meeting minutes with AI","featureList":["Track 90,000+ jurisdictions","Monitor 1.8M nonprofits","Analyze meeting minutes","Legislative bill tracking","Campaign finance data"],"name":"Open Navigator","offers":{"@type":"Offer","price":"0","priceCurrency":"USD"},"operatingSystem":"Web","screenshot":"https://www.communityone.com/img/docusaurus-social-card.jpg","softwareVersion":"1.0.0"}</script><div role=region aria-label="Skip to main content"><a class=skipToContent_fXgn href=#__docusaurus_skipToContent_fallback>Skip to main content</a></div><nav aria-label=Main class="theme-layout-navbar navbar navbar--fixed-top"><div class=navbar__inner><div class="theme-layout-navbar-left navbar__items"><button aria-label="Toggle navigation bar" aria-expanded=false class="navbar__toggle clean-btn" type=button><svg width=30 height=30 viewBox="0 0 30 30" aria-hidden=true><path stroke=currentColor stroke-linecap=round stroke-miterlimit=10 stroke-width=2 d="M4 7h22M4 15h22M4 23h22"/></svg></button><a href=https://www.communityone.com target=_self rel="noopener noreferrer" class=navbar__brand><div class=navbar__logo><img src=/img/communityone_logo.svg alt="CommunityOne Logo" class="themedComponent_mlkZ themedComponent--light_NVdE"/><img src=/img/communityone_logo.svg alt="CommunityOne Logo" class="themedComponent_mlkZ themedComponent--dark_xIcU"/></div><b class="navbar__title text--truncate">Open Navigator Home</b></a><a class="navbar__item navbar__link" href=/docs/intro>Getting Started</a><a class="navbar__item navbar__link" href=/docs/for-families>Families & Individuals</a><a class="navbar__item navbar__link" href=/docs/for-advocates>Policy Makers</a><a class="navbar__item navbar__link" href=/docs/for-developers>Developers</a><a class="navbar__item navbar__link" href=/docs/data-sources/citations>Data and Terms</a><a class="navbar__item navbar__link" href=/blog>Blog</a></div><div class="theme-layout-navbar-right navbar__items navbar__items--right"><a href=https://github.com/getcommunityone/open-navigator-for-engagement target=_blank rel="noopener noreferrer" class="navbar__item navbar__link">GitHub<svg width=13.5 height=13.5 aria-label="(opens in new tab)" class=iconExternalLink_nPIU><use href=#theme-svg-external-link /></svg></a><div class="toggle_vylO colorModeToggle_DEke"><button class="clean-btn toggleButton_gllP toggleButtonDisabled_aARS" type=button disabled title="system mode" aria-label="Switch between dark and light mode (currently system mode)"><svg viewBox="0 0 24 24" width=24 height=24 aria-hidden=true class="toggleIcon_g3eP lightToggleIcon_pyhR"><path fill=currentColor d="M12,9c1.65,0,3,1.35,3,3s-1.35,3-3,3s-3-1.35-3-3S10.35,9,12,9 M12,7c-2.76,0-5,2.24-5,5s2.24,5,5,5s5-2.24,5-5 S14.76,7,12,7L12,7z M2,13l2,0c0.55,0,1-0.45,1-1s-0.45-1-1-1l-2,0c-0.55,0-1,0.45-1,1S1.45,13,2,13z M20,13l2,0c0.55,0,1-0.45,1-1 s-0.45-1-1-1l-2,0c-0.55,0-1,0.45-1,1S19.45,13,20,13z M11,2v2c0,0.55,0.45,1,1,1s1-0.45,1-1V2c0-0.55-0.45-1-1-1S11,1.45,11,2z M11,20v2c0,0.55,0.45,1,1,1s1-0.45,1-1v-2c0-0.55-0.45-1-1-1C11.45,19,11,19.45,11,20z M5.99,4.58c-0.39-0.39-1.03-0.39-1.41,0 c-0.39,0.39-0.39,1.03,0,1.41l1.06,1.06c0.39,0.39,1.03,0.39,1.41,0s0.39-1.03,0-1.41L5.99,4.58z M18.36,16.95 c-0.39-0.39-1.03-0.39-1.41,0c-0.39,0.39-0.39,1.03,0,1.41l1.06,1.06c0.39,0.39,1.03,0.39,1.41,0c0.39-0.39,0.39-1.03,0-1.41 L18.36,16.95z M19.42,5.99c0.39-0.39,0.39-1.03,0-1.41c-0.39-0.39-1.03-0.39-1.41,0l-1.06,1.06c-0.39,0.39-0.39,1.03,0,1.41 s1.03,0.39,1.41,0L19.42,5.99z M7.05,18.36c0.39-0.39,0.39-1.03,0-1.41c-0.39-0.39-1.03-0.39-1.41,0l-1.06,1.06 c-0.39,0.39-0.39,1.03,0,1.41s1.03,0.39,1.41,0L7.05,18.36z"/></svg><svg viewBox="0 0 24 24" width=24 height=24 aria-hidden=true class="toggleIcon_g3eP darkToggleIcon_wfgR"><path fill=currentColor d="M9.37,5.51C9.19,6.15,9.1,6.82,9.1,7.5c0,4.08,3.32,7.4,7.4,7.4c0.68,0,1.35-0.09,1.99-0.27C17.45,17.19,14.93,19,12,19 c-3.86,0-7-3.14-7-7C5,9.07,6.81,6.55,9.37,5.51z M12,3c-4.97,0-9,4.03-9,9s4.03,9,9,9s9-4.03,9-9c0-0.46-0.04-0.92-0.1-1.36 c-0.98,1.37-2.58,2.26-4.4,2.26c-2.98,0-5.4-2.42-5.4-5.4c0-1.81,0.89-3.42,2.26-4.4C12.92,3.04,12.46,3,12,3L12,3z"/></svg><svg viewBox="0 0 24 24" width=24 height=24 aria-hidden=true class="toggleIcon_g3eP systemToggleIcon_QzmC"><path fill=currentColor d="m12 21c4.971 0 9-4.029 9-9s-4.029-9-9-9-9 4.029-9 9 4.029 9 9 9zm4.95-13.95c1.313 1.313 2.05 3.093 2.05 4.95s-0.738 3.637-2.05 4.95c-1.313 1.313-3.093 2.05-4.95 2.05v-14c1.857 0 3.637 0.737 4.95 2.05z"/></svg></button></div><div class=navbarSearchContainer_Bca1></div></div></div><div role=presentation class=navbar-sidebar__backdrop></div></nav><div id=__docusaurus_skipToContent_fallback class="theme-layout-main main-wrapper mainWrapper_z2l0"><div class=docsWrapper_hBAB><button aria-label="Scroll back to top" class="clean-btn theme-back-to-top-button backToTopButton_sjWU" type=button></button><div class=docRoot_UBD9><aside class="theme-doc-sidebar-container docSidebarContainer_YfHR"><div class=sidebarViewport_aRkj><div class=sidebar_njMd><nav aria-label="Docs sidebar" class="menu thin-scrollbar menu_SIkG"><ul class="theme-doc-sidebar-menu menu__list"><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item"><div class=menu__list-item-collapsible><a class="categoryLink_byQd menu__link menu__link--sublist menu__link--sublist-caret" role=button aria-expanded=true href=/docs/intro><span title="Getting Started" class=categoryLinkLabel_W154>Getting Started</span></a></div><ul class=menu__list><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class=menu__link tabindex=0 href=/docs/intro><span title=Introduction class=linkLabel_WmDU>Introduction</span></a><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class=menu__link tabindex=0 href=/docs/open-navigator><span title="Open Navigator" class=linkLabel_WmDU>Open Navigator</span></a></ul><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item"><div class=menu__list-item-collapsible><a class="categoryLink_byQd menu__link menu__link--sublist" href=/docs/for-families><span title="Families & Individuals" class=categoryLinkLabel_W154>Families & Individuals</span></a><button aria-label="Collapse sidebar category 'Families & Individuals'" aria-expanded=true type=button class="clean-btn menu__caret"></button></div><ul class=menu__list><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-2 menu__list-item menu__list-item--collapsed"><div class=menu__list-item-collapsible><a class="categoryLink_byQd menu__link menu__link--sublist menu__link--sublist-caret" role=button aria-expanded=false tabindex=0 href=/docs/families/community-events><span title="Resources for Families" class=categoryLinkLabel_W154>Resources for Families</span></a></div><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class=menu__link tabindex=0 href=/docs/open-navigator><span title="Getting Started with Open Navigator" class=linkLabel_WmDU>Getting Started with Open Navigator</span></a><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-2 menu__list-item"><a class=menu__link tabindex=0 href=/docs/data-sources/citations><span title="Data and Citations" class=linkLabel_WmDU>Data and Citations</span></a></ul><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item"><div class=menu__list-item-collapsible><a class="categoryLink_byQd menu__link menu__link--sublist" href=/docs/for-advocates><span title="Policy Makers & Advocates" class=categoryLinkLabel_W154>Policy Makers & Advocates</span></a><button aria-label="Collapse sidebar category 'Policy Makers & Advocates'" aria-expanded=true type=button class="clean-btn menu__caret"></button></div><ul class=menu__list><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-2 menu__list-item menu__list-item--collapsed"><div class=menu__list-item-collapsible><a class="categoryLink_byQd menu__link menu__link--sublist menu__link--sublist-caret" role=button aria-expanded=false tabindex=0 href=/docs/data-sources/overview><span title="Understanding the Data" class=categoryLinkLabel_W154>Understanding the Data</span></a></div><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-2 menu__list-item menu__list-item--collapsed"><div class=menu__list-item-collapsible><a class="categoryLink_byQd menu__link menu__link--sublist menu__link--sublist-caret" role=button aria-expanded=false tabindex=0 href=/docs/guides/political-economy><span title="Analysis & Strategy" class=categoryLinkLabel_W154>Analysis & Strategy</span></a></div><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-2 menu__list-item menu__list-item--collapsed"><div class=menu__list-item-collapsible><a class="categoryLink_byQd menu__link menu__link--sublist menu__link--sublist-caret" role=button aria-expanded=false tabindex=0 href=/docs/case-studies/tuscaloosa-complete><span title="Real-World Examples" class=categoryLinkLabel_W154>Real-World Examples</span></a></div></ul><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-1 menu__list-item"><div class=menu__list-item-collapsible><a class="categoryLink_byQd menu__link menu__link--sublist menu__link--active" href=/docs/for-developers><span title="Developers & Technical Users" class=categoryLinkLabel_W154>Developers & Technical Users</span></a><button aria-label="Collapse sidebar category 'Developers & Technical Users'" aria-expanded=true type=button class="clean-btn menu__caret"></button></div><ul class=menu__list><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-2 menu__list-item menu__list-item--collapsed"><div class=menu__list-item-collapsible><a class="categoryLink_byQd menu__link menu__link--sublist menu__link--sublist-caret" role=button aria-expanded=false tabindex=0 href=/docs/quickstart><span title="Setup & Installation" class=categoryLinkLabel_W154>Setup & Installation</span></a></div><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-2 menu__list-item menu__list-item--collapsed"><div class=menu__list-item-collapsible><a class="categoryLink_byQd menu__link menu__link--sublist menu__link--sublist-caret" role=button aria-expanded=false tabindex=0 href=/docs/data-sources/citations><span title="Data Sources (Technical)" class=categoryLinkLabel_W154>Data Sources (Technical)</span></a></div><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-2 menu__list-item"><div class=menu__list-item-collapsible><a class="categoryLink_byQd menu__link menu__link--sublist menu__link--sublist-caret menu__link--active" role=button aria-expanded=true tabindex=0 href=/docs/guides/jurisdiction-setup><span title="How-To Guides" class=categoryLinkLabel_W154>How-To Guides</span></a></div><ul class=menu__list><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-3 menu__list-item"><a class=menu__link tabindex=0 href=/docs/guides/jurisdiction-setup><span title="Jurisdiction Discovery - Quick Start Guide" class=linkLabel_WmDU>Jurisdiction Discovery - Quick Start Guide</span></a><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-3 menu__list-item"><a class=menu__link tabindex=0 href=/docs/guides/huggingface-publishing><span title="HuggingFace Dataset Publishing Guide" class=linkLabel_WmDU>HuggingFace Dataset Publishing Guide</span></a><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-3 menu__list-item"><a class=menu__link tabindex=0 href=/docs/guides/huggingface-quickstart><span title="🚀 QUICK START: FREE STORAGE WITH HUGGING FACE" class=linkLabel_WmDU>🚀 QUICK START: FREE STORAGE WITH HUGGING FACE</span></a><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-3 menu__list-item"><a class=menu__link tabindex=0 href=/docs/guides/huggingface-features><span title="✅ HuggingFace Dataset Sharing Added!" class=linkLabel_WmDU>✅ HuggingFace Dataset Sharing Added!</span></a><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-3 menu__list-item"><a class=menu__link tabindex=0 href=/docs/guides/huggingface-limits><span title="⚠️ HUGGING FACE FILE LIMITS & SOLUTIONS" class=linkLabel_WmDU>⚠️ HUGGING FACE FILE LIMITS & SOLUTIONS</span></a><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-3 menu__list-item"><a class=menu__link tabindex=0 href=/docs/guides/handling-formats><span title="📄 HANDLING MULTIPLE DOCUMENT FORMATS" class=linkLabel_WmDU>📄 HANDLING MULTIPLE DOCUMENT FORMATS</span></a><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-3 menu__list-item"><a class=menu__link tabindex=0 href=/docs/guides/document-libraries><span title="📦 INSTALLING DOCUMENT PROCESSING LIBRARIES" class=linkLabel_WmDU>📦 INSTALLING DOCUMENT PROCESSING LIBRARIES</span></a><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-3 menu__list-item"><a class=menu__link tabindex=0 href=/docs/guides/scraper-improvements><span title="Scraper Improvements Summary" class=linkLabel_WmDU>Scraper Improvements Summary</span></a><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-3 menu__list-item"><a class="menu__link menu__link--active" aria-current=page tabindex=0 href=/docs/guides/search-patterns><span title="Scale and Search Patterns: End-to-End Civic Tech Projects" class=linkLabel_WmDU>Scale and Search Patterns: End-to-End Civic Tech Projects</span></a><li class="theme-doc-sidebar-item-link theme-doc-sidebar-item-link-level-3 menu__list-item"><a class=menu__link tabindex=0 href=/docs/guides/split-screen><span title="Split-Screen System: Government Decisions ↔ Community Response" class=linkLabel_WmDU>Split-Screen System: Government Decisions ↔ Community Response</span></a></ul><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-2 menu__list-item menu__list-item--collapsed"><div class=menu__list-item-collapsible><a class="categoryLink_byQd menu__link menu__link--sublist menu__link--sublist-caret" role=button aria-expanded=false tabindex=0 href=/docs/integrations/mcp-server><span title=Integrations class=categoryLinkLabel_W154>Integrations</span></a></div><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-2 menu__list-item menu__list-item--collapsed"><div class=menu__list-item-collapsible><a class="categoryLink_byQd menu__link menu__link--sublist menu__link--sublist-caret" role=button aria-expanded=false tabindex=0 href=/docs/deployment/databricks-apps><span title=Deployment class=categoryLinkLabel_W154>Deployment</span></a></div><li class="theme-doc-sidebar-item-category theme-doc-sidebar-item-category-level-2 menu__list-item menu__list-item--collapsed"><div class=menu__list-item-collapsible><a class="categoryLink_byQd menu__link menu__link--sublist menu__link--sublist-caret" role=button aria-expanded=false tabindex=0 href=/docs/development/database-setup><span title=Development class=categoryLinkLabel_W154>Development</span></a></div></ul></ul></nav></div></div></aside><main class=docMainContainer_TBSr><div class="container padding-top--md padding-bottom--lg"><div class=row><div class="col docItemCol_VOVn"><div class=docItemContainer_Djhp><article><nav class="theme-doc-breadcrumbs breadcrumbsContainer_Z_bl" aria-label=Breadcrumbs><ul class=breadcrumbs><li class=breadcrumbs__item><a aria-label="Home page" class=breadcrumbs__link href=/><svg viewBox="0 0 24 24" class=breadcrumbHomeIcon_YNFT><path d="M10 19v-5h4v5c0 .55.45 1 1 1h3c.55 0 1-.45 1-1v-7h1.7c.46 0 .68-.57.33-.87L12.67 3.6c-.38-.34-.96-.34-1.34 0l-8.36 7.53c-.34.3-.13.87.33.87H5v7c0 .55.45 1 1 1h3c.55 0 1-.45 1-1z" fill=currentColor /></svg></a><li class=breadcrumbs__item><a class=breadcrumbs__link href=/docs/for-developers><span>Developers & Technical Users</span></a><li class=breadcrumbs__item><span class=breadcrumbs__link>How-To Guides</span><li class="breadcrumbs__item breadcrumbs__item--active"><span class=breadcrumbs__link>Scale and Search Patterns: End-to-End Civic Tech Projects</span></ul></nav><div class="tocCollapsible_ETCw theme-doc-toc-mobile tocMobile_ITEo"><button type=button class="clean-btn tocCollapsibleButton_TO0P">On this page</button></div><div class="theme-doc-markdown markdown"><header><h1>Scale and Search Patterns: End-to-End Civic Tech Projects</h1></header> | |
| <p>This guide analyzes <strong>6 additional civic tech projects</strong> focused on full-stack deployments, large-scale data aggregation, and public search portals. These complement our existing integration (Civic Scraper, City Scrapers, CDP, Engagic, Councilmatic) with new patterns for:</p> | |
| <ul> | |
| <li class="">🤖 <strong>AI summarization</strong> (OpenTowns, MeetingBank)</li> | |
| <li class="">🔍 <strong>Multi-jurisdiction search</strong> (CivicBand, LocalView)</li> | |
| <li class="">🔔 <strong>Keyword alerting</strong> (OpenTowns)</li> | |
| <li class="">📊 <strong>Research-grade pipelines</strong> (LocalView, MeetingBank)</li> | |
| <li class="">🌍 <strong>International adaptability</strong> (OpenCouncil)</li> | |
| </ul> | |
| <hr/> | |
| <h2 class="anchor anchorTargetStickyNavbar_Vzrq" id=-whats-new-vs-our-existing-integration>🎯 What's NEW vs. Our Existing Integration<a href=#-whats-new-vs-our-existing-integration class=hash-link aria-label="Direct link to 🎯 What's NEW vs. Our Existing Integration" title="Direct link to 🎯 What's NEW vs. Our Existing Integration" translate=no></a></h2> | |
| <table><thead><tr><th>Pattern<th>Already Have<th>NEW from These Projects<tbody><tr><td>Platform detection<td>✅ Civic Scraper<td>-<tr><td>Event schema<td>✅ City Scrapers<td>-<tr><td>Video ingestion<td>✅ CDP<td>✅ LocalView scale patterns<tr><td>Matter tracking<td>✅ Engagic<td>-<tr><td>Search UX<td>✅ Councilmatic<td>✅ CivicBand cross-jurisdiction<tr><td><strong>AI Summarization</strong><td>❌<td>✅ <strong>OpenTowns, MeetingBank</strong><tr><td><strong>Keyword Alerts</strong><td>❌<td>✅ <strong>OpenTowns</strong><tr><td><strong>Scale (1,000+ jurisdictions)</strong><td>⚠️ Partial<td>✅ <strong>CivicBand, LocalView</strong><tr><td><strong>International patterns</strong><td>❌<td>✅ <strong>OpenCouncil</strong></table> | |
| <hr/> | |
| <h2 class="anchor anchorTargetStickyNavbar_Vzrq" id=-project-analysis>📚 Project Analysis<a href=#-project-analysis class=hash-link aria-label="Direct link to 📚 Project Analysis" title="Direct link to 📚 Project Analysis" translate=no></a></h2> | |
| <h3 class="anchor anchorTargetStickyNavbar_Vzrq" id=1-council-data-project-cdp--already-integrated>1. Council Data Project (CDP) ⭐ Already Integrated<a href=#1-council-data-project-cdp--already-integrated class=hash-link aria-label="Direct link to 1. Council Data Project (CDP) ⭐ Already Integrated" title="Direct link to 1. Council Data Project (CDP) ⭐ Already Integrated" translate=no></a></h3> | |
| <p><strong>Status</strong>: Already documented in <code>INTEGRATION_GUIDE.md</code></p> | |
| <p><strong>Key patterns we already use</strong>:</p> | |
| <ul> | |
| <li class="">Video transcript ingestion</li> | |
| <li class="">Searchable transcript storage</li> | |
| <li class="">Event indexing pipeline</li> | |
| </ul> | |
| <p><strong>See</strong>: <code>docs/INTEGRATION_GUIDE.md</code> Section 4</p> | |
| <hr/> | |
| <h3 class="anchor anchorTargetStickyNavbar_Vzrq" id=2-opentowns--ai-summarization-pioneer>2. OpenTowns 🆕 AI Summarization Pioneer<a href=#2-opentowns--ai-summarization-pioneer class=hash-link aria-label="Direct link to 2. OpenTowns 🆕 AI Summarization Pioneer" title="Direct link to 2. OpenTowns 🆕 AI Summarization Pioneer" translate=no></a></h3> | |
| <p><strong>GitHub</strong>: <a href=https://opentowns.org target=_blank rel="noopener noreferrer" class="">https://opentowns.org</a><br/> | |
| <strong>License</strong>: Open civic-tech (check specific repo)<br/> | |
| <strong>Focus</strong>: Small towns, AI-generated summaries, keyword alerts</p> | |
| <h4 class="anchor anchorTargetStickyNavbar_Vzrq" id=-what-to-adopt>🔥 What to Adopt<a href=#-what-to-adopt class=hash-link aria-label="Direct link to 🔥 What to Adopt" title="Direct link to 🔥 What to Adopt" translate=no></a></h4> | |
| <p><strong>A. AI Summarization Pattern</strong></p> | |
| <div class="language-python codeBlockContainer_Ckt0 theme-code-block" style=--prism-color:#393A34;--prism-background-color:#f6f8fa><div class=codeBlockContent_QJqH><pre tabindex=0 class="prism-code language-python codeBlock_bY9V thin-scrollbar" style=color:#393A34;background-color:#f6f8fa><code class=codeBlockLines_e6Vv><div class=token-line style=color:#393A34><span class="token comment" style=color:#999988;font-style:italic># They generate readable summaries from raw transcripts/PDFs</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token comment" style=color:#999988;font-style:italic># Pattern: transcript → summary → key decisions</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain" style=display:inline-block></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>from</span><span class="token plain"> openai </span><span class="token keyword" style=color:#00009f>import</span><span class="token plain"> OpenAI</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>from</span><span class="token plain"> models</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">meeting_event </span><span class="token keyword" style=color:#00009f>import</span><span class="token plain"> MeetingEvent</span><br/></div><div class=token-line style=color:#393A34><span class="token plain" style=display:inline-block></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>async</span><span class="token plain"> </span><span class="token keyword" style=color:#00009f>def</span><span class="token plain"> </span><span class="token function" style=color:#d73a49>generate_meeting_summary</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">event</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> MeetingEvent</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> transcript</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>-</span><span class="token operator" style=color:#393A34>></span><span class="token plain"> </span><span class="token builtin">dict</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token triple-quoted-string string" style=color:#e3116c>"""</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> OpenTowns pattern: Generate human-readable meeting summaries.</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> </span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> Returns:</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> {</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> 'executive_summary': str, # 2-3 sentences</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> 'key_decisions': list[str], # Bullet points</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> 'health_policy_items': list[str], # Filtered for oral health</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> 'next_actions': list[str] # Follow-up items</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> }</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> client </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> OpenAI</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> prompt </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token string-interpolation string" style=color:#e3116c>f"""</span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> Summarize this local government meeting for public understanding.</span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> </span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> Meeting: </span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>{</span><span class="token string-interpolation interpolation">event</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>.</span><span class="token string-interpolation interpolation">title</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>}</span><span class="token string-interpolation string" style=color:#e3116c></span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> Date: </span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>{</span><span class="token string-interpolation interpolation">event</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>.</span><span class="token string-interpolation interpolation">start</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>.</span><span class="token string-interpolation interpolation">strftime</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>(</span><span class="token string-interpolation interpolation string" style=color:#e3116c>'%B %d, %Y'</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>)</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>}</span><span class="token string-interpolation string" style=color:#e3116c></span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> Transcript: </span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>{</span><span class="token string-interpolation interpolation">transcript</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>[</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>:</span><span class="token string-interpolation interpolation format-spec">10000]</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>}</span><span class="token string-interpolation string" style=color:#e3116c> # First 10k chars</span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> </span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> Provide:</span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> 1. Executive summary (2-3 sentences)</span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> 2. Key decisions made (bullet points)</span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> 3. Health policy items (if any)</span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> 4. Next actions/follow-ups</span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> </span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> Focus on: What decisions were made? What happens next?</span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> response </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> client</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">chat</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">completions</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">create</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> model</span><span class="token operator" style=color:#393A34>=</span><span class="token string" style=color:#e3116c>"gpt-4o-mini"</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Cost-effective for summaries</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> messages</span><span class="token operator" style=color:#393A34>=</span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token string" style=color:#e3116c>"role"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"system"</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"content"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"You are a civic engagement assistant helping residents understand local government."</span><span class="token punctuation" style=color:#393A34>}</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token string" style=color:#e3116c>"role"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"user"</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"content"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> prompt</span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> temperature</span><span class="token operator" style=color:#393A34>=</span><span class="token number" style=color:#36acaa>0.3</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Lower for factual accuracy</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Parse response into structured format</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> summary_text </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> response</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">choices</span><span class="token punctuation" style=color:#393A34>[</span><span class="token number" style=color:#36acaa>0</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">message</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">content</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>return</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'executive_summary'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> extract_section</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">summary_text</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'Executive summary'</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'key_decisions'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> extract_bullets</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">summary_text</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'Key decisions'</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'health_policy_items'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> extract_bullets</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">summary_text</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'Health policy'</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'next_actions'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> extract_bullets</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">summary_text</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'Next actions'</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'raw_summary'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> summary_text</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>}</span><br/></div></code></pre></div></div> | |
| <p><strong>B. Keyword Alert System</strong></p> | |
| <div class="language-python codeBlockContainer_Ckt0 theme-code-block" style=--prism-color:#393A34;--prism-background-color:#f6f8fa><div class=codeBlockContent_QJqH><pre tabindex=0 class="prism-code language-python codeBlock_bY9V thin-scrollbar" style=color:#393A34;background-color:#f6f8fa><code class=codeBlockLines_e6Vv><div class=token-line style=color:#393A34><span class="token comment" style=color:#999988;font-style:italic># OpenTowns sends alerts when keywords appear in meetings</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token comment" style=color:#999988;font-style:italic># Pattern: Watch list → match detection → user notification</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain" style=display:inline-block></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>from</span><span class="token plain"> typing </span><span class="token keyword" style=color:#00009f>import</span><span class="token plain"> List</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> Dict</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>import</span><span class="token plain"> re</span><br/></div><div class=token-line style=color:#393A34><span class="token plain" style=display:inline-block></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>class</span><span class="token plain"> </span><span class="token class-name">KeywordAlertSystem</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token triple-quoted-string string" style=color:#e3116c>"""</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> OpenTowns pattern: Alert users when keywords appear in meetings.</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Oral health keyword categories</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> KEYWORD_CATEGORIES </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'fluoridation'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'fluoride'</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'fluoridation'</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'water treatment'</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'community water fluoridation'</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'CWF'</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'dental_access'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'dental'</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'dentist'</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'oral health'</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'teeth'</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'medicaid dental'</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'dental clinic'</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'public_health'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'health department'</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'public health'</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'CDC'</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'preventive care'</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'health equity'</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>def</span><span class="token plain"> </span><span class="token function" style=color:#d73a49>detect_keywords</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">self</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> text</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>-</span><span class="token operator" style=color:#393A34>></span><span class="token plain"> Dict</span><span class="token punctuation" style=color:#393A34>[</span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> List</span><span class="token punctuation" style=color:#393A34>[</span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token triple-quoted-string string" style=color:#e3116c>"""</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> Find all matching keywords in text.</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> </span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> Returns: {'fluoridation': ['fluoride', 'CWF'], ...}</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> text_lower </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> text</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">lower</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> matches </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>for</span><span class="token plain"> category</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> keywords </span><span class="token keyword" style=color:#00009f>in</span><span class="token plain"> self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">KEYWORD_CATEGORIES</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">items</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> found </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>for</span><span class="token plain"> keyword </span><span class="token keyword" style=color:#00009f>in</span><span class="token plain"> keywords</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Word boundary matching</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> pattern </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>r'\b'</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>+</span><span class="token plain"> re</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">escape</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">keyword</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">lower</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>+</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>r'\b'</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>if</span><span class="token plain"> re</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">search</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">pattern</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> text_lower</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> found</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">append</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">keyword</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>if</span><span class="token plain"> found</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> matches</span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain">category</span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> found</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>return</span><span class="token plain"> matches</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>def</span><span class="token plain"> </span><span class="token function" style=color:#d73a49>generate_alert</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">self</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> event</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> MeetingEvent</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> matches</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> Dict</span><span class="token punctuation" style=color:#393A34>[</span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> List</span><span class="token punctuation" style=color:#393A34>[</span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>-</span><span class="token operator" style=color:#393A34>></span><span class="token plain"> </span><span class="token builtin">dict</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token triple-quoted-string string" style=color:#e3116c>"""</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> Create alert notification for users.</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>return</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'alert_type'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'keyword_match'</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'jurisdiction'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token string-interpolation string" style=color:#e3116c>f"</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>{</span><span class="token string-interpolation interpolation">event</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>.</span><span class="token string-interpolation interpolation">jurisdiction_name</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>}</span><span class="token string-interpolation string" style=color:#e3116c>, </span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>{</span><span class="token string-interpolation interpolation">event</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>.</span><span class="token string-interpolation interpolation">state_code</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>}</span><span class="token string-interpolation string" style=color:#e3116c>"</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'meeting_title'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> event</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">title</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'meeting_date'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> event</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">start</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">isoformat</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'categories_matched'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">list</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">matches</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">keys</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'keywords_found'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain">kw </span><span class="token keyword" style=color:#00009f>for</span><span class="token plain"> kws </span><span class="token keyword" style=color:#00009f>in</span><span class="token plain"> matches</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">values</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token keyword" style=color:#00009f>for</span><span class="token plain"> kw </span><span class="token keyword" style=color:#00009f>in</span><span class="token plain"> kws</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'meeting_url'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> event</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">source</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'priority'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'high'</span><span class="token plain"> </span><span class="token keyword" style=color:#00009f>if</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'fluoridation'</span><span class="token plain"> </span><span class="token keyword" style=color:#00009f>in</span><span class="token plain"> matches </span><span class="token keyword" style=color:#00009f>else</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'medium'</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>}</span><br/></div></code></pre></div></div> | |
| <p><strong>Implementation Priority</strong>: 🔥 <strong>HIGH</strong> - Summaries make data usable for advocates</p> | |
| <hr/> | |
| <h3 class="anchor anchorTargetStickyNavbar_Vzrq" id=3-localview--research-grade-scale>3. LocalView 🆕 Research-Grade Scale<a href=#3-localview--research-grade-scale class=hash-link aria-label="Direct link to 3. LocalView 🆕 Research-Grade Scale" title="Direct link to 3. LocalView 🆕 Research-Grade Scale" translate=no></a></h3> | |
| <p><strong>Website</strong>: <a href=https://www.localview.net target=_blank rel="noopener noreferrer" class="">https://www.localview.net</a><br/> | |
| <strong>GitHub</strong>: <a href=https://mellonurbanism.harvard.edu/localview target=_blank rel="noopener noreferrer" class="">https://mellonurbanism.harvard.edu/localview</a><br/> | |
| <strong>License</strong>: Open-source data pipeline<br/> | |
| <strong>Scale</strong>: Nationwide coverage, largest public dataset</p> | |
| <h4 class="anchor anchorTargetStickyNavbar_Vzrq" id=-what-to-adopt-1>🔥 What to Adopt<a href=#-what-to-adopt-1 class=hash-link aria-label="Direct link to 🔥 What to Adopt" title="Direct link to 🔥 What to Adopt" translate=no></a></h4> | |
| <p><strong>A. Scale Architecture Patterns</strong></p> | |
| <p>LocalView handles <strong>thousands of jurisdictions</strong> with:</p> | |
| <ol> | |
| <li class=""><strong>Batch processing</strong> (not real-time)</li> | |
| <li class=""><strong>Distributed storage</strong> (videos + transcripts)</li> | |
| <li class=""><strong>Quality metrics</strong> (completeness scoring)</li> | |
| </ol> | |
| <div class="language-python codeBlockContainer_Ckt0 theme-code-block" style=--prism-color:#393A34;--prism-background-color:#f6f8fa><div class=codeBlockContent_QJqH><pre tabindex=0 class="prism-code language-python codeBlock_bY9V thin-scrollbar" style=color:#393A34;background-color:#f6f8fa><code class=codeBlockLines_e6Vv><div class=token-line style=color:#393A34><span class="token comment" style=color:#999988;font-style:italic># LocalView pattern: Process jurisdictions in batches with quality tracking</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain" style=display:inline-block></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>from</span><span class="token plain"> dataclasses </span><span class="token keyword" style=color:#00009f>import</span><span class="token plain"> dataclass</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>from</span><span class="token plain"> datetime </span><span class="token keyword" style=color:#00009f>import</span><span class="token plain"> datetime</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>from</span><span class="token plain"> typing </span><span class="token keyword" style=color:#00009f>import</span><span class="token plain"> Optional</span><br/></div><div class=token-line style=color:#393A34><span class="token plain" style=display:inline-block></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token decorator annotation punctuation" style=color:#393A34>@dataclass</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>class</span><span class="token plain"> </span><span class="token class-name">JurisdictionQuality</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token triple-quoted-string string" style=color:#e3116c>"""</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> LocalView pattern: Track data quality per jurisdiction.</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> jurisdiction_name</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">str</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> state_code</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">str</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Completeness metrics</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> total_meetings_expected</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">int</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Based on calendar</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> total_meetings_found</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">int</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> meetings_with_agendas</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">int</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> meetings_with_minutes</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">int</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> meetings_with_videos</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">int</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> meetings_with_transcripts</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">int</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Freshness</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> last_scraped</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> datetime</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> last_meeting_found</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> Optional</span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain">datetime</span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> scraping_frequency</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">str</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># 'daily', 'weekly', 'monthly'</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Health metrics</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> consecutive_failures</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">int</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> last_success</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> Optional</span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain">datetime</span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token decorator annotation punctuation" style=color:#393A34>@property</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>def</span><span class="token plain"> </span><span class="token function" style=color:#d73a49>completeness_score</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">self</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>-</span><span class="token operator" style=color:#393A34>></span><span class="token plain"> </span><span class="token builtin">float</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token triple-quoted-string string" style=color:#e3116c>"""</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> Overall data quality score (0-100).</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>if</span><span class="token plain"> self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">total_meetings_expected </span><span class="token operator" style=color:#393A34>==</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>0</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>return</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>0.0</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> found_rate </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">total_meetings_found </span><span class="token operator" style=color:#393A34>/</span><span class="token plain"> self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">total_meetings_expected</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> agenda_rate </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">meetings_with_agendas </span><span class="token operator" style=color:#393A34>/</span><span class="token plain"> </span><span class="token builtin">max</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">total_meetings_found</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>1</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> minutes_rate </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">meetings_with_minutes </span><span class="token operator" style=color:#393A34>/</span><span class="token plain"> </span><span class="token builtin">max</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">total_meetings_found</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>1</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Weighted average</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> score </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> found_rate </span><span class="token operator" style=color:#393A34>*</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>40</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>+</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># 40%: Finding meetings</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> agenda_rate </span><span class="token operator" style=color:#393A34>*</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>30</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>+</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># 30%: Having agendas</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> minutes_rate </span><span class="token operator" style=color:#393A34>*</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>30</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># 30%: Having minutes</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>return</span><span class="token plain"> </span><span class="token builtin">min</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">score </span><span class="token operator" style=color:#393A34>*</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>100</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>100.0</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token decorator annotation punctuation" style=color:#393A34>@property</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>def</span><span class="token plain"> </span><span class="token function" style=color:#d73a49>health_status</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">self</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>-</span><span class="token operator" style=color:#393A34>></span><span class="token plain"> </span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token triple-quoted-string string" style=color:#e3116c>"""</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> Scraper health: healthy, degraded, failed</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>if</span><span class="token plain"> self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">consecutive_failures </span><span class="token operator" style=color:#393A34>>=</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>5</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>return</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'failed'</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>elif</span><span class="token plain"> self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">consecutive_failures </span><span class="token operator" style=color:#393A34>>=</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>2</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>return</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'degraded'</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>else</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>return</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'healthy'</span><br/></div></code></pre></div></div> | |
| <p><strong>B. Batch Processing Strategy</strong></p> | |
| <div class="language-python codeBlockContainer_Ckt0 theme-code-block" style=--prism-color:#393A34;--prism-background-color:#f6f8fa><div class=codeBlockContent_QJqH><pre tabindex=0 class="prism-code language-python codeBlock_bY9V thin-scrollbar" style=color:#393A34;background-color:#f6f8fa><code class=codeBlockLines_e6Vv><div class=token-line style=color:#393A34><span class="token comment" style=color:#999988;font-style:italic># LocalView processes in batches, not all-at-once</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain" style=display:inline-block></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>from</span><span class="token plain"> pyspark</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">sql </span><span class="token keyword" style=color:#00009f>import</span><span class="token plain"> SparkSession</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>from</span><span class="token plain"> typing </span><span class="token keyword" style=color:#00009f>import</span><span class="token plain"> Iterator</span><br/></div><div class=token-line style=color:#393A34><span class="token plain" style=display:inline-block></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>def</span><span class="token plain"> </span><span class="token function" style=color:#d73a49>process_jurisdictions_in_batches</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> spark</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> SparkSession</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> batch_size</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">int</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>100</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> priority_filter</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">str</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'high'</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>-</span><span class="token operator" style=color:#393A34>></span><span class="token plain"> Iterator</span><span class="token punctuation" style=color:#393A34>[</span><span class="token builtin">dict</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token triple-quoted-string string" style=color:#e3116c>"""</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> LocalView pattern: Process large numbers of jurisdictions efficiently.</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> </span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> Strategy:</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> 1. Load high-priority jurisdictions first</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> 2. Process in batches to manage memory</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> 3. Track quality metrics per batch</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> 4. Resume from failures</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Load targets from Gold layer</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> targets_df </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> spark</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">read</span><span class="token punctuation" style=color:#393A34>.</span><span class="token builtin">format</span><span class="token punctuation" style=color:#393A34>(</span><span class="token string" style=color:#e3116c>"delta"</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">load</span><span class="token punctuation" style=color:#393A34>(</span><span class="token string" style=color:#e3116c>"data/delta/gold/scraping_targets"</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Filter and sort</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> priority_targets </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> targets_df \</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>.</span><span class="token builtin">filter</span><span class="token punctuation" style=color:#393A34>(</span><span class="token string-interpolation string" style=color:#e3116c>f"priority_tier = '</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>{</span><span class="token string-interpolation interpolation">priority_filter</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>}</span><span class="token string-interpolation string" style=color:#e3116c>'"</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> \</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">orderBy</span><span class="token punctuation" style=color:#393A34>(</span><span class="token string" style=color:#e3116c>"priority_score"</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> ascending</span><span class="token operator" style=color:#393A34>=</span><span class="token boolean" style=color:#36acaa>False</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> total_targets </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> priority_targets</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">count</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Process in batches</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>for</span><span class="token plain"> offset </span><span class="token keyword" style=color:#00009f>in</span><span class="token plain"> </span><span class="token builtin">range</span><span class="token punctuation" style=color:#393A34>(</span><span class="token number" style=color:#36acaa>0</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> total_targets</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> batch_size</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> batch_df </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> priority_targets</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">limit</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">batch_size</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">offset</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">offset</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> batch_results </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'batch_number'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> offset </span><span class="token operator" style=color:#393A34>//</span><span class="token plain"> batch_size </span><span class="token operator" style=color:#393A34>+</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>1</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'batch_size'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> batch_size</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'jurisdictions_processed'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>0</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'meetings_found'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>0</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'errors'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>for</span><span class="token plain"> row </span><span class="token keyword" style=color:#00009f>in</span><span class="token plain"> batch_df</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">collect</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>try</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Scrape jurisdiction</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> meetings </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> scrape_jurisdiction</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">row</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'url'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> row</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'platform'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> batch_results</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'jurisdictions_processed'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>+=</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>1</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> batch_results</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'meetings_found'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>+=</span><span class="token plain"> </span><span class="token builtin">len</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">meetings</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>except</span><span class="token plain"> Exception </span><span class="token keyword" style=color:#00009f>as</span><span class="token plain"> e</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> batch_results</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'errors'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">append</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>{</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'jurisdiction'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> row</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'jurisdiction_name'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'error'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">e</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>}</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>yield</span><span class="token plain"> batch_results</span><br/></div></code></pre></div></div> | |
| <p><strong>Implementation Priority</strong>: 🔥 <strong>HIGH</strong> - Essential for scaling to 32,333 municipalities</p> | |
| <hr/> | |
| <h3 class="anchor anchorTargetStickyNavbar_Vzrq" id=4-meetingbank--summarization-research>4. MeetingBank 🆕 Summarization Research<a href=#4-meetingbank--summarization-research class=hash-link aria-label="Direct link to 4. MeetingBank 🆕 Summarization Research" title="Direct link to 4. MeetingBank 🆕 Summarization Research" translate=no></a></h3> | |
| <p><strong>Website</strong>: <a href=https://meetingbank.github.io target=_blank rel="noopener noreferrer" class="">https://meetingbank.github.io</a><br/> | |
| <strong>GitHub</strong>: Linked from site<br/> | |
| <strong>License</strong>: Open dataset<br/> | |
| <strong>Focus</strong>: 6 cities, high-quality summarization benchmark</p> | |
| <h4 class="anchor anchorTargetStickyNavbar_Vzrq" id=-what-to-adopt-2>🔥 What to Adopt<a href=#-what-to-adopt-2 class=hash-link aria-label="Direct link to 🔥 What to Adopt" title="Direct link to 🔥 What to Adopt" translate=no></a></h4> | |
| <p><strong>A. Summarization Quality Benchmarks</strong></p> | |
| <p>MeetingBank is used in academic research for summarization. They have:</p> | |
| <ul> | |
| <li class=""><strong>Gold-standard human summaries</strong> (for validation)</li> | |
| <li class=""><strong>Multiple summary lengths</strong> (short, medium, long)</li> | |
| <li class=""><strong>Evaluation metrics</strong> (ROUGE, BERTScore)</li> | |
| </ul> | |
| <div class="language-python codeBlockContainer_Ckt0 theme-code-block" style=--prism-color:#393A34;--prism-background-color:#f6f8fa><div class=codeBlockContent_QJqH><pre tabindex=0 class="prism-code language-python codeBlock_bY9V thin-scrollbar" style=color:#393A34;background-color:#f6f8fa><code class=codeBlockLines_e6Vv><div class=token-line style=color:#393A34><span class="token comment" style=color:#999988;font-style:italic># MeetingBank pattern: Validate AI summaries against quality benchmarks</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain" style=display:inline-block></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>from</span><span class="token plain"> typing </span><span class="token keyword" style=color:#00009f>import</span><span class="token plain"> Dict</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>import</span><span class="token plain"> numpy </span><span class="token keyword" style=color:#00009f>as</span><span class="token plain"> np</span><br/></div><div class=token-line style=color:#393A34><span class="token plain" style=display:inline-block></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>class</span><span class="token plain"> </span><span class="token class-name">SummaryQualityValidator</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token triple-quoted-string string" style=color:#e3116c>"""</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> MeetingBank pattern: Ensure AI summaries meet quality standards.</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Quality thresholds from academic research</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> MIN_ROUGE_L </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>0.25</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># ROUGE-L F1 score</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> MIN_LENGTH_RATIO </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>0.05</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Summary should be 5-20% of original</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> MAX_LENGTH_RATIO </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>0.20</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>def</span><span class="token plain"> </span><span class="token function" style=color:#d73a49>validate_summary</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">self</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> original</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> summary</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>-</span><span class="token operator" style=color:#393A34>></span><span class="token plain"> Dict</span><span class="token punctuation" style=color:#393A34>[</span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token builtin">any</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token triple-quoted-string string" style=color:#e3116c>"""</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> Check if summary meets quality standards.</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Length checks</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> orig_words </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token builtin">len</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">original</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">split</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> summ_words </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token builtin">len</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">summary</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">split</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> length_ratio </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> summ_words </span><span class="token operator" style=color:#393A34>/</span><span class="token plain"> orig_words </span><span class="token keyword" style=color:#00009f>if</span><span class="token plain"> orig_words </span><span class="token operator" style=color:#393A34>></span><span class="token plain"> </span><span class="token number" style=color:#36acaa>0</span><span class="token plain"> </span><span class="token keyword" style=color:#00009f>else</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>0</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Basic quality checks</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> checks </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'length_appropriate'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">MIN_LENGTH_RATIO </span><span class="token operator" style=color:#393A34><=</span><span class="token plain"> length_ratio </span><span class="token operator" style=color:#393A34><=</span><span class="token plain"> self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">MAX_LENGTH_RATIO</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'has_key_terms'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">_check_key_terms</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">original</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> summary</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'no_repetition'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">_check_repetition</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">summary</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'proper_structure'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">_check_structure</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">summary</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>return</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'passes_validation'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">all</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">checks</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">values</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'checks'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> checks</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'length_ratio'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> length_ratio</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'word_count'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> summ_words</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'quality_score'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">sum</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">checks</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">values</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>/</span><span class="token plain"> </span><span class="token builtin">len</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">checks</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>def</span><span class="token plain"> </span><span class="token function" style=color:#d73a49>_check_key_terms</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">self</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> original</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> summary</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>-</span><span class="token operator" style=color:#393A34>></span><span class="token plain"> </span><span class="token builtin">bool</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token triple-quoted-string string" style=color:#e3116c>"""</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> Ensure summary includes key terms from original.</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Extract important terms (simplified - use TF-IDF in production)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> orig_words </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token builtin">set</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">original</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">lower</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">split</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> summ_words </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token builtin">set</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">summary</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">lower</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">split</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># At least 30% overlap of unique terms</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> overlap </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token builtin">len</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">orig_words </span><span class="token operator" style=color:#393A34>&</span><span class="token plain"> summ_words</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>/</span><span class="token plain"> </span><span class="token builtin">len</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">orig_words</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>return</span><span class="token plain"> overlap </span><span class="token operator" style=color:#393A34>>=</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>0.30</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>def</span><span class="token plain"> </span><span class="token function" style=color:#d73a49>_check_repetition</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">self</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> summary</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>-</span><span class="token operator" style=color:#393A34>></span><span class="token plain"> </span><span class="token builtin">bool</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token triple-quoted-string string" style=color:#e3116c>"""</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> Check for excessive repetition (indicates poor quality).</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> sentences </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> summary</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">split</span><span class="token punctuation" style=color:#393A34>(</span><span class="token string" style=color:#e3116c>'.'</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> unique_ratio </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token builtin">len</span><span class="token punctuation" style=color:#393A34>(</span><span class="token builtin">set</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">sentences</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>/</span><span class="token plain"> </span><span class="token builtin">len</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">sentences</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token keyword" style=color:#00009f>if</span><span class="token plain"> sentences </span><span class="token keyword" style=color:#00009f>else</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>0</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>return</span><span class="token plain"> unique_ratio </span><span class="token operator" style=color:#393A34>>=</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>0.80</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># At least 80% unique sentences</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>def</span><span class="token plain"> </span><span class="token function" style=color:#d73a49>_check_structure</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">self</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> summary</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>-</span><span class="token operator" style=color:#393A34>></span><span class="token plain"> </span><span class="token builtin">bool</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token triple-quoted-string string" style=color:#e3116c>"""</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> Check for proper summary structure.</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Should have multiple sentences</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> sentences </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain">s</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">strip</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token keyword" style=color:#00009f>for</span><span class="token plain"> s </span><span class="token keyword" style=color:#00009f>in</span><span class="token plain"> summary</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">split</span><span class="token punctuation" style=color:#393A34>(</span><span class="token string" style=color:#e3116c>'.'</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token keyword" style=color:#00009f>if</span><span class="token plain"> s</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">strip</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>return</span><span class="token plain"> </span><span class="token builtin">len</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">sentences</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>>=</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>2</span><span class="token plain"> </span><span class="token keyword" style=color:#00009f>and</span><span class="token plain"> </span><span class="token builtin">len</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">sentences</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token operator" style=color:#393A34><=</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>10</span><br/></div></code></pre></div></div> | |
| <p><strong>Implementation Priority</strong>: 🟡 <strong>MEDIUM</strong> - Important for quality, but MVP can use basic summaries</p> | |
| <hr/> | |
| <h3 class="anchor anchorTargetStickyNavbar_Vzrq" id=5-civicband--multi-jurisdiction-search>5. CivicBand 🆕 Multi-Jurisdiction Search<a href=#5-civicband--multi-jurisdiction-search class=hash-link aria-label="Direct link to 5. CivicBand 🆕 Multi-Jurisdiction Search" title="Direct link to 5. CivicBand 🆕 Multi-Jurisdiction Search" translate=no></a></h3> | |
| <p><strong>Website</strong>: <a href=https://civic.band target=_blank rel="noopener noreferrer" class="">https://civic.band</a><br/> | |
| <strong>GitHub</strong>: Linked from site (Raft Foundation)<br/> | |
| <strong>Scale</strong>: 1,000+ municipalities<br/> | |
| <strong>Focus</strong>: Google-like search across jurisdictions</p> | |
| <h4 class="anchor anchorTargetStickyNavbar_Vzrq" id=-what-to-adopt-3>🔥 What to Adopt<a href=#-what-to-adopt-3 class=hash-link aria-label="Direct link to 🔥 What to Adopt" title="Direct link to 🔥 What to Adopt" translate=no></a></h4> | |
| <p><strong>A. Cross-Jurisdiction Search Architecture</strong></p> | |
| <p>CivicBand lets users search "fluoridation" and get results from <strong>all municipalities</strong> at once.</p> | |
| <div class="language-python codeBlockContainer_Ckt0 theme-code-block" style=--prism-color:#393A34;--prism-background-color:#f6f8fa><div class=codeBlockContent_QJqH><pre tabindex=0 class="prism-code language-python codeBlock_bY9V thin-scrollbar" style=color:#393A34;background-color:#f6f8fa><code class=codeBlockLines_e6Vv><div class=token-line style=color:#393A34><span class="token comment" style=color:#999988;font-style:italic># CivicBand pattern: Federated search across jurisdictions</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain" style=display:inline-block></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>from</span><span class="token plain"> elasticsearch </span><span class="token keyword" style=color:#00009f>import</span><span class="token plain"> Elasticsearch </span><span class="token comment" style=color:#999988;font-style:italic># Or Meilisearch for open-source</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>from</span><span class="token plain"> typing </span><span class="token keyword" style=color:#00009f>import</span><span class="token plain"> List</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> Dict</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>from</span><span class="token plain"> models</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">meeting_event </span><span class="token keyword" style=color:#00009f>import</span><span class="token plain"> MeetingEvent</span><br/></div><div class=token-line style=color:#393A34><span class="token plain" style=display:inline-block></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>class</span><span class="token plain"> </span><span class="token class-name">CrossJurisdictionSearch</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token triple-quoted-string string" style=color:#e3116c>"""</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> CivicBand pattern: Search meetings across all jurisdictions.</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>def</span><span class="token plain"> </span><span class="token function" style=color:#d73a49>__init__</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">self</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Use Meilisearch (open-source) or Elasticsearch</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">es </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> Elasticsearch</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'http://localhost:9200'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">index_name </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'meeting_events'</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>def</span><span class="token plain"> </span><span class="token function" style=color:#d73a49>index_meeting</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">self</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> event</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> MeetingEvent</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token triple-quoted-string string" style=color:#e3116c>"""</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> Add meeting to search index.</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> doc </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'id'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> event</span><span class="token punctuation" style=color:#393A34>.</span><span class="token builtin">id</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'title'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> event</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">title</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'description'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> event</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">description</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'jurisdiction'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> event</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">jurisdiction_name</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'state'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> event</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">state_code</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'date'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> event</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">start</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">isoformat</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'full_text'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">_build_searchable_text</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">event</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'agenda_url'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">next</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">link</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">href </span><span class="token keyword" style=color:#00009f>for</span><span class="token plain"> link </span><span class="token keyword" style=color:#00009f>in</span><span class="token plain"> event</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">links </span><span class="token keyword" style=color:#00009f>if</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'agenda'</span><span class="token plain"> </span><span class="token keyword" style=color:#00009f>in</span><span class="token plain"> link</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">title</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">lower</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token boolean" style=color:#36acaa>None</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'oral_health_relevant'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> event</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">oral_health_relevant</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'keywords'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> event</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">keywords_found</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">es</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">index</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">index</span><span class="token operator" style=color:#393A34>=</span><span class="token plain">self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">index_name</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token builtin">id</span><span class="token operator" style=color:#393A34>=</span><span class="token plain">event</span><span class="token punctuation" style=color:#393A34>.</span><span class="token builtin">id</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> document</span><span class="token operator" style=color:#393A34>=</span><span class="token plain">doc</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>def</span><span class="token plain"> </span><span class="token function" style=color:#d73a49>search</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> self</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> query</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> states</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> List</span><span class="token punctuation" style=color:#393A34>[</span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token boolean" style=color:#36acaa>None</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> date_range</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">tuple</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token boolean" style=color:#36acaa>None</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> oral_health_only</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">bool</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token boolean" style=color:#36acaa>False</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>-</span><span class="token operator" style=color:#393A34>></span><span class="token plain"> List</span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain">Dict</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token triple-quoted-string string" style=color:#e3116c>"""</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> Search across all jurisdictions.</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> </span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> Example:</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> search("fluoridation", states=['AL', 'GA'], oral_health_only=True)</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> must_clauses </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token string" style=color:#e3116c>"multi_match"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>"query"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> query</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>"fields"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>"title^3"</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"description^2"</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"full_text"</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Boost title matches</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>"type"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"best_fields"</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>}</span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Filter by state</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>if</span><span class="token plain"> states</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> must_clauses</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">append</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>{</span><span class="token string" style=color:#e3116c>"terms"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token string" style=color:#e3116c>"state"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> states</span><span class="token punctuation" style=color:#393A34>}</span><span class="token punctuation" style=color:#393A34>}</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Filter by date range</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>if</span><span class="token plain"> date_range</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> must_clauses</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">append</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>{</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>"range"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token string" style=color:#e3116c>"date"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token string" style=color:#e3116c>"gte"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> date_range</span><span class="token punctuation" style=color:#393A34>[</span><span class="token number" style=color:#36acaa>0</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"lte"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> date_range</span><span class="token punctuation" style=color:#393A34>[</span><span class="token number" style=color:#36acaa>1</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>}</span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>}</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Filter oral health only</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>if</span><span class="token plain"> oral_health_only</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> must_clauses</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">append</span><span class="token punctuation" style=color:#393A34>(</span><span class="token punctuation" style=color:#393A34>{</span><span class="token string" style=color:#e3116c>"term"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token string" style=color:#e3116c>"oral_health_relevant"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token boolean" style=color:#36acaa>True</span><span class="token punctuation" style=color:#393A34>}</span><span class="token punctuation" style=color:#393A34>}</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> search_query </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>"query"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token string" style=color:#e3116c>"bool"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token string" style=color:#e3116c>"must"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> must_clauses</span><span class="token punctuation" style=color:#393A34>}</span><span class="token punctuation" style=color:#393A34>}</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>"size"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>100</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>"highlight"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>"fields"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>"title"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token punctuation" style=color:#393A34>}</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>"description"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token punctuation" style=color:#393A34>}</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>"full_text"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token string" style=color:#e3116c>"fragment_size"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>150</span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>}</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>"sort"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token string" style=color:#e3116c>"_score"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"desc"</span><span class="token punctuation" style=color:#393A34>}</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token string" style=color:#e3116c>"date"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"desc"</span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> results </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">es</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">search</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">index</span><span class="token operator" style=color:#393A34>=</span><span class="token plain">self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">index_name</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> body</span><span class="token operator" style=color:#393A34>=</span><span class="token plain">search_query</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>return</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token punctuation" style=color:#393A34>{</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'jurisdiction'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> hit</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'_source'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'jurisdiction'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'state'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> hit</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'_source'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'state'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'title'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> hit</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'_source'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'title'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'date'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> hit</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'_source'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'date'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'snippet'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> hit</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">get</span><span class="token punctuation" style=color:#393A34>(</span><span class="token string" style=color:#e3116c>'highlight'</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token punctuation" style=color:#393A34>}</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">get</span><span class="token punctuation" style=color:#393A34>(</span><span class="token string" style=color:#e3116c>'full_text'</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>''</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>[</span><span class="token number" style=color:#36acaa>0</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'url'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> hit</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'_source'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'agenda_url'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'relevance_score'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> hit</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'_score'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"> </span><span class="token keyword" style=color:#00009f>for</span><span class="token plain"> hit </span><span class="token keyword" style=color:#00009f>in</span><span class="token plain"> results</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'hits'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'hits'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>def</span><span class="token plain"> </span><span class="token function" style=color:#d73a49>_build_searchable_text</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">self</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> event</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> MeetingEvent</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>-</span><span class="token operator" style=color:#393A34>></span><span class="token plain"> </span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token triple-quoted-string string" style=color:#e3116c>"""</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> Combine all text fields for indexing.</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> parts </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> event</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">title </span><span class="token keyword" style=color:#00009f>or</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>''</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> event</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">description </span><span class="token keyword" style=color:#00009f>or</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>''</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>' '</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">join</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">event</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">keywords_found</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>' '</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">join</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">link</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">title </span><span class="token keyword" style=color:#00009f>for</span><span class="token plain"> link </span><span class="token keyword" style=color:#00009f>in</span><span class="token plain"> event</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">links</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>return</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>' '</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">join</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">parts</span><span class="token punctuation" style=color:#393A34>)</span><br/></div></code></pre></div></div> | |
| <p><strong>B. Jurisdiction Faceting</strong></p> | |
| <div class="language-python codeBlockContainer_Ckt0 theme-code-block" style=--prism-color:#393A34;--prism-background-color:#f6f8fa><div class=codeBlockContent_QJqH><pre tabindex=0 class="prism-code language-python codeBlock_bY9V thin-scrollbar" style=color:#393A34;background-color:#f6f8fa><code class=codeBlockLines_e6Vv><div class=token-line style=color:#393A34><span class="token comment" style=color:#999988;font-style:italic># CivicBand shows result counts by jurisdiction</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain" style=display:inline-block></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>def</span><span class="token plain"> </span><span class="token function" style=color:#d73a49>get_search_facets</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">query</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>-</span><span class="token operator" style=color:#393A34>></span><span class="token plain"> Dict</span><span class="token punctuation" style=color:#393A34>[</span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token builtin">int</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token triple-quoted-string string" style=color:#e3116c>"""</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> Show how many results per jurisdiction.</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> </span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> Example output:</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> {</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> 'Birmingham, AL': 12,</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> 'Atlanta, GA': 8,</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> 'Montgomery, AL': 5</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> }</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> search_query </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>"query"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token string" style=color:#e3116c>"multi_match"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token string" style=color:#e3116c>"query"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> query</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"fields"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>"title"</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"full_text"</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>}</span><span class="token punctuation" style=color:#393A34>}</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>"size"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>0</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># We only want aggregations</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>"aggs"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>"by_jurisdiction"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>"terms"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>"field"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"jurisdiction.keyword"</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>"size"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>50</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Top 50 jurisdictions</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>}</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>"aggs"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>"by_state"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>"terms"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token string" style=color:#e3116c>"field"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"state.keyword"</span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> results </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">es</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">search</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">index</span><span class="token operator" style=color:#393A34>=</span><span class="token plain">self</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">index_name</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> body</span><span class="token operator" style=color:#393A34>=</span><span class="token plain">search_query</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> facets </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>for</span><span class="token plain"> bucket </span><span class="token keyword" style=color:#00009f>in</span><span class="token plain"> results</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'aggregations'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'by_jurisdiction'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'buckets'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> jurisdiction </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> bucket</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'key'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> count </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> bucket</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'doc_count'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> state </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> bucket</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'by_state'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'buckets'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>[</span><span class="token number" style=color:#36acaa>0</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'key'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> facets</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string-interpolation string" style=color:#e3116c>f"</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>{</span><span class="token string-interpolation interpolation">jurisdiction</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>}</span><span class="token string-interpolation string" style=color:#e3116c>, </span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>{</span><span class="token string-interpolation interpolation">state</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>}</span><span class="token string-interpolation string" style=color:#e3116c>"</span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> count</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>return</span><span class="token plain"> facets</span><br/></div></code></pre></div></div> | |
| <p><strong>Implementation Priority</strong>: 🟡 <strong>MEDIUM</strong> - Valuable for end-users, but scraping comes first</p> | |
| <hr/> | |
| <h3 class="anchor anchorTargetStickyNavbar_Vzrq" id=6-opencouncil--international-adaptability>6. OpenCouncil 🆕 International Adaptability<a href=#6-opencouncil--international-adaptability class=hash-link aria-label="Direct link to 6. OpenCouncil 🆕 International Adaptability" title="Direct link to 6. OpenCouncil 🆕 International Adaptability" translate=no></a></h3> | |
| <p><strong>Website</strong>: <a href=https://opencouncil.gr target=_blank rel="noopener noreferrer" class="">https://opencouncil.gr</a><br/> | |
| <strong>GitHub</strong>: <a href=https://github.com/schemalabz/opencouncil target=_blank rel="noopener noreferrer" class="">https://github.com/schemalabz/opencouncil</a><br/> | |
| <strong>License</strong>: Open-source<br/> | |
| <strong>Focus</strong>: Greek councils, but adaptable to U.S.</p> | |
| <h4 class="anchor anchorTargetStickyNavbar_Vzrq" id=-what-to-adopt-4>🔥 What to Adopt<a href=#-what-to-adopt-4 class=hash-link aria-label="Direct link to 🔥 What to Adopt" title="Direct link to 🔥 What to Adopt" translate=no></a></h4> | |
| <p><strong>A. Internationalization Patterns</strong></p> | |
| <p>OpenCouncil works in Greece (different government structure). This teaches us:</p> | |
| <ul> | |
| <li class=""><strong>Flexible schema</strong> (not hardcoded to U.S. structures)</li> | |
| <li class=""><strong>Configurable jurisdiction types</strong> (councils, boards, commissions)</li> | |
| <li class=""><strong>Multi-language support</strong> (not needed now, but good architecture)</li> | |
| </ul> | |
| <div class="language-python codeBlockContainer_Ckt0 theme-code-block" style=--prism-color:#393A34;--prism-background-color:#f6f8fa><div class=codeBlockContent_QJqH><pre tabindex=0 class="prism-code language-python codeBlock_bY9V thin-scrollbar" style=color:#393A34;background-color:#f6f8fa><code class=codeBlockLines_e6Vv><div class=token-line style=color:#393A34><span class="token comment" style=color:#999988;font-style:italic># OpenCouncil pattern: Flexible jurisdiction configuration</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain" style=display:inline-block></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>from</span><span class="token plain"> enum </span><span class="token keyword" style=color:#00009f>import</span><span class="token plain"> Enum</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>from</span><span class="token plain"> dataclasses </span><span class="token keyword" style=color:#00009f>import</span><span class="token plain"> dataclass</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>from</span><span class="token plain"> typing </span><span class="token keyword" style=color:#00009f>import</span><span class="token plain"> List</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> Optional</span><br/></div><div class=token-line style=color:#393A34><span class="token plain" style=display:inline-block></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>class</span><span class="token plain"> </span><span class="token class-name">GovernmentLevel</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">Enum</span><span class="token punctuation" style=color:#393A34>)</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token triple-quoted-string string" style=color:#e3116c>"""</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> OpenCouncil pattern: Support multiple government structures.</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> MUNICIPAL </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"municipal"</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># City/town councils</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> COUNTY </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"county"</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># County boards</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> TOWNSHIP </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"township"</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Township boards</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> SCHOOL_DISTRICT </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"school"</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># School boards</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> SPECIAL_DISTRICT </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"special"</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Water, fire, etc.</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> STATE </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"state"</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># State agencies (future)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain" style=display:inline-block></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token decorator annotation punctuation" style=color:#393A34>@dataclass</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>class</span><span class="token plain"> </span><span class="token class-name">JurisdictionConfig</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token triple-quoted-string string" style=color:#e3116c>"""</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> OpenCouncil pattern: Configure each jurisdiction's unique structure.</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> jurisdiction_name</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">str</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> government_level</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> GovernmentLevel</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Meeting schedule</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> typical_meeting_frequency</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">str</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># 'weekly', 'biweekly', 'monthly'</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> typical_meeting_days</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> List</span><span class="token punctuation" style=color:#393A34>[</span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># ['Monday', 'Thursday']</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> typical_meeting_time</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">str</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># '18:00'</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Website structure</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> calendar_url</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> Optional</span><span class="token punctuation" style=color:#393A34>[</span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> agenda_url_pattern</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> Optional</span><span class="token punctuation" style=color:#393A34>[</span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Template: "https://example.gov/agenda-{date}"</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> minutes_url_pattern</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> Optional</span><span class="token punctuation" style=color:#393A34>[</span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Legislative bodies</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> bodies</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> List</span><span class="token punctuation" style=color:#393A34>[</span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># ['City Council', 'Planning Commission', 'Board of Health']</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># Custom fields</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> metadata</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">dict</span><span class="token plain"> </span><span class="token comment" style=color:#999988;font-style:italic># For jurisdiction-specific data</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain" style=display:inline-block></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token comment" style=color:#999988;font-style:italic># Example: Configure Birmingham, AL</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain">BIRMINGHAM_CONFIG </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> JurisdictionConfig</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> jurisdiction_name</span><span class="token operator" style=color:#393A34>=</span><span class="token string" style=color:#e3116c>"Birmingham"</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> government_level</span><span class="token operator" style=color:#393A34>=</span><span class="token plain">GovernmentLevel</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">MUNICIPAL</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> typical_meeting_frequency</span><span class="token operator" style=color:#393A34>=</span><span class="token string" style=color:#e3116c>'biweekly'</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> typical_meeting_days</span><span class="token operator" style=color:#393A34>=</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'Tuesday'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> typical_meeting_time</span><span class="token operator" style=color:#393A34>=</span><span class="token string" style=color:#e3116c>'18:00'</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> calendar_url</span><span class="token operator" style=color:#393A34>=</span><span class="token string" style=color:#e3116c>"https://birminghamal.gov/council/meetings"</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> bodies</span><span class="token operator" style=color:#393A34>=</span><span class="token punctuation" style=color:#393A34>[</span><span class="token string" style=color:#e3116c>'City Council'</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'Board of Health'</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'Planning Commission'</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> metadata</span><span class="token operator" style=color:#393A34>=</span><span class="token punctuation" style=color:#393A34>{</span><span class="token string" style=color:#e3116c>'population'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token number" style=color:#36acaa>200733</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'oral_health_priority'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'high'</span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token punctuation" style=color:#393A34>)</span><br/></div></code></pre></div></div> | |
| <p><strong>Implementation Priority</strong>: 🟢 <strong>LOW</strong> - Good architecture, but not urgent</p> | |
| <hr/> | |
| <h2 class="anchor anchorTargetStickyNavbar_Vzrq" id=-implementation-roadmap>🎯 Implementation Roadmap<a href=#-implementation-roadmap class=hash-link aria-label="Direct link to 🎯 Implementation Roadmap" title="Direct link to 🎯 Implementation Roadmap" translate=no></a></h2> | |
| <h3 class="anchor anchorTargetStickyNavbar_Vzrq" id=phase-1-ai-summarization-opentowns-pattern->Phase 1: AI Summarization (OpenTowns pattern) 🔥<a href=#phase-1-ai-summarization-opentowns-pattern- class=hash-link aria-label="Direct link to Phase 1: AI Summarization (OpenTowns pattern) 🔥" title="Direct link to Phase 1: AI Summarization (OpenTowns pattern) 🔥" translate=no></a></h3> | |
| <p><strong>Priority</strong>: HIGH<br/> | |
| <strong>Timeline</strong>: 1-2 weeks<br/> | |
| <strong>Depends on</strong>: Existing OpenAI integration</p> | |
| <div class="language-python codeBlockContainer_Ckt0 theme-code-block" style=--prism-color:#393A34;--prism-background-color:#f6f8fa><div class=codeBlockContent_QJqH><pre tabindex=0 class="prism-code language-python codeBlock_bY9V thin-scrollbar" style=color:#393A34;background-color:#f6f8fa><code class=codeBlockLines_e6Vv><div class=token-line style=color:#393A34><span class="token comment" style=color:#999988;font-style:italic># TODO: Implement in extraction/summarizer.py</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Generate executive summaries </span><span class="token keyword" style=color:#00009f>from</span><span class="token plain"> meeting transcripts</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Extract key decisions </span><span class="token keyword" style=color:#00009f>as</span><span class="token plain"> bullet points</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Identify health policy items</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Add quality validation </span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">MeetingBank patterns</span><span class="token punctuation" style=color:#393A34>)</span><br/></div></code></pre></div></div> | |
| <h3 class="anchor anchorTargetStickyNavbar_Vzrq" id=phase-2-keyword-alerts-opentowns-pattern->Phase 2: Keyword Alerts (OpenTowns pattern) 🔥<a href=#phase-2-keyword-alerts-opentowns-pattern- class=hash-link aria-label="Direct link to Phase 2: Keyword Alerts (OpenTowns pattern) 🔥" title="Direct link to Phase 2: Keyword Alerts (OpenTowns pattern) 🔥" translate=no></a></h3> | |
| <p><strong>Priority</strong>: HIGH<br/> | |
| <strong>Timeline</strong>: 1 week<br/> | |
| <strong>Depends on</strong>: Meeting data ingestion</p> | |
| <div class="language-python codeBlockContainer_Ckt0 theme-code-block" style=--prism-color:#393A34;--prism-background-color:#f6f8fa><div class=codeBlockContent_QJqH><pre tabindex=0 class="prism-code language-python codeBlock_bY9V thin-scrollbar" style=color:#393A34;background-color:#f6f8fa><code class=codeBlockLines_e6Vv><div class=token-line style=color:#393A34><span class="token comment" style=color:#999988;font-style:italic># TODO: Implement in alerts/keyword_monitor.py</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Define oral health keyword categories</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Pattern matching </span><span class="token keyword" style=color:#00009f>with</span><span class="token plain"> word boundaries</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Generate alerts </span><span class="token keyword" style=color:#00009f>for</span><span class="token plain"> users</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Email</span><span class="token operator" style=color:#393A34>/</span><span class="token plain">webhook notification system</span><br/></div></code></pre></div></div> | |
| <h3 class="anchor anchorTargetStickyNavbar_Vzrq" id=phase-3-scale-architecture-localview-pattern->Phase 3: Scale Architecture (LocalView pattern) 🔥<a href=#phase-3-scale-architecture-localview-pattern- class=hash-link aria-label="Direct link to Phase 3: Scale Architecture (LocalView pattern) 🔥" title="Direct link to Phase 3: Scale Architecture (LocalView pattern) 🔥" translate=no></a></h3> | |
| <p><strong>Priority</strong>: HIGH<br/> | |
| <strong>Timeline</strong>: 2 weeks<br/> | |
| <strong>Depends on</strong>: Platform scrapers</p> | |
| <div class="language-python codeBlockContainer_Ckt0 theme-code-block" style=--prism-color:#393A34;--prism-background-color:#f6f8fa><div class=codeBlockContent_QJqH><pre tabindex=0 class="prism-code language-python codeBlock_bY9V thin-scrollbar" style=color:#393A34;background-color:#f6f8fa><code class=codeBlockLines_e6Vv><div class=token-line style=color:#393A34><span class="token comment" style=color:#999988;font-style:italic># TODO: Implement in discovery/batch_processor.py</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Quality metrics per jurisdiction</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Batch processing </span><span class="token punctuation" style=color:#393A34>(</span><span class="token number" style=color:#36acaa>100</span><span class="token plain"> at a time</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Failure tracking </span><span class="token keyword" style=color:#00009f>and</span><span class="token plain"> retry</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Completeness scoring</span><br/></div></code></pre></div></div> | |
| <h3 class="anchor anchorTargetStickyNavbar_Vzrq" id=phase-4-multi-jurisdiction-search-civicband-pattern->Phase 4: Multi-Jurisdiction Search (CivicBand pattern) 🟡<a href=#phase-4-multi-jurisdiction-search-civicband-pattern- class=hash-link aria-label="Direct link to Phase 4: Multi-Jurisdiction Search (CivicBand pattern) 🟡" title="Direct link to Phase 4: Multi-Jurisdiction Search (CivicBand pattern) 🟡" translate=no></a></h3> | |
| <p><strong>Priority</strong>: MEDIUM<br/> | |
| <strong>Timeline</strong>: 2-3 weeks<br/> | |
| <strong>Depends on</strong>: Significant meeting data</p> | |
| <div class="language-python codeBlockContainer_Ckt0 theme-code-block" style=--prism-color:#393A34;--prism-background-color:#f6f8fa><div class=codeBlockContent_QJqH><pre tabindex=0 class="prism-code language-python codeBlock_bY9V thin-scrollbar" style=color:#393A34;background-color:#f6f8fa><code class=codeBlockLines_e6Vv><div class=token-line style=color:#393A34><span class="token comment" style=color:#999988;font-style:italic># TODO: Implement in search/federated_search.py</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Set up Elasticsearch </span><span class="token keyword" style=color:#00009f>or</span><span class="token plain"> Meilisearch</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Index </span><span class="token builtin">all</span><span class="token plain"> meetings</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Cross</span><span class="token operator" style=color:#393A34>-</span><span class="token plain">jurisdiction search API</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Jurisdiction faceting</span><br/></div></code></pre></div></div> | |
| <h3 class="anchor anchorTargetStickyNavbar_Vzrq" id=phase-5-quality-validation-meetingbank-pattern->Phase 5: Quality Validation (MeetingBank pattern) 🟡<a href=#phase-5-quality-validation-meetingbank-pattern- class=hash-link aria-label="Direct link to Phase 5: Quality Validation (MeetingBank pattern) 🟡" title="Direct link to Phase 5: Quality Validation (MeetingBank pattern) 🟡" translate=no></a></h3> | |
| <p><strong>Priority</strong>: MEDIUM<br/> | |
| <strong>Timeline</strong>: 1 week<br/> | |
| <strong>Depends on</strong>: AI summarization</p> | |
| <div class="language-python codeBlockContainer_Ckt0 theme-code-block" style=--prism-color:#393A34;--prism-background-color:#f6f8fa><div class=codeBlockContent_QJqH><pre tabindex=0 class="prism-code language-python codeBlock_bY9V thin-scrollbar" style=color:#393A34;background-color:#f6f8fa><code class=codeBlockLines_e6Vv><div class=token-line style=color:#393A34><span class="token comment" style=color:#999988;font-style:italic># TODO: Implement in extraction/quality_validator.py</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Summary length validation</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Key term extraction</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Repetition detection</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Structure checking</span><br/></div></code></pre></div></div> | |
| <h3 class="anchor anchorTargetStickyNavbar_Vzrq" id=phase-6-flexible-config-opencouncil-pattern->Phase 6: Flexible Config (OpenCouncil pattern) 🟢<a href=#phase-6-flexible-config-opencouncil-pattern- class=hash-link aria-label="Direct link to Phase 6: Flexible Config (OpenCouncil pattern) 🟢" title="Direct link to Phase 6: Flexible Config (OpenCouncil pattern) 🟢" translate=no></a></h3> | |
| <p><strong>Priority</strong>: LOW<br/> | |
| <strong>Timeline</strong>: 1 week<br/> | |
| <strong>Depends on</strong>: None</p> | |
| <div class="language-python codeBlockContainer_Ckt0 theme-code-block" style=--prism-color:#393A34;--prism-background-color:#f6f8fa><div class=codeBlockContent_QJqH><pre tabindex=0 class="prism-code language-python codeBlock_bY9V thin-scrollbar" style=color:#393A34;background-color:#f6f8fa><code class=codeBlockLines_e6Vv><div class=token-line style=color:#393A34><span class="token comment" style=color:#999988;font-style:italic># TODO: Implement in config/jurisdiction_configs.py</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Per</span><span class="token operator" style=color:#393A34>-</span><span class="token plain">jurisdiction configuration</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Meeting schedule patterns</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token operator" style=color:#393A34>-</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token plain"> Legislative body tracking</span><br/></div></code></pre></div></div> | |
| <hr/> | |
| <h2 class="anchor anchorTargetStickyNavbar_Vzrq" id=-comparison-with-existing-integration>📊 Comparison with Existing Integration<a href=#-comparison-with-existing-integration class=hash-link aria-label="Direct link to 📊 Comparison with Existing Integration" title="Direct link to 📊 Comparison with Existing Integration" translate=no></a></h2> | |
| <table><thead><tr><th>Capability<th>Original 5 Projects<th>New 6 Projects<th>Status<tbody><tr><td>Platform detection<td>✅ Civic Scraper<td>-<td><strong>Complete</strong><tr><td>Event schema<td>✅ City Scrapers<td>-<td><strong>Complete</strong><tr><td>Video ingestion<td>✅ CDP<td>✅ LocalView (scale)<td><strong>Need scale patterns</strong><tr><td>Matter tracking<td>✅ Engagic<td>-<td><strong>Complete</strong><tr><td>Person/vote tracking<td>✅ Councilmatic<td>-<td>Roadmapped<tr><td><strong>AI Summarization</strong><td>❌<td>✅ OpenTowns, MeetingBank<td><strong>TODO: High priority</strong><tr><td><strong>Keyword Alerts</strong><td>❌<td>✅ OpenTowns<td><strong>TODO: High priority</strong><tr><td><strong>Cross-jurisdiction search</strong><td>⚠️ Basic<td>✅ CivicBand<td><strong>TODO: Medium priority</strong><tr><td><strong>Quality metrics</strong><td>❌<td>✅ LocalView, MeetingBank<td><strong>TODO: Medium priority</strong><tr><td><strong>Batch processing</strong><td>⚠️ Basic<td>✅ LocalView<td><strong>TODO: High priority</strong></table> | |
| <hr/> | |
| <h2 class="anchor anchorTargetStickyNavbar_Vzrq" id=-quick-start-integrate-summarization>💻 Quick Start: Integrate Summarization<a href=#-quick-start-integrate-summarization class=hash-link aria-label="Direct link to 💻 Quick Start: Integrate Summarization" title="Direct link to 💻 Quick Start: Integrate Summarization" translate=no></a></h2> | |
| <p>Here's how to add OpenTowns-style summarization <strong>right now</strong>:</p> | |
| <div class="language-python codeBlockContainer_Ckt0 theme-code-block" style=--prism-color:#393A34;--prism-background-color:#f6f8fa><div class=codeBlockContent_QJqH><pre tabindex=0 class="prism-code language-python codeBlock_bY9V thin-scrollbar" style=color:#393A34;background-color:#f6f8fa><code class=codeBlockLines_e6Vv><div class=token-line style=color:#393A34><span class="token comment" style=color:#999988;font-style:italic># File: extraction/summarizer.py</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain" style=display:inline-block></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>from</span><span class="token plain"> openai </span><span class="token keyword" style=color:#00009f>import</span><span class="token plain"> OpenAI</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>from</span><span class="token plain"> models</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">meeting_event </span><span class="token keyword" style=color:#00009f>import</span><span class="token plain"> MeetingEvent</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>from</span><span class="token plain"> config</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">settings </span><span class="token keyword" style=color:#00009f>import</span><span class="token plain"> settings</span><br/></div><div class=token-line style=color:#393A34><span class="token plain" style=display:inline-block></span><br/></div><div class=token-line style=color:#393A34><span class="token plain">client </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> OpenAI</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">api_key</span><span class="token operator" style=color:#393A34>=</span><span class="token plain">settings</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">openai_api_key</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain" style=display:inline-block></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token keyword" style=color:#00009f>def</span><span class="token plain"> </span><span class="token function" style=color:#d73a49>summarize_meeting</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain">event</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> MeetingEvent</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> full_text</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token builtin">str</span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"> </span><span class="token operator" style=color:#393A34>-</span><span class="token operator" style=color:#393A34>></span><span class="token plain"> </span><span class="token builtin">dict</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token triple-quoted-string string" style=color:#e3116c>"""</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> Generate OpenTowns-style summary with oral health focus.</span><br/></div><div class=token-line style=color:#393A34><span class="token triple-quoted-string string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> prompt </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> </span><span class="token string-interpolation string" style=color:#e3116c>f"""</span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> You are summarizing a local government meeting for public health advocates.</span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> </span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> Meeting: </span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>{</span><span class="token string-interpolation interpolation">event</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>.</span><span class="token string-interpolation interpolation">title</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>}</span><span class="token string-interpolation string" style=color:#e3116c></span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> Jurisdiction: </span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>{</span><span class="token string-interpolation interpolation">event</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>.</span><span class="token string-interpolation interpolation">jurisdiction_name</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>}</span><span class="token string-interpolation string" style=color:#e3116c>, </span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>{</span><span class="token string-interpolation interpolation">event</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>.</span><span class="token string-interpolation interpolation">state_code</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>}</span><span class="token string-interpolation string" style=color:#e3116c></span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> Date: </span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>{</span><span class="token string-interpolation interpolation">event</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>.</span><span class="token string-interpolation interpolation">start</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>.</span><span class="token string-interpolation interpolation">strftime</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>(</span><span class="token string-interpolation interpolation string" style=color:#e3116c>'%B %d, %Y'</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>)</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>}</span><span class="token string-interpolation string" style=color:#e3116c></span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> </span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> Full text (first 8000 chars):</span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> </span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>{</span><span class="token string-interpolation interpolation">full_text</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>[</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>:</span><span class="token string-interpolation interpolation format-spec">8000]</span><span class="token string-interpolation interpolation punctuation" style=color:#393A34>}</span><span class="token string-interpolation string" style=color:#e3116c></span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> </span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> Provide:</span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> 1. Executive Summary (2-3 sentences)</span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> 2. Key Decisions (bullet list)</span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> 3. Oral Health Items (if any - fluoridation, dental access, etc.)</span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> 4. Next Actions (follow-ups, future meetings)</span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> </span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> Focus on: What was decided? What's happening next?</span><br/></div><div class=token-line style=color:#393A34><span class="token string-interpolation string" style=color:#e3116c> """</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> response </span><span class="token operator" style=color:#393A34>=</span><span class="token plain"> client</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">chat</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">completions</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">create</span><span class="token punctuation" style=color:#393A34>(</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> model</span><span class="token operator" style=color:#393A34>=</span><span class="token string" style=color:#e3116c>"gpt-4o-mini"</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> messages</span><span class="token operator" style=color:#393A34>=</span><span class="token punctuation" style=color:#393A34>[</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token string" style=color:#e3116c>"role"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"system"</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"content"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"You summarize local government meetings for public understanding."</span><span class="token punctuation" style=color:#393A34>}</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token string" style=color:#e3116c>"role"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"user"</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>"content"</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> prompt</span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> temperature</span><span class="token operator" style=color:#393A34>=</span><span class="token number" style=color:#36acaa>0.3</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token keyword" style=color:#00009f>return</span><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>{</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'summary'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> response</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">choices</span><span class="token punctuation" style=color:#393A34>[</span><span class="token number" style=color:#36acaa>0</span><span class="token punctuation" style=color:#393A34>]</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">message</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">content</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'model'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> </span><span class="token string" style=color:#e3116c>'gpt-4o-mini'</span><span class="token punctuation" style=color:#393A34>,</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token string" style=color:#e3116c>'tokens_used'</span><span class="token punctuation" style=color:#393A34>:</span><span class="token plain"> response</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">usage</span><span class="token punctuation" style=color:#393A34>.</span><span class="token plain">total_tokens</span><br/></div><div class=token-line style=color:#393A34><span class="token plain"> </span><span class="token punctuation" style=color:#393A34>}</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain" style=display:inline-block></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token comment" style=color:#999988;font-style:italic># Usage:</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token comment" style=color:#999988;font-style:italic># summary = summarize_meeting(event, full_transcript)</span><span class="token plain"></span><br/></div><div class=token-line style=color:#393A34><span class="token plain"></span><span class="token comment" style=color:#999988;font-style:italic># event.description = summary['summary']</span><br/></div></code></pre></div></div> | |
| <hr/> | |
| <h2 class="anchor anchorTargetStickyNavbar_Vzrq" id=-next-steps>🎬 Next Steps<a href=#-next-steps class=hash-link aria-label="Direct link to 🎬 Next Steps" title="Direct link to 🎬 Next Steps" translate=no></a></h2> | |
| <ol> | |
| <li class=""><strong>Implement AI summarization</strong> (OpenTowns pattern) → Makes data usable</li> | |
| <li class=""><strong>Add keyword alerts</strong> (OpenTowns pattern) → Engage advocates</li> | |
| <li class=""><strong>Add batch processing</strong> (LocalView pattern) → Scale to 1,000+ jurisdictions</li> | |
| <li class=""><strong>Build search interface</strong> (CivicBand pattern) → User discovery</li> | |
| <li class=""><strong>Add quality metrics</strong> (LocalView + MeetingBank) → Monitor data health</li> | |
| </ol> | |
| <hr/> | |
| <h2 class="anchor anchorTargetStickyNavbar_Vzrq" id=-references>📖 References<a href=#-references class=hash-link aria-label="Direct link to 📖 References" title="Direct link to 📖 References" translate=no></a></h2> | |
| <ul> | |
| <li class=""><strong>OpenTowns</strong>: <a href=https://opentowns.org target=_blank rel="noopener noreferrer" class="">https://opentowns.org</a></li> | |
| <li class=""><strong>LocalView</strong>: <a href=https://www.localview.net target=_blank rel="noopener noreferrer" class="">https://www.localview.net</a></li> | |
| <li class=""><strong>MeetingBank</strong>: <a href=https://meetingbank.github.io target=_blank rel="noopener noreferrer" class="">https://meetingbank.github.io</a></li> | |
| <li class=""><strong>CivicBand</strong>: <a href=https://civic.band target=_blank rel="noopener noreferrer" class="">https://civic.band</a></li> | |
| <li class=""><strong>OpenCouncil</strong>: <a href=https://github.com/schemalabz/opencouncil target=_blank rel="noopener noreferrer" class="">https://github.com/schemalabz/opencouncil</a></li> | |
| <li class=""><strong>Council Data Project</strong>: <a href=https://councildataproject.org target=_blank rel="noopener noreferrer" class="">https://councildataproject.org</a> (see INTEGRATION_GUIDE.md)</li> | |
| </ul> | |
| <hr/> | |
| <h2 class="anchor anchorTargetStickyNavbar_Vzrq" id=-license--attribution>📝 License & Attribution<a href=#-license--attribution class=hash-link aria-label="Direct link to 📝 License & Attribution" title="Direct link to 📝 License & Attribution" translate=no></a></h2> | |
| <p>All patterns documented here are derived from open-source projects:</p> | |
| <ul> | |
| <li class="">OpenTowns: Open civic-tech project</li> | |
| <li class="">LocalView: Open-source (Harvard Mellon Urbanism)</li> | |
| <li class="">MeetingBank: Open dataset</li> | |
| <li class="">CivicBand: Open-source (Raft Foundation)</li> | |
| <li class="">OpenCouncil: Open-source (MIT)</li> | |
| <li class="">CDP: MIT License</li> | |
| </ul> | |
| <p>When using code patterns, maintain attribution per each project's license.</div><footer class="theme-doc-footer docusaurus-mt-lg"><div class="row margin-top--sm theme-doc-footer-edit-meta-row"><div class="col noPrint_WFHX"><a href=https://github.com/getcommunityone/open-navigator-for-engagement/tree/main/website/docs/guides/search-patterns.md target=_blank rel="noopener noreferrer" class=theme-edit-this-page><svg fill=currentColor height=20 width=20 viewBox="0 0 40 40" class=iconEdit_Z9Sw aria-hidden=true><g><path d="m34.5 11.7l-3 3.1-6.3-6.3 3.1-3q0.5-0.5 1.2-0.5t1.1 0.5l3.9 3.9q0.5 0.4 0.5 1.1t-0.5 1.2z m-29.5 17.1l18.4-18.5 6.3 6.3-18.4 18.4h-6.3v-6.2z"/></g></svg>Edit this page</a></div><div class="col lastUpdated_JAkA"></div></div></footer></article><nav class="docusaurus-mt-lg pagination-nav" aria-label="Docs pages"><a class="pagination-nav__link pagination-nav__link--prev" href=/docs/guides/scraper-improvements><div class=pagination-nav__sublabel>Previous</div><div class=pagination-nav__label>Scraper Improvements Summary</div></a><a class="pagination-nav__link pagination-nav__link--next" href=/docs/guides/split-screen><div class=pagination-nav__sublabel>Next</div><div class=pagination-nav__label>Split-Screen System: Government Decisions ↔ Community Response</div></a></nav></div></div><div class="col col--3"><div class="tableOfContents_bqdL thin-scrollbar theme-doc-toc-desktop"><ul class="table-of-contents table-of-contents__left-border"><li><a href=#-whats-new-vs-our-existing-integration class="table-of-contents__link toc-highlight">🎯 What's NEW vs. Our Existing Integration</a><li><a href=#-project-analysis class="table-of-contents__link toc-highlight">📚 Project Analysis</a><ul><li><a href=#1-council-data-project-cdp--already-integrated class="table-of-contents__link toc-highlight">1. Council Data Project (CDP) ⭐ Already Integrated</a><li><a href=#2-opentowns--ai-summarization-pioneer class="table-of-contents__link toc-highlight">2. OpenTowns 🆕 AI Summarization Pioneer</a><ul><li><a href=#-what-to-adopt class="table-of-contents__link toc-highlight">🔥 What to Adopt</a></ul><li><a href=#3-localview--research-grade-scale class="table-of-contents__link toc-highlight">3. LocalView 🆕 Research-Grade Scale</a><ul><li><a href=#-what-to-adopt-1 class="table-of-contents__link toc-highlight">🔥 What to Adopt</a></ul><li><a href=#4-meetingbank--summarization-research class="table-of-contents__link toc-highlight">4. MeetingBank 🆕 Summarization Research</a><ul><li><a href=#-what-to-adopt-2 class="table-of-contents__link toc-highlight">🔥 What to Adopt</a></ul><li><a href=#5-civicband--multi-jurisdiction-search class="table-of-contents__link toc-highlight">5. CivicBand 🆕 Multi-Jurisdiction Search</a><ul><li><a href=#-what-to-adopt-3 class="table-of-contents__link toc-highlight">🔥 What to Adopt</a></ul><li><a href=#6-opencouncil--international-adaptability class="table-of-contents__link toc-highlight">6. OpenCouncil 🆕 International Adaptability</a><ul><li><a href=#-what-to-adopt-4 class="table-of-contents__link toc-highlight">🔥 What to Adopt</a></ul></ul><li><a href=#-implementation-roadmap class="table-of-contents__link toc-highlight">🎯 Implementation Roadmap</a><ul><li><a href=#phase-1-ai-summarization-opentowns-pattern- class="table-of-contents__link toc-highlight">Phase 1: AI Summarization (OpenTowns pattern) 🔥</a><li><a href=#phase-2-keyword-alerts-opentowns-pattern- class="table-of-contents__link toc-highlight">Phase 2: Keyword Alerts (OpenTowns pattern) 🔥</a><li><a href=#phase-3-scale-architecture-localview-pattern- class="table-of-contents__link toc-highlight">Phase 3: Scale Architecture (LocalView pattern) 🔥</a><li><a href=#phase-4-multi-jurisdiction-search-civicband-pattern- class="table-of-contents__link toc-highlight">Phase 4: Multi-Jurisdiction Search (CivicBand pattern) 🟡</a><li><a href=#phase-5-quality-validation-meetingbank-pattern- class="table-of-contents__link toc-highlight">Phase 5: Quality Validation (MeetingBank pattern) 🟡</a><li><a href=#phase-6-flexible-config-opencouncil-pattern- class="table-of-contents__link toc-highlight">Phase 6: Flexible Config (OpenCouncil pattern) 🟢</a></ul><li><a href=#-comparison-with-existing-integration class="table-of-contents__link toc-highlight">📊 Comparison with Existing Integration</a><li><a href=#-quick-start-integrate-summarization class="table-of-contents__link toc-highlight">💻 Quick Start: Integrate Summarization</a><li><a href=#-next-steps class="table-of-contents__link toc-highlight">🎬 Next Steps</a><li><a href=#-references class="table-of-contents__link toc-highlight">📖 References</a><li><a href=#-license--attribution class="table-of-contents__link toc-highlight">📝 License & Attribution</a></ul></div></div></div></div></main></div></div></div><footer class="theme-layout-footer footer footer--dark"><div class="container container-fluid"><div class="row footer__links"><div class="theme-layout-footer-column col footer__col"><div class=footer__title>Documentation</div><ul class="footer__items clean-list"><li class=footer__item><a class=footer__link-item href=/docs/intro>Getting Started</a><li class=footer__item><a class=footer__link-item href=/docs/data-sources/citations>Citations & Data Sources</a><li class=footer__item><a class=footer__link-item href=/docs/data-sources/overview>Data Sources</a><li class=footer__item><a class=footer__link-item href=/docs/for-developers>For Developers</a></ul></div><div class="theme-layout-footer-column col footer__col"><div class=footer__title>Resources</div><ul class="footer__items clean-list"><li class=footer__item><a href=https://www.communityone.com target=_blank rel="noopener noreferrer" class=footer__link-item>Launch Open Navigator<svg width=13.5 height=13.5 aria-label="(opens in new tab)" class=iconExternalLink_nPIU><use href=#theme-svg-external-link /></svg></a><li class=footer__item><a href=https://github.com/getcommunityone/open-navigator-for-engagement target=_blank rel="noopener noreferrer" class=footer__link-item>GitHub<svg width=13.5 height=13.5 aria-label="(opens in new tab)" class=iconExternalLink_nPIU><use href=#theme-svg-external-link /></svg></a><li class=footer__item><a href=https://www.groundvue.org/ target=_blank rel="noopener noreferrer" class=footer__link-item>GroundVue (Partner)<svg width=13.5 height=13.5 aria-label="(opens in new tab)" class=iconExternalLink_nPIU><use href=#theme-svg-external-link /></svg></a></ul></div><div class="theme-layout-footer-column col footer__col"><div class=footer__title>Community</div><ul class="footer__items clean-list"><li class=footer__item><a href=https://www.instagram.com/getcommunityone/ target=_blank rel="noopener noreferrer" class=footer__link-item>Instagram<svg width=13.5 height=13.5 aria-label="(opens in new tab)" class=iconExternalLink_nPIU><use href=#theme-svg-external-link /></svg></a><li class=footer__item><a href=https://www.facebook.com/getcommunityone target=_blank rel="noopener noreferrer" class=footer__link-item>Facebook<svg width=13.5 height=13.5 aria-label="(opens in new tab)" class=iconExternalLink_nPIU><use href=#theme-svg-external-link /></svg></a><li class=footer__item><a href=https://x.com/getcommunityone/ target=_blank rel="noopener noreferrer" class=footer__link-item>X (Twitter)<svg width=13.5 height=13.5 aria-label="(opens in new tab)" class=iconExternalLink_nPIU><use href=#theme-svg-external-link /></svg></a><li class=footer__item><a href=https://www.linkedin.com/company/getcommunityone target=_blank rel="noopener noreferrer" class=footer__link-item>LinkedIn<svg width=13.5 height=13.5 aria-label="(opens in new tab)" class=iconExternalLink_nPIU><use href=#theme-svg-external-link /></svg></a><li class=footer__item><a href=https://www.youtube.com/@getcommunityone target=_blank rel="noopener noreferrer" class=footer__link-item>YouTube<svg width=13.5 height=13.5 aria-label="(opens in new tab)" class=iconExternalLink_nPIU><use href=#theme-svg-external-link /></svg></a><li class=footer__item><a href=https://discord.gg/uH6Dytek target=_blank rel="noopener noreferrer" class=footer__link-item>Discord<svg width=13.5 height=13.5 aria-label="(opens in new tab)" class=iconExternalLink_nPIU><use href=#theme-svg-external-link /></svg></a></ul></div><div class="theme-layout-footer-column col footer__col"><div class=footer__title>Legal</div><ul class="footer__items clean-list"><li class=footer__item><a class=footer__link-item href=/docs/legal/privacy-policy>Privacy Policy</a><li class=footer__item><a class=footer__link-item href=/docs/legal/terms-of-service>Terms of Service</a><li class=footer__item><a class=footer__link-item href=/docs/legal/data-provider-terms>Data Provider Terms</a></ul></div><div class="theme-layout-footer-column col footer__col"><div class=footer__title>More</div><ul class="footer__items clean-list"><li class=footer__item><a class=footer__link-item href=/blog>Blog</a><li class=footer__item><a href=https://github.com/getcommunityone/open-navigator-for-engagement/blob/main/LICENSE target=_blank rel="noopener noreferrer" class=footer__link-item>License (MIT)<svg width=13.5 height=13.5 aria-label="(opens in new tab)" class=iconExternalLink_nPIU><use href=#theme-svg-external-link /></svg></a></ul></div></div><div class="footer__bottom text--center"><div class=footer__copyright>Copyright © 2026 Community One. Built with Docusaurus.</div></div></div></footer></div></body> |