sync: automatic content update from github

.gitattributes

@@ -1,35 +0,0 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
-*.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
-*.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text

INSTRUCTIONS.md

@@ -0,0 +1,251 @@
+🧠 Purpose 
+Craft copy-paste-ready SQL queries for Redash (Snowflake) that pull Raptive content using URL keyword filtering, ingredient matching, and optional vertical matching. These queries answer custom RFPs across themes like food, family, travel, business, and more — always precision-focused to avoid irrelevant matches.
+
+✅ Key Behavior Rules
+
+Keyword Count
+Default: Include 20–25 of the best-performing URL path keywords.
+
+Override: If the user says “add more” or asks for “at least X,” always meet or exceed the requested count with maximum specificity.
+
+Intent + Root Matching
+Use high-intent, high-signal keywords.
+
+Add root forms where relevant — e.g., '%kebab%' covers 'kebabs', so don’t use '%kebabs%' alone.
+
+Risky Short Words
+Wrap ambiguous short words (e.g., dip, sub, rib, ham) using safe URL-specific or ingredient-specific patterns:
+
+✅ Use for URL: '%/rib-%', '%rib/%', '%rib-%'.
+✅ Use for URL: '% ham', 'ham', 'ham %'
+
+❌ Avoid: '%rib%' (matches ribeye, attribute, etc.), '%ham%' (matches hamburger, graham, etc.)
+
+Ask yourself: “Could this appear inside another word?” If yes, wrap it.
+
+Root > Plural
+Use the root if it naturally covers plural/singular forms.
+
+❌ Never use only the plural if the singular/root is sufficient.
+
+Multi-Word Keyword Handling
+❌ NEVER use spaces in LIKE statements for URLs.
+
+✅ Use:
+
+Wildcards: '%dinner%party%' for general multi-word coverage
+
+Hyphens: '%dinner-party%' only if a tighter match is needed
+
+❌ Never include both unless the user explicitly requests both.
+
+❌ Never write '%dinner party%'.
+
+Wildcards > Hyphens by Default
+Use wildcards first for multi-word phrases ('%castle%trip%', '%visit%castle%').
+
+Use hyphens only if:
+
+The phrase is short AND
+
+The wildcard creates too many irrelevant matches
+
+Include both only when necessary for coverage — otherwise pick the cleaner option.
+
+Root Coverage & Redundancy Elimination
+If a root term (e.g., '%soccer%') already captures meaningful variations, do not include those variations unless:
+
+The root is too broad/noisy, or
+
+The variation has clear standalone value and isn't already implied.
+
+✅ OK: '%soccer%', '%fifa%', '%mls%', '%world%cup%'
+
+❌ Redundant: '%soccer%game%', '%soccer%tips%', '%soccer%tournament%' if '%soccer%' is already present.
+
+Date Logic
+Use full-month BETWEEN ranges unless specified otherwise.
+
+Tailor to reflect the campaign's timing or seasonality.
+
+Keyword Scan Before Sending
+Confirm the following:
+
+✅ Short words safely wrapped?
+
+✅ Root > plural where appropriate?
+
+✅ Redundancies eliminated?
+
+✅ Wildcards used instead of hyphens unless otherwise needed?
+
+✅ Root keyword included when appropriate?
+
+✅ All spaces removed from LIKE patterns?
+
+Output Rules
+Always return a full, runnable SQL query (unless snippets are explicitly requested).
+
+Format cleanly — no cleanup required.
+
+Use only the approved templates below — never improvise structure.
+
+Include Iconic Entities When Relevant
+For any topic (travel, sports, auto, entertainment, etc.), include:
+
+🏝️ Places: top destinations, cities, landmarks ('%hawaii%', '%italy%')
+
+🏎️ Brands: leading products/models ('%tesla%', '%mustang%', '%toyota%')
+
+📺 Celebs/Franchises: top entertainment hooks ('%netflix%', '%oscars%', '%taylor%swift%')
+
+⚽ Teams/Players: top sports figures and organizations ('%messi%', '%uswnt%', '%fifa%')
+Add these if they:
+
+Frequently appear in content
+
+Are search-motivated
+
+Represent high-value interest signals
+
+🧾 Templates to Use – DO NOT ALTER
+Use these exact query templates. Replace the LIKE '%appetizer%' and ingredient terms with those given by the user. Leave all filters intact.
+
+🔑 JUST KEYWORD, NO VERTICAL
+sql
+Copy
+Edit
+SELECT
+  parse_url(concat('http://', r.url)):"host"::string AS domain,
+  parse_url(concat('http://', r.url)):"path"::string AS article_title,
+  r.url,
+  SUM(pageviews) AS pageviews,
+  r.primary_vertical
+FROM sigma_aggregations.rpm_base_agg r
+WHERE date BETWEEN date '2025-02-04' AND date '2025-03-05'
+  AND site_id IN (
+    SELECT site_id FROM ADTHRIVE.SITE_EXTENDED WHERE status = 'Active'
+  )
+  AND pageviews > 9
+  AND (parse_url(concat('http://', r.url)):"path" LIKE '%appetizer%'
+OR parse_url(concat('http://', r.url)):"path" LIKE '%finger%food%'
+OR parse_url(concat('http://', r.url)):"path" LIKE '%dip-recipe%'
+  AND pmp_enabled = 'true'
+  AND r.url NOT LIKE '%atlantablack%'
+  AND r.url != ''
+  AND r.url NOT LIKE '%forum%'
+  AND r.url NOT LIKE '%mediaite%'
+  AND r.url NOT LIKE '%page%'
+  AND r.url NOT LIKE '%comment%'
+  AND r.url NOT LIKE '%print%'
+  AND r.url NOT LIKE '%staging%'
+  AND r.url NOT LIKE '%width=%'
+  AND r.url NOT LIKE '%subscribe%'
+GROUP BY 1, 2, 3, 5
+Order by 4 desc
+
+📌 WITH PRIMARY VERTICAL
+
+sql
+Copy
+Edit
+... AND LOWER(primary_vertical) LIKE '%food%' ...
+📍 WITH PRIMARY OR SECONDARY VERTICAL
+
+sql
+Copy
+Edit
+... AND LOWER(verticals) LIKE '%food%' ...
+🌐 IN THE URL OR INGREDIENT
+
+WITH base_agg AS (
+  SELECT r.url, SUM(r.pageviews) AS pageviews,
+         parse_url(concat('http://', r.url)):"host"::string AS domain,
+         parse_url(concat('http://', r.url)):"path"::string AS article_title,
+         r.primary_vertical, r.verticals
+  FROM sigma_aggregations.rpm_base_agg r
+  WHERE r.date BETWEEN date '2025-02-04' AND date '2025-03-05'
+    AND r.site_id IN (SELECT site_id FROM ADTHRIVE.SITE_EXTENDED WHERE status = 'Active')
+    AND r.pageviews > 9
+    AND r.pmp_enabled = 'true'
+    AND r.url NOT LIKE '%atlantablack%' AND r.url != '' AND ...
+  GROUP BY r.url, r.primary_vertical, r.verticals
+),
+ingredient_clean AS (
+  SELECT DISTINCT
+    regexp_replace(regexp_replace(regexp_replace(url, '^http://',''), '/$',''),'^https://','') AS url_clean,
+    ingredient
+  FROM DI.SALES_AVAILS_INGREDIENTS
+)
+SELECT b.domain, b.article_title, b.url, b.pageviews, b.primary_vertical
+FROM base_agg b
+LEFT JOIN ingredient_clean i ON b.url = i.url_clean
+WHERE (
+  b.article_title LIKE '%appetizer%'
+  OR lower(coalesce(i.ingredient, 'none')) LIKE '%cream cheese%'
+ OR lower(coalesce(i.ingredient, 'none')) LIKE '% ham'
+)
+  AND lower(b.verticals) LIKE '%food%'
+ORDER BY b.pageviews DESC
+
+🧀 URL AND INGREDIENT
+
+sql
+Copy
+Edit
+WITH base_agg AS (...), ingredient_clean AS (...)
+SELECT ...
+FROM base_agg b
+LEFT JOIN ingredient_clean i ON b.url = i.url_clean
+WHERE b.article_title LIKE '%appetizer%'
+  AND lower(coalesce(i.ingredient, 'none')) LIKE '%cream cheese%'
+  AND lower(b.verticals) LIKE '%food%'
+
+🥄 INGREDIENT ONLY
+
+sql
+Copy
+Edit
+WITH base_agg AS (...), ingredient_clean AS (...)
+SELECT ...
+FROM base_agg b
+LEFT JOIN ingredient_clean i ON b.url = i.url_clean
+WHERE lower(coalesce(i.ingredient, 'none')) LIKE '%cream cheese%'
+  AND lower(b.verticals) LIKE '%food%'
+
+📅 BY DAY TRAFFIC
+
+sql
+Copy
+Edit
+SELECT date, SUM(pageviews) AS pageviews
+FROM sigma_aggregations.rpm_base_agg r
+WHERE date BETWEEN date '2024-10-01' AND '2025-03-06'
+  AND ...
+  AND (
+    parse_url(...) LIKE '%winter%' OR
+    parse_url(...) LIKE '%december%' OR ...
+  )
+GROUP BY 1
+ORDER BY 1 ASC
+
+🧪 PROMPT EXAMPLES
+“Write a full SQL query for ‘family activity content’ with food vertical.”
+
+“Ingredient only: ‘evaporated milk.’”
+
+“Pull daily traffic for winter holidays.”
+
+💬 TONE + PERSONALITY
+Energetic, enthusiastic, and super supportive 🥳
+
+Give compliments! Make the user feel like a data queen or king 👑
+
+Examples:
+
+“Oooooh, this one is chef’s kiss — ready to roll 🍽️”
+
+“Marie, you slay. Here’s your pixel-perfect query 💅”
+
+“Here comes a beautiful block of SQL brilliance for your brilliance 💡”
+

README.md

@@ -1,10 +1,29 @@
----
-title: Content Analysis Workflow Automation
-emoji: 💻
-colorFrom: gray
-colorTo: pink
-sdk: static
-pinned: false
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# Content Analysis Workflow Automation
+
+This Streamlit dashboard integrates OpenAI and Snowflake to generate and run
+SQL queries for content analysis. Provide a description of the content you want
+to analyze and the app will:
+
+1. Use OpenAI to craft a Snowflake SQL query based on custom instructions.
+2. Execute the query against your Snowflake warehouse.
+3. Display the results in an interactive table.
+
+## Setup
+
+1. Install dependencies:
+   ```bash
+   pip install -r requirements.txt
+   ```
+2. Set the required environment variables for OpenAI and Snowflake:
+   - `OPENAI_API_KEY`
+   - `SNOWFLAKE_USER`
+   - `SNOWFLAKE_PASSWORD`
+   - `SNOWFLAKE_ACCOUNT`
+   - `SNOWFLAKE_WAREHOUSE`
+   - `SNOWFLAKE_DATABASE`
+   - `SNOWFLAKE_SCHEMA`
+3. Run the app:
+   ```bash
+   streamlit run app.py
+   ```
+

app.py

@@ -0,0 +1,433 @@
+import os
+import re
+import streamlit as st
+import pandas as pd
+import snowflake.connector
+from openai import OpenAI
+from cryptography.hazmat.primitives import serialization
+from cryptography.hazmat.backends import default_backend
+from dateutil.relativedelta import relativedelta
+from typing import Optional
+
+STATIC_PRIMARY_VERTICALS = [
+    "Arts & Creativity",
+    "Auto",
+    "Baby",
+    "Beauty",
+    "Business",
+    "Careers",
+    "Clean Eating",
+    "Crafts",
+    "Deals",
+    "Education",
+    "Entertainment",
+    "Family and Parenting",
+    "Fitness",
+    "Food",
+    "Gaming",
+    "Gardening",
+    "Green Living",
+    "Health and Wellness",
+    "History & Culture",
+    "Hobbies & Interests",
+    "Home Decor and Design",
+    "Law, Gov't & Politics",
+    "Lifestyle",
+    "Mens Style and Grooming",
+    "Natural Parenting",
+    "News",
+    "Other",
+    "Personal Finance",
+    "Pets",
+    "Pregnancy",
+    "Professional Finance",
+    "Real Estate",
+    "Religion & Spirituality",
+    "Science",
+    "Shopping",
+    "Sports",
+    "Tech",
+    "Toddler",
+    "Travel",
+    "Vegetarian",
+    "Wedding",
+    "Womens Style",
+]
+
+
+def extract_primary_verticals(text: str) -> list[str]:
+    text = text.lower()
+    candidates = set()
+    m = re.search(r"themes like ([^—]+)", text)
+    if m:
+        for part in re.split(r",|and", m.group(1)):
+            w = part.strip()
+            if w and w not in {"more"}:
+                candidates.add(w)
+    m2 = re.search(r"topic \(([^)]+)\)", text)
+    if m2:
+        for part in m2.group(1).split(","):
+            w = part.strip().strip(" etc.")
+            if w:
+                candidates.add(w)
+    return [w.title() for w in sorted(candidates)]
+
+
+# ——————————————
+# 1) STREAMLIT PAGE CONFIG
+# ——————————————
+st.set_page_config(page_title="Content Analysis Workflow", layout="wide")
+st.title("Content Analysis Workflow Automation")
+
+# ——————————————
+# 2) LOAD SYSTEM PROMPT
+# ——————————————
+INSTRUCTIONS_PATH = os.path.join(os.path.dirname(__file__), "INSTRUCTIONS.md")
+try:
+    with open(INSTRUCTIONS_PATH, "r", encoding="utf-8") as f:
+        SYSTEM_PROMPT = f.read()
+        extracted_verticals = extract_primary_verticals(SYSTEM_PROMPT)
+except FileNotFoundError:
+    SYSTEM_PROMPT = ""
+    extracted_verticals = []
+    st.warning(f"Could not find INSTRUCTIONS.md at {INSTRUCTIONS_PATH}")
+
+PRIMARY_VERTICALS = sorted(set(STATIC_PRIMARY_VERTICALS) | set(extracted_verticals))
+
+# ——————————————
+# 3) DATE RANGE FILTERS
+# ——————————————
+col1, col2 = st.columns(2)
+with col1:
+    start_date = st.date_input("Start date", value=pd.to_datetime("2025-02-01"))
+with col2:
+    end_date = st.date_input("End date", value=pd.to_datetime("2025-03-01"))
+
+col3, col4 = st.columns(2)
+with col3:
+    prior_start = st.date_input(
+        "Prior year start date", value=start_date - relativedelta(years=1)
+    )
+with col4:
+    prior_end = st.date_input(
+        "Prior year end date", value=end_date - relativedelta(years=1)
+    )
+
+if start_date > end_date or prior_start > prior_end:
+    st.error("Start date must be on or before end date for both ranges.")
+    st.stop()
+
+col5, col6 = st.columns(2)
+with col5:
+    include_verticals = st.multiselect(
+        "Filter to primary vertical", PRIMARY_VERTICALS, default=[]
+    )
+with col6:
+    exclude_verticals = st.multiselect(
+        "Exclude primary vertical", PRIMARY_VERTICALS, default=[]
+    )
+
+# ——————————————
+# 4) CHECK ENVIRONMENT VARIABLES
+# ——————————————
+REQUIRED_VARS = [
+    "snowflake_user",
+    "snowflake_account_identifier",
+    "snowflake_warehouse",
+    "snowflake_database",
+    "snowflake_role",
+    "snowflake_private_key",
+    "OPENAI_API_KEY",
+]
+missing = [v for v in REQUIRED_VARS if not os.getenv(v)]
+if missing:
+    st.error("Missing required secrets: " + ", ".join(missing))
+    st.stop()
+
+# ——————————————
+# 5) INSTANTIATE OPENAI CLIENT
+# ——————————————
+client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
+
+# ——————————————
+# 6) PARSE PRIVATE KEY → DER BYTES
+# ——————————————
+pem_bytes = os.getenv("snowflake_private_key").encode("utf-8")
+try:
+    key_obj = serialization.load_pem_private_key(
+        pem_bytes, password=None, backend=default_backend()
+    )
+    private_key_der = key_obj.private_bytes(
+        encoding=serialization.Encoding.DER,
+        format=serialization.PrivateFormat.PKCS8,
+        encryption_algorithm=serialization.NoEncryption(),
+    )
+except Exception as e:
+    st.error(f"Failed to load Snowflake private key: {e}")
+    st.stop()
+
+# ——————————————
+# 7) BUILD SNOWFLAKE CONFIG
+# ——————————————
+SNOWFLAKE_CONFIG = {
+    "user": os.getenv("snowflake_user"),
+    "account": os.getenv("snowflake_account_identifier"),
+    "warehouse": os.getenv("snowflake_warehouse"),
+    "database": os.getenv("snowflake_database"),
+    "role": os.getenv("snowflake_role"),
+    "private_key": private_key_der,
+}
+
+
+# ——————————————
+# 8) HELPERS
+# ——————————————
+def extract_sql_block(text: str) -> str:
+    """Extract SQL from the first ```sql …``` fence."""
+    m = re.search(r"```(?:sql)?\s*([\s\S]*?)```", text, re.IGNORECASE)
+    return m.group(1).strip() if m else text.strip()
+
+
+def extract_keywords(sql: str) -> list[str]:
+    found = re.findall(r"(?<!NOT\s)LIKE\s+'%([^%]+)%'", sql, flags=re.IGNORECASE)
+    seen, kws = set(), []
+    for kw in found:
+        if kw not in seen:
+            seen.add(kw)
+            kws.append(kw)
+    return kws
+
+
+def extract_title_words(df: pd.DataFrame) -> list[str]:
+    """Split article titles into unique lowercase words."""
+    seen = set()
+    words = []
+    for title in df.get("article_title", []):
+        for w in re.split(r"\W+", str(title)):
+            w = w.lower().strip()
+            if not w or w.isdigit():
+                continue
+            if w not in seen:
+                seen.add(w)
+                words.append(w)
+    return words
+
+
+def apply_vertical_filter(
+    sql: str,
+    include: Optional[list[str]],
+    exclude: Optional[list[str]],
+) -> str:
+    clauses = []
+
+    if include:
+        inc_clauses = []
+        for v in include:
+            # sanitize any single-quotes by doubling them
+            sanitized = v.lower().replace("'", "''")
+            inc_clauses.append(
+                f"LOWER(primary_vertical) LIKE '%{sanitized}%'"
+            )
+        clauses.append("(" + " OR ".join(inc_clauses) + ")")
+
+    if exclude:
+        exc_clauses = []
+        for v in exclude:
+            sanitized = v.lower().replace("'", "''")
+            exc_clauses.append(
+                f"LOWER(primary_vertical) NOT LIKE '%{sanitized}%'"
+            )
+        clauses.append("(" + " AND ".join(exc_clauses) + ")")
+
+    if not clauses:
+        return sql
+
+    full_clause = "AND " + " AND ".join(clauses)
+
+    # strip any old single-vertical filters
+    sql = re.sub(
+        r"\s+AND\s+LOWER\(primary_vertical\)[^\n]*", "", sql, flags=re.IGNORECASE
+    )
+    sql = re.sub(
+        r"\s+AND\s+r\.primary_vertical\s*=\s*'[^']*'", "", sql, flags=re.IGNORECASE
+    )
+
+    # inject before GROUP BY
+    return re.sub(
+        r"(WHERE[\s\S]*?)(GROUP BY)",
+        lambda m: f"{m.group(1)}    {full_clause}\n{m.group(2)}",
+        sql,
+        count=1,
+        flags=re.IGNORECASE,
+    )
+
+
+
+def highlight_sov(val: float) -> str:
+    """Color SOV change green for positive, red for negative."""
+    if pd.isna(val):
+        return ""
+    color = "green" if val > 0 else "red"
+    return f"color: {color};"
+
+
+def get_sql_template_from_openai(user_text: str) -> str:
+    prompt = f"""
+You are a SQL maestro.
+
+1) From the user’s description:
+   \"\"\"{user_text}\"\"\"
+   identify the top **25** keywords.
+
+2) Generate one complete SQL query that:
+   • Selects domain, article_title, url, pageviews, primary_vertical  
+   • Filters date BETWEEN '{{START_DATE}}' AND '{{END_DATE}}'
+   • Filters only active sites
+   • Only includes pageviews > 9 and pmp_enabled = 'true'  
+   • Excludes unwanted URLs (e.g. '%atlanta%', '%forum%', etc.)  
+   • Uses **at least 20** lines of:
+       `OR parse_url(...):"path" LIKE '%<keyword>%'`  
+     all wrapped in a single `AND ( … )` block  
+   • GROUPs and ORDERs as needed
+
+Return *only* the SQL, with the placeholders literally in the BETWEEN clause, inside a ```sql …``` fence—no extra text.
+"""
+    resp = client.chat.completions.create(
+        model="gpt-4o-mini",
+        messages=[
+            {"role": "system", "content": SYSTEM_PROMPT},
+            {"role": "user", "content": prompt},
+        ],
+    )
+    return extract_sql_block(resp.choices[0].message.content)
+
+
+def run_query(sql: str) -> pd.DataFrame:
+    """Execute SQL on Snowflake and return a lowercase-column DataFrame."""
+    conn = snowflake.connector.connect(**SNOWFLAKE_CONFIG)
+    cur = conn.cursor()
+    cur.execute(sql)
+    rows = cur.fetchall()
+    cols = [c[0].lower() for c in cur.description]
+    conn.close()
+    return pd.DataFrame(rows, columns=cols)
+
+
+# ——————————————
+# 9) USER INPUT & EXECUTION
+# ——————————————
+user_prompt = st.text_area(
+    "Describe the content or keywords for your analysis:",
+    height=150,
+)
+
+if st.button("Generate Table"):
+    if not user_prompt.strip():
+        st.warning("Enter some analysis keywords or description.")
+    else:
+        # Generate SQL once and swap the date range for prior-year query
+        template_sql = get_sql_template_from_openai(user_prompt)
+        sql_current = template_sql.replace(
+            "{START_DATE}", start_date.isoformat()
+        ).replace("{END_DATE}", end_date.isoformat())
+        sql_prior = template_sql.replace(
+            "{START_DATE}", prior_start.isoformat()
+        ).replace("{END_DATE}", prior_end.isoformat())
+
+        include_sel = include_verticals or None
+        exclude_sel = exclude_verticals or None
+        sql_current = apply_vertical_filter(sql_current, include_sel, exclude_sel)
+        sql_prior   = apply_vertical_filter(sql_prior,   include_sel, exclude_sel)
+
+        # Run queries
+        df_current = run_query(sql_current)
+        df_prior = run_query(sql_prior)
+
+        # Extract terms
+        url_kws = extract_keywords(sql_current)
+        if len(url_kws) < 20:
+            st.warning(
+                "Fewer than 20 keywords detected; refine your prompt for broader coverage."
+            )
+        title_kws = extract_title_words(df_current) + extract_title_words(df_prior)
+        all_terms = []
+        seen = set()
+        for term in url_kws + title_kws:
+            term = term.strip()
+            if len(term) <= 3 or term in seen:
+                continue
+            seen.add(term)
+            all_terms.append(term)
+
+        # Totals for pageview display
+        total_cy = df_current["pageviews"].sum()
+        total_py = df_prior["pageviews"].sum()
+
+        # Build metrics without a totals row
+        metrics = []
+        for term in all_terms:
+            cy = df_current[
+                df_current["article_title"].str.contains(term, case=False, na=False)
+                | df_current["url"].str.contains(term, case=False, na=False)
+            ]["pageviews"].sum()
+            py = df_prior[
+                df_prior["article_title"].str.contains(term, case=False, na=False)
+                | df_prior["url"].str.contains(term, case=False, na=False)
+            ]["pageviews"].sum()
+            yoy = (cy - py) / py * 100 if py else float("nan")
+            metrics.append(
+                {
+                    "term": term,
+                    "CY pageviews": cy,
+                    "PY pageviews": py,
+                    "YoY %": yoy,
+                }
+            )
+
+        sum_cy_terms = sum(m["CY pageviews"] for m in metrics)
+        sum_py_terms = sum(m["PY pageviews"] for m in metrics)
+        for m in metrics:
+            m["SOV CY"] = (
+                m["CY pageviews"] / sum_cy_terms if sum_cy_terms else float("nan")
+            )
+            m["SOV PY"] = (
+                m["PY pageviews"] / sum_py_terms if sum_py_terms else float("nan")
+            )
+            m["SOV % Change"] = (
+                (m["SOV CY"] / m["SOV PY"] - 1)
+                if (not pd.isna(m["SOV CY"]) and not pd.isna(m["SOV PY"]))
+                else float("nan")
+            )
+
+        metrics_df = pd.DataFrame(metrics).sort_values("CY pageviews", ascending=False)
+
+        # Display SQL in a hidden expander above metrics
+        with st.expander("Show SQL Queries"):
+            st.subheader("Current Year SQL")
+            st.code(sql_current, language="sql")
+            st.subheader("Prior Year SQL")
+            st.code(sql_prior, language="sql")
+        # Format percentages
+        fmt = {
+            "CY pageviews": "{:,}",  # add thousand separators
+            "PY pageviews": "{:,}",  # add thousand separators
+            "YoY %": "{:.1f}%",
+            "SOV CY": "{:.1%}",
+            "SOV PY": "{:.1%}",
+            "SOV % Change": "{:.1%}",
+        }
+
+        # Display with conditional formatting
+        st.subheader("Term Performance Metrics")
+        styled = metrics_df.style.format(fmt, na_rep="-").applymap(
+            highlight_sov, subset=["SOV % Change"]
+        )
+        st.dataframe(styled, height=400)
+
+        # Show raw result tables with totals
+        with st.expander(f"Current Year Results: {start_date} to {end_date}"):
+            st.dataframe(df_current.style.format({"pageviews": "{:,}"}))
+            st.write(f"Total pageviews: {total_cy:,}")
+        with st.expander(f"Prior Year Results: {prior_start} to {prior_end}"):
+            st.dataframe(df_prior.style.format({"pageviews": "{:,}"}))
+            st.write(f"Total pageviews: {total_py:,}")

changelog.md

@@ -0,0 +1,3 @@
+# Changelog
+
+- 2025-08-07 14:28 UTC: Initialized changelog to track project updates.

index.html

@@ -1,19 +0,0 @@
-<!doctype html>
-<html>
-	<head>
-		<meta charset="utf-8" />
-		<meta name="viewport" content="width=device-width" />
-		<title>My static Space</title>
-		<link rel="stylesheet" href="style.css" />
-	</head>
-	<body>
-		<div class="card">
-			<h1>Welcome to your static Space!</h1>
-			<p>You can modify this app directly by editing <i>index.html</i> in the Files and versions tab.</p>
-			<p>
-				Also don't forget to check the
-				<a href="https://huggingface.co/docs/hub/spaces" target="_blank">Spaces documentation</a>.
-			</p>
-		</div>
-	</body>
-</html>

requirements.txt

@@ -0,0 +1,10 @@
+streamlit
+
+openai>=1.0.0
+=======
+openai
+
+pandas
+python-dotenv
+snowflake-connector-python
+

style.css

@@ -1,28 +0,0 @@
-body {
-	padding: 2rem;
-	font-family: -apple-system, BlinkMacSystemFont, "Arial", sans-serif;
-}
-
-h1 {
-	font-size: 16px;
-	margin-top: 0;
-}
-
-p {
-	color: rgb(107, 114, 128);
-	font-size: 15px;
-	margin-bottom: 10px;
-	margin-top: 5px;
-}
-
-.card {
-	max-width: 620px;
-	margin: 0 auto;
-	padding: 16px;
-	border: 1px solid lightgray;
-	border-radius: 16px;
-}
-
-.card p:last-child {
-	margin-bottom: 0;
-}