src/streamlit_app.py

import streamlit as st
import pandas as pd
import altair as alt
import os

# Page Configuration
st.set_page_config(layout="wide", page_title="IMDb Cinematic Trends")

# Enable Altair Dark Theme for visibility on black backgrounds
alt.themes.enable('dark')

# Data Loading 
@st.cache_data
def load_all_data():
    main_path = 'src/merged_imdb_sample.csv' if os.path.exists('src/merged_imdb_sample.csv') else 'merged_imdb_sample.csv'
    ctx_path = 'src/principals_context_sample.csv' if os.path.exists('src/principals_context_sample.csv') else 'principals_context_sample.csv'
    
    # 1. Load Primary Data
    df = pd.read_csv(main_path)
    df = df[df['startYear'] > 1900].copy()
    df['decade'] = (df['startYear'] // 10 * 10).astype(int).astype(str) + 's'
    df['genre_list'] = df['genres'].str.split(',')
    df_exploded = df.explode('genre_list')
    df_exploded['genre_list'] = df_exploded['genre_list'].str.strip()
    
    # 2. Load Contextual Data
    df_ctx = pd.read_csv(ctx_path)
    return df_exploded, df_ctx

try:
    df, df_ctx = load_all_data()
except FileNotFoundError:
    st.error("⚠️ CSV files not found. Please ensure the datasets are in the 'src' folder or root directory.")
    st.stop()

# Header & Navigation
st.title("The Architects of Entertainment: Mapping Quality and Collaboration")
st.write("**Authors:** Group 6")
st.write("Le Kim Ngan Hoang - lkhoang2")
st.write("Karen Xiong - karenhx2")
st.write("Liting Chen - lchen235")
st.write("Kelsey Li - yitingl7")

col_nav1, col_nav2, col_nav3 = st.columns(3)
with col_nav1:
    st.link_button("📂 Primary Dataset", "https://github.com/Lekimnganhoang3010/IS_445_Final-Project/blob/main/merged_imdb_sample.csv")
with col_nav2:
    st.link_button("📂 Contextual Dataset", "https://github.com/Lekimnganhoang3010/IS_445_Final-Project/blob/main/principals_context_sample.csv")


st.divider()

# Narrative Paragraph 1
st.markdown("""
### The Evolution of Cinematic Quality
Cinema is often discussed through the lens of individual success—a high rating or a famous director. 
As seen in our primary dashboard, certain decades show marked shifts in average ratings for different genres. 
By exploring the trends below, we can identify 'Golden Eras' and the creators who defined them. 
Each square in the heatmap's grid corresponds to a specific genre during a specific decade, with its color correlating
to the average ratings of movies under that genre during that decade (the color scale is denoted to the right of the 
heatmap). Uncolored squares indicate a lack of information from our original dataset (i.e., no movies were found to 
be under that genre during that decade). Upon clicking a given colored cell, the graph displaying the top 10 directors 
(in terms of total votes received) for that specific genre-decade combination is appropriately updated. Clicking the 
aforementioned dark cells results in a chart displaying the overall top 10 directors across all of the studied movies.
This can be useful in providing recommendations to users, both for those who are interested in specific genres and/or 
decades of movies and those who are generally looking for highly rated types of movies. Furthermore, information about 
the directors is important in terms of informing users about which director's movies are most well-rated if they have a 
favorite genre and/or decade of movies. This dashboard is extremely informative, providing an overview of the most well-rated
genres and decades, along with the top directors for them, and how those ratings have changed over time.
""")

# Main Interactive Dashboard with Altair
st.header("1. Interactive Trend Explorer")
st.info("💡 **Interaction:** Click a cell in the Heatmap to filter the Top Directors chart below.")

selection = alt.selection_point(
    fields=['genre_list', 'decade'], 
    name='cell',
    toggle=False
)

# 2. Heatmap
heatmap = alt.Chart(df).mark_rect().encode(
    x=alt.X('decade:O', title='Decade', axis=alt.Axis(labelLimit=200)),
    y=alt.Y('genre_list:N', title='Genre'),
    color=alt.Color('mean(averageRating):Q', 
                    scale=alt.Scale(scheme='redyellowgreen'), 
                    title='Avg Rating'),
    stroke=alt.condition(selection, alt.value('white'), alt.value('transparent')),
    strokeWidth=alt.condition(selection, alt.value(2), alt.value(0)),
    tooltip=['genre_list', 'decade', 'mean(averageRating)']
).properties(
    width=800,
    height=400,
    title="Average Movie Ratings by Genre and Decade"
).add_params(selection)


# 3. Bar Chart
bar_chart = alt.Chart(df).mark_bar().encode(
    x=alt.X('total_votes:Q', title='Total Popularity (Votes)'),
    y=alt.Y('directorName:N', sort='-x', title='Director', axis=alt.Axis(labelLimit=200)),
    color=alt.value('#4682B4'),
    tooltip=['directorName', alt.Tooltip('total_votes:Q', format=',')] 
).transform_filter( 
    selection # Ensure 'selection' is defined above this block!
).transform_aggregate(
    total_votes='sum(numVotes)',
    groupby=['directorName']
).transform_window(
    rank='rank()',
    sort=[alt.SortField('total_votes', order='descending')]
).transform_filter(
    alt.datum.rank <= 10
).properties(
    width=800,
    height=300,
    title="Top 10 Directors (Select a cell to view)"
)
# Render charts with standard dark-mode visibility
st.altair_chart(heatmap & bar_chart, use_container_width=True)

# Narrative Paragraph 2
st.markdown("""
### Behind the Screen: Professional Infrastructure
While our dashboard focuses on ratings, the first contextual view below, **Professional Roles**, reveals 
the vast workforce required to achieve these scores. For every director highlighted in our dashboard, 
there is an infrastructure of writers, composers, and cinematographers. This visualization shifts the 
perspective from a single 'Auteur' to a realistic view of cinema as a multifaceted professional ecosystem. 
This contextual visualization examines how different movie credit roles are positioned in the IMDb 
principal dataset. In this dataset, each row represents a person connected to a movie, such as an actor, 
actress, director, writer, or producer. The ordering column shows the order in which that person appears in 
the movie’s principal credits. A smaller number means that the person appears earlier in the credits, so 
this chart focuses on the first few credit positions to understand which roles are usually given the most
visibility. This visualization is useful because it gives background context for the main movie dataset. 
The main dataset focuses on movie-level information, such as title, genre, runtime, year, average rating, 
and number of votes. However, movies are not only defined by their ratings or genres - they are also shaped 
by the people involved in creating them. By looking at the credit order, readers can better understand 
which types of contributors are most visible in IMDb’s movie records. For example, if actors and actresses
appear most often in the earliest credit positions, this suggests that on-screen performers are usually 
emphasized more strongly than "behind the scenes" roles. While the main dataset focuses on movie ratings, 
genres, and runtimes, this contextual visualization adds background about the people behind those movies. 
It suggests that movie records are not only about the films themselves, but also about how different 
contributors, such as actors, actresses, directors, and writers, are presented to the public.
""")


# Contextual Charts
st.header("2. Production Context")
ctx_col1, ctx_col2 = st.columns(2)

with ctx_col1:
    role_dist = alt.Chart(df_ctx).mark_bar().encode(
        x=alt.X('count():Q', title='Number of Credits'),
        y=alt.Y('category:N', sort='-x', title='Role Category', axis=alt.Axis(labelLimit=200)),
        color=alt.value('#57A44C'),
        tooltip=['category', 'count()']
    ).properties(
        title="Distribution of Professional Roles",
        height=400
    )
    st.altair_chart(role_dist, use_container_width=True)
    st.caption("Source: IMDb Principals Sample. Highlights technical vs. creative workforce balance.")

with ctx_col2:
    crew_counts = df_ctx.groupby('tconst').size().reset_index(name='crew_count')
    collab_density = alt.Chart(crew_counts).mark_bar().encode(
        x=alt.X('crew_count:Q', bin=alt.Bin(maxbins=10), title='Key Personnel per Film'),
        y=alt.Y('count():Q', title='Frequency'),
        color=alt.value('#F2CF5B')
    ).properties(
        title="Collaboration Density (Team Size)",
        height=400
    )
    st.altair_chart(collab_density, use_container_width=True)
    st.caption("Source: IMDb Principals Sample. Most successful films require 10-25 key leaders.")



# Narrative Paragraph
st.markdown("""
### The Final Story: Complexity and Quality
Our final contextual chart, **Collaboration Density**, highlights the scale of these productions. 
This chart portrays the frequency of the different numbers of key staff members working on each 
film that was observed from our dataset, with higher bars corresponding to greater frequency (i.e., 
more movies with that particular number of personnel). Most high-quality films in our sample rely on 
10 to 25 principal personnel in leadership roles, with the highest peak at 15 to 20 personnel. 
Additionally, we do not observe more than 35 personnel working on any of the films from our original 
dataset, indicating that this is a general threshold in terms of trends in film staff size.
These observations suggest a strong correlation between cinematic quality and team density, providing further
context for our primary dashboard showcasing the movies' average ratings.
""")

st.divider()

st.markdown("""
### Citation
#### Original Dataset

The name of the dataset is “merged_imdb_sample”. This aggregated dataset is created using IMDb Non-Commercial Datasets: https://datasets.imdbws.com/

Given that the original datasets are computationally expensive and exceed typical project and GitHub storage limits, a sampling and preprocessing strategy was applied: 
- A subset of approximately 1021 movie records was randomly sampled from the primary table (**title.basics**) after filtering for valid (non-null) entries.
- Related tables (**title.ratings, title.crew, and name.basics**) were then filtered using matching keys (tconst, nconst) to ensure relational consistency.
- Only relevant columns (e.g., title, year, genre, rating, votes, director name) were retained to reduce dimensionality.
- Missing values were selectively handled to preserve as much usable data as possible without significantly reducing the dataset size.



#### Contextual dataset
To add further depth to the story of how certain genres and directors rose to prominence, I have identified the following contextual dataset:

- **Dataset Name**: IMDb Title Principals 
- **Link**: https://datasets.imdbws.com/
- **Contextual Utility**: 

While the current dashboard focuses on directors, the title.principals dataset provides data on the entire "creative team," including lead actors, writers, and cinematographers. 
This would allow us to investigate if the high ratings in specific genres (like Sci-Fi or Noir) were driven by specific recurring collaborations or technical shifts in cinematography.""")