Upload 3 files

README.md

@@ -1,20 +1,87 @@
----
-title: LangChain VideoWeb Summarizer
-emoji: 🚀
-colorFrom: red
-colorTo: red
-sdk: docker
-app_port: 8501
-tags:
-- streamlit
-pinned: false
-short_description: Streamlit template space
-license: mit
----
-
-# Welcome to Streamlit!
-
-Edit `/src/streamlit_app.py` to customize this app to your heart's desire. :heart:
-
-If you have any questions, checkout our [documentation](https://docs.streamlit.io) and [community
-forums](https://discuss.streamlit.io).
+# AI Content Summarizer
+
+This Streamlit application leverages AI to summarize web articles and YouTube videos. Users can input a URL, and the app will provide a concise summary of the content.
+It summarizes YouTube videos or any website content using **LangChain**, **Google's Gemini API**, and **LangChain's document loaders**.
+It uses `LangChain` and `ChatGoogleGenerativeAI` with `gemini-1.5-flash` model.
+
+## Features
+
+- **Web Article Summarization**: Enter the URL of any online article to get a summary.
+- **YouTube Video Summarization**: Provide a YouTube video URL to receive a transcript summary.
+- **Secure API Key Handling**: Uses Streamlit's sidebar for API key input, ensuring it's not exposed in the main interface.
+- **User-Friendly Interface**: Simple and intuitive design for ease of use.
+
+## Technologies Used
+
+- **Streamlit**: For creating the web application interface.
+- **Langchain**: To handle the summarization chain and document processing.
+- **Langchain**: To handle the summarization chain and document processing.
+- **[LangChain Google GenAI](https://python.langchain.com/docs/integrations/llms/google_generative_ai)** – for LLM integration.
+- **Validators**: To ensure the validity of input URLs.
+- **youtube_transcript_api**: For fetching transcripts from YouTube videos.
+- **pytube**: To extract video information from YouTube.
+- **Unstructured**: For parsing and extracting content from HTML (web articles).
+
+## Setup and Installation
+
+1.  **Clone the repository:**
+    ```bash
+    git clone https://github.com/your-username/ai-content-summarizer.git
+    cd ai-content-summarizer
+    ```
+
+2.  **Create a virtual environment and activate it:**
+    ```bash
+    python -m venv venv
+    source venv/bin/activate  # On Windows use `venv\Scripts\activate`
+    ```
+
+3.  **Install the required dependencies:**
+    ```bash
+    pip install -r requirements.txt
+    ```
+
+4.  **Set your Google API Key:**
+    - Obtain an API key from [Google AI Studio](https://aistudio.google.com/app/apikey).
+    - You can set it as an environment variable:
+      ```bash
+      export GOOGLE_API_KEY='your_google_api_key_here'
+      ```
+    - Alternatively, you can enter it directly in the application's sidebar when prompted.
+
+## How to Run
+
+1.  **Ensure your virtual environment is activated and dependencies are installed.**
+2.  **Run the Streamlit application:**
+    ```bash
+    streamlit run app.py
+    ```
+3.  Open your web browser and navigate to the local URL provided by Streamlit (usually `http://localhost:8501`).
+4.  Enter your **Google API Key** in the sidebar.
+5.  Paste the URL of a web article or YouTube video into the input field and click "Summarize".
+
+## Example Usage
+
+1.  **Find an interesting article online or a YouTube video.**
+2.  **Copy its URL.**
+3.  **Paste the URL into the app's input field.**
+4.  **Click "Summarize" and wait for the AI to generate the summary.**
+
+## Contributing
+
+Contributions are welcome! If you have suggestions for improvements or new features, please feel free to:
+
+1.  Fork the repository.
+2.  Create a new branch (`git checkout -b feature/your-feature-name`).
+3.  Make your changes.
+4.  Commit your changes (`git commit -m 'Add some feature'`).
+5.  Push to the branch (`git push origin feature/your-feature-name`).
+6.  Open a Pull Request.
+
+## License
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details (if you add one).
+
+## Notes
+- You must have a **valid Google API Key** to use this app.
+- The quality of the summary depends on the clarity and structure of the source content.

app.py

@@ -0,0 +1,165 @@
+import streamlit as st
+from langchain_google_genai import ChatGoogleGenerativeAI
+from langchain.chains.summarize import load_summarize_chain
+from langchain.docstore.document import Document
+from langchain_core.prompts import PromptTemplate
+import validators
+# from youtube_transcript_api import YouTubeTranscriptApi # Replaced by YoutubeLoader
+# from pytube import YouTube # Replaced by YoutubeLoader
+# from unstructured.partition.html import partition_html # Replaced by UnstructuredURLLoader
+# import requests # Replaced by UnstructuredURLLoader
+from langchain_community.document_loaders import YoutubeLoader, UnstructuredURLLoader
+from youtube_transcript_api import TranscriptsDisabled, NoTranscriptFound
+
+
+# Set page config
+st.set_page_config(page_title="AI Content Summarizer", page_icon="🚀", layout="wide")
+
+# Custom CSS for styling
+st.markdown("""
+<style>
+    .main-header {
+        font-size: 36px !important;
+        color: #4CAF50;
+        text-align: center;
+        margin-bottom: 30px;
+    }
+    .sub-header {
+        font-size: 24px !important;
+        color: #FF6347;
+        margin-top: 20px;
+        margin-bottom: 10px;
+    }
+    .text-input {
+        width: 100%;
+        padding: 10px;
+        border-radius: 5px;
+        border: 1px solid #ddd;
+        margin-bottom: 20px;
+    }
+    .submit-button {
+        background-color: #4CAF50;
+        color: white;
+        padding: 10px 20px;
+        border: none;
+        border-radius: 5px;
+        cursor: pointer;
+        font-size: 16px;
+    }
+    .submit-button:hover {
+        background-color: #45a049;
+    }
+    .summary-output {
+        background-color: #f9f9f9;
+        padding: 20px;
+        border-radius: 5px;
+        border: 1px solid #eee;
+        margin-top: 20px;
+        color: #333333; /* Added for text visibility */
+    }
+    .error-message {
+        color: red;
+        font-weight: bold;
+    }
+</style>
+""", unsafe_allow_html=True)
+
+# API Key Input
+st.sidebar.title("API Key Configuration")
+google_api_key = st.sidebar.text_input("🔑 Google API Key", type="password")
+
+# --- Helper Functions ---
+def get_llm(api_key: str):
+    """Initializes and returns the ChatGoogleGenerativeAI instance."""
+    return ChatGoogleGenerativeAI(model="gemini-1.5-flash", google_api_key=api_key, temperature=0)
+
+def load_documents(url: str) -> list[Document]:
+    """Loads documents from a URL (YouTube or web article)."""
+    docs = []
+    try:
+        if "youtube.com" in url or "youtu.be" in url:
+            st.info("Processing YouTube URL...")
+            try:
+                loader = YoutubeLoader.from_youtube_url(
+                    url,
+                    add_video_info=False, # Keep this as False
+                    language=['en']
+                )
+                with st.spinner("Fetching and parsing YouTube content..."):
+                    docs = loader.load()
+            except TranscriptsDisabled:
+                st.error(f"Transcripts are disabled for the YouTube video: {url}")
+                return []
+            except NoTranscriptFound:
+                st.error(f"No English transcripts found for the YouTube video: {url}. The video might not have transcripts or not in English.")
+                return []
+            except Exception as e:
+                st.error(f"Error loading YouTube content for {url}: {str(e)}. This could be due to parsing issues or video unavailability.")
+                return []
+        else:
+            st.info("Processing web article URL...")
+            headers = {
+                "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"
+            }
+            loader = UnstructuredURLLoader(urls=[url], headers=headers)
+            with st.spinner("Fetching and parsing web content..."):
+                docs = loader.load()
+
+        if not docs:
+            st.warning("No content could be extracted from the URL. For YouTube, check if transcripts are available and in English. For websites, the page might be empty or structured in a way that's hard to parse.")
+            return []
+        return docs
+
+    except Exception as e: # This is the outermost catch-all
+        st.error(f"An unexpected error occurred during document loading: {str(e)}")
+        return []
+
+
+prompt_template_str = """
+Provide simple understandable summary in around 300 words for the following content:
+Content: {text}
+"""
+prompt = PromptTemplate(template=prompt_template_str, input_variables=["text"])
+
+def generate_summary(llm, docs: list[Document]):
+    """Generates a summary using the LLM and loaded documents."""
+    if not docs:
+        return "No content to summarize."
+    chain = load_summarize_chain(llm, chain_type="stuff", prompt=prompt)
+    with st.spinner("AI is summarizing the content..."):
+        summary = chain.invoke({"input_documents": docs})
+        return summary["output_text"]
+
+# --- Main Application ---
+st.markdown("<div class='main-header'>AI Content Summarizer 🚀</div>", unsafe_allow_html=True)
+st.write("This app summarizes web articles and YouTube videos using AI. Enter a URL below to get started.")
+
+# Input URL
+url_input = st.text_input("Enter URL (Article or YouTube):", key="url_input_main", help="Paste the URL of the article or YouTube video you want to summarize.")
+
+# Submit button
+if st.button("Summarize Content", key="submit_button_main"):
+    if not google_api_key:
+        st.error("🚫 Please enter your Google API Key in the sidebar.")
+    elif not url_input:
+        st.warning("⚠️ Please enter a URL.")
+    elif not validators.url(url_input):
+        st.error("🚫 Invalid URL. Please enter a valid URL.")
+    else:
+        try:
+            st.markdown("<div class='sub-header'>Processing...</div>", unsafe_allow_html=True)
+
+            llm = get_llm(api_key=google_api_key)
+
+            docs = load_documents(url=url_input)
+
+            if docs:
+                summary_result = generate_summary(llm=llm, docs=docs)
+                st.markdown("<div class='sub-header'>Summary:</div>", unsafe_allow_html=True)
+                st.success("Summary generated successfully!")
+                st.markdown(f"<div class='summary-output'>{summary_result}</div>", unsafe_allow_html=True)
+            # Error handling for empty docs is done within load_documents
+
+        except Exception as e:
+            st.error(f"An unexpected error occurred: {e}")
+            st.markdown(f"<div class='error-message'>Details: {str(e)}</div>", unsafe_allow_html=True)

requirements.txt

@@ -1,3 +1,9 @@
-altair
-pandas
-streamlit
+streamlit
+langchain
+langchain-google-genai
+validators
+youtube_transcript_api>=0.6.2
+pytube>=15.0.0
+unstructured
+pytest
+pytest-mock