Update README.md

README.md

@@ -11,3 +11,145 @@ short_description: This project uses AI to transcribe and summarize media conte
 ---
 
 Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+
+# AI-Powered Audio & Video to Text Conversion
+
+## Project Description
+This project enables users to convert audio and video content into text using AI. Additionally, it allows interaction with the extracted text through question-answering, summarization, and text-to-speech functionalities.
+
+## Introduction
+With the increasing amount of digital content, users often face challenges in accessing information from audio and video files efficiently. This project aims to facilitate information retrieval by leveraging AI to transcribe content into text, summarize key points, and provide an interactive Q&A system. Users can also listen to the summarized text if preferred.
+
+## Project Objectives
+1. **Enhance the speed of information retrieval** by enabling quick searches within extracted text.
+2. **Provide an interactive experience** where users can ask questions about the content and receive instant answers.
+3. **Support both Arabic and English languages** to accommodate a wider user base.
+4. **Utilize AI to analyze and comprehend content**, allowing effective extraction and interaction with information.
+
+## Features
+- **Speech-to-Text**: Extracts text from audio and video files.
+- **Text Summarization**: Generates concise summaries of extracted text.
+- **Question Answering**: Answers user queries based on extracted text.
+- **Text-to-Speech**: Converts text into speech for an auditory experience.
+- **Multilingual Support**: Works in both Arabic and English.
+
+## Technology Stack
+### 1. Programming Language
+- **Python**: The core programming language used for this project due to its extensive support for AI and data processing libraries.
+
+### 2. Libraries & Frameworks
+- **Gradio**: Used for creating an interactive web-based UI for users to upload files and interact with the extracted text.
+- **Hugging Face Transformers**: Provides pre-trained models for automatic speech recognition, summarization, translation, and question-answering.
+- **MoviePy**: Extracts audio from video files for further processing.
+- **Librosa & SoundFile**: Handles audio processing, including loading, resampling, and segmenting audio clips.
+- **gTTS (Google Text-to-Speech)**: Converts text into spoken words.
+- **LangDetect**: Detects the language of the extracted text to provide appropriate processing.
+
+## Code Breakdown
+### 1. **Setting Up the Environment**
+The script starts by checking if a GPU (CUDA) is available, which significantly speeds up AI model inference.
+```python
+import torch
+device = "cuda" if torch.cuda.is_available() else "cpu"
+```
+
+### 2. **Loading AI Models**
+Several AI models are loaded to handle different functionalities:
+- **Whisper (Speech Recognition)**: Converts audio into text.
+- **BART (Summarization)**: Generates concise summaries.
+- **Helsinki-NLP (Translation)**: Translates text between Arabic and English.
+- **BERT (Question Answering)**: Finds answers from extracted text.
+```python
+from transformers import pipeline
+pipe = pipeline("automatic-speech-recognition", model="openai/whisper-medium", device=0 if device == "cuda" else -1)
+bart_model = AutoModelForSeq2SeqLM.from_pretrained("ahmedabdo/arabic-summarizer-bart")
+bart_tokenizer = AutoTokenizer.from_pretrained("ahmedabdo/arabic-summarizer-bart")
+translate_ar_to_en = pipeline("translation", model="Helsinki-NLP/opus-mt-ar-en")
+translate_en_to_ar = pipeline("translation", model="Helsinki-NLP/opus-mt-en-ar")
+qa_pipeline = pipeline("question-answering", model="deepset/bert-base-cased-squad2", tokenizer="deepset/bert-base-cased-squad2")
+```
+
+### 3. **Audio Processing**
+Handles both audio and video files. If a video is uploaded, it extracts the audio and converts it into a compatible format.
+```python
+from moviepy.video.io.VideoFileClip import VideoFileClip
+import librosa
+import soundfile as sf
+
+def convert_audio_to_text(uploaded_file):
+    if not uploaded_file:
+        return "⛔ Please upload a file first"
+
+    input_path = uploaded_file if isinstance(uploaded_file, str) else uploaded_file.name
+    output_path = "/tmp/processed.wav"
+
+    if input_path.split('.')[-1].lower() in ['mp4', 'avi', 'mov', 'mkv']:
+        VideoFileClip(input_path).audio.write_audiofile(output_path, codec='pcm_s16le')
+    else:
+        output_path = input_path
+
+    audio, rate = librosa.load(output_path, sr=16000)
+    return pipe(output_path)["text"]
+```
+
+### 4. **Summarization**
+Summarizes extracted text using the pre-trained BART model.
+```python
+def summarize_text(text):
+    inputs = bart_tokenizer(text, return_tensors="pt", max_length=1024, truncation=True).to(device)
+    summary_ids = bart_model.generate(inputs.input_ids, max_length=150, num_beams=4, early_stopping=True)
+    return bart_tokenizer.decode(summary_ids[0], skip_special_tokens=True)
+```
+
+### 5. **Question Answering**
+Users can input a question based on the extracted text, and the system finds the most relevant answer.
+```python
+def answer_question(text, question):
+    translated_context = translate_ar_to_en(text)[0]['translation_text']
+    translated_question = translate_ar_to_en(question)[0]['translation_text']
+    results = qa_pipeline({'question': translated_question, 'context': translated_context}, top_k=3)
+    best_result = max(results, key=lambda res: res['score'])
+    return translate_en_to_ar(best_result['answer'])[0]['translation_text']
+```
+
+### 6. **Text-to-Speech**
+Converts text into an audio file using gTTS.
+```python
+def text_to_speech(text):
+    tts = gTTS(text=text, lang='ar' if detect(text) == 'ar' else 'en', slow=False)
+    output = "/tmp/tts.wav"
+    tts.save(output)
+    return output
+```
+
+## Installation & Setup
+### Prerequisites
+- Python 3.8+
+- Pip package manager
+- GPU (optional but recommended for better performance)
+
+### Installation Steps
+1. Clone the repository:
+   ```sh
+   git clone https://github.com/your-repo/ai-audio-video-text.git
+   cd ai-audio-video-text
+   ```
+2. Install dependencies:
+   ```sh
+   pip install -r requirements.txt
+   ```
+3. Run the application:
+   ```sh
+   python main.py
+   ```
+
+## Contributing
+Contributions are welcome! Feel free to fork the repository and submit pull requests.
+
+## License
+This project is licensed under the MIT License.
+
+## Contact
+For inquiries, contact [ٍShathah2030@gmail.com] 
+
+