Prince9191 commited on
Commit
8af5a8b
·
verified ·
1 Parent(s): 1313208

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +63 -69
README.md CHANGED
@@ -1,104 +1,98 @@
1
- # A Object Identification and text to speech model using HuggingFace Transformers
2
 
3
- Learn how to build the below pipeline using Gradio (for user interface and deployment), Facebook' detr-resnet-50 model for Object Identification and kakao-enterprise' vits-ljs model for text to speech.
4
 
5
- <img src="pipeline.PNG" alt="project pipeline" width="400">
6
 
7
 
8
- ## Description
9
 
10
- The Image Description and Audio Transcript App is a web-based application that leverages artificial intelligence to generate descriptions for uploaded images. Additionally, the tool provides an audio transcript of the description for users with visual impairments, making it more accessible.
11
 
12
- This app uses BLIP (Bootstrapping Language-Image Pre-training) for image captioning and gTTS (Google Text-to-Speech) for converting the description into an audio file.
13
 
14
- ## Features
 
 
 
 
15
 
16
- * Upload an image and receive an AI-generated description.
17
- * Convert the description into an audio file for accessibility.
18
- * Responsive web interface built using Gradio.
19
- * Simple, user-friendly design for a seamless experience.
20
 
21
- ## Technologies Used
 
 
 
 
22
 
23
- * Programming Language: Python 3.7+
24
- * AI Model: BLIP for image captioning
25
- * Text-to-Speech: gTTS (Google Text-to-Speech)
26
- * Web Interface: Gradio
27
- * Libraries: PyTorch, Transformers, Gradio, gTTS
28
 
29
- ## Libraries and Dependencies
 
 
 
 
 
 
 
 
 
30
 
31
- * torch: Deep learning framework for the BLIP model
32
- * transformers: Hugging Face library for pre-trained models like BLIP
33
- * gtts: Library for text-to-speech conversion
34
- * gradio: For building the web interface
35
 
36
- ## To install the necessary packages, run:
37
 
 
38
  ```bash
39
- pip install torch transformers gtts gradio
40
- ```
41
 
42
- ## Installation and Setup
43
 
44
- * Clone the repository:
45
- ```bash
46
- git clone https://github.com/your-username/image-description-audio-transcript.git
47
- cd image-description-audio-transcript
48
- ```
49
-
50
- * Create a virtual environment (optional but recommended):
51
- ```bash
52
  python -m venv venv
53
- source venv/bin/activate # On Windows, use venv\Scripts\activate
54
- ```
55
 
56
- * Install the required packages:
57
- ```bash
58
- pip install torch transformers gtts gradio
59
- ```
60
 
61
- * Ensure that the necessary models are downloaded: The BLIP model will automatically be downloaded when the script is run, and gTTS will use an online service to convert text to speech.
62
 
63
- ## Usage
 
 
 
 
 
 
64
 
65
- 1. Run the application:
66
- ```bash
67
  python object_detection.py
68
- ```
69
- 2. Open a web browser and navigate to http://127.0.0.1:7860 to access the app.
70
- 3. Upload an image through the provided input.
71
- 4. Click the `Generate Description` button to get a text description of the image.
72
- 5. Click the `Click here for an audio transcript` button to hear the description.
73
 
74
- ## Configuration
 
 
 
 
 
 
 
75
 
76
- You can modify the following parameters in the app.py file:
77
 
78
- * host: The IP address on which the server runs (default: '127.0.0.1')
79
- * port: The port number (default: 7860)
80
- * debug: Debug mode for development (default: True)
81
 
82
- ## Contributing
83
 
84
- Contributions to improve the Image Description and Audio Transcript App are welcome. Please follow these steps:
85
 
86
- * Fork the repository.
87
- * Create a new branch (git checkout -b feature/AmazingFeature).
88
- * Commit your changes (git commit -m 'Add some AmazingFeature').
89
- * Push to the branch (git push origin feature/AmazingFeature).
90
- * Open a Pull Request.
91
 
92
- ## License
 
 
 
93
 
94
- This project is licensed under the MIT License - see the LICENSE.md file for details.
95
 
96
- ## Acknowledgments
97
 
98
- * Salesforce for the BLIP image captioning model.
99
- * Google for the gTTS service.
100
- * Gradio for the easy-to-use interface framework.
101
 
102
- ## Disclaimer
103
 
104
- This tool is designed to assist with generating descriptions and audio transcripts from images, but always review the generated content for accuracy and appropriateness before use.
 
1
+ Absolutely! Here’s the full rephrased content you can easily copy and paste:
2
 
3
+ ⸻
4
 
 
5
 
6
 
7
+ # Image Description and Audio Transcript App
8
 
9
+ An AI-powered web app that identifies objects in images and converts the generated descriptions into speech using Hugging Face Transformers.
10
 
11
+ ---
12
 
13
+ ## Overview
14
+ This project showcases how to build a pipeline using:
15
+ - **BLIP** for image captioning
16
+ - **gTTS** (Google Text-to-Speech) for audio generation
17
+ - **Gradio** for the user interface and deployment on Hugging Face Spaces
18
 
19
+ ---
 
 
 
20
 
21
+ ## What It Does
22
+ - Upload an image → Get an AI-generated description
23
+ - Automatically convert the description into audio
24
+ - Built with accessibility in mind for users with visual impairments
25
+ - Runs on a clean, responsive web UI using **Gradio**
26
 
27
+ ---
 
 
 
 
28
 
29
+ ## Tech Stack
30
+ - **Language**: Python 3.7+
31
+ - **AI Models**:
32
+ - `Salesforce/blip-image-captioning-base` – for generating image captions
33
+ - `gtts` – for converting text into speech
34
+ - **Frameworks/Libraries**:
35
+ - `torch` – powering the models
36
+ - `transformers` – loading and running pre-trained models
37
+ - `gradio` – creating the interactive frontend
38
+ - `Pillow`, `matplotlib`, `inflect` – for image handling and fine-tuning the output
39
 
40
+ ---
 
 
 
41
 
42
+ ## Installation
43
 
44
+ 1. **Clone the repo** (or upload files to your Hugging Face Space):
45
  ```bash
46
+ git clone https://github.com/your-username/image-caption-audio-app.git
47
+ cd image-caption-audio-app
48
 
49
+ 2. (Optional) Create a virtual environment:
50
 
 
 
 
 
 
 
 
 
51
  python -m venv venv
52
+ source venv/bin/activate # For Windows: venv\Scripts\activate
 
53
 
54
+ 3. Install dependencies:
 
 
 
55
 
56
+ pip install torch transformers gtts gradio Pillow matplotlib inflect
57
 
58
+ If you’re using Hugging Face Spaces, simply include a requirements.txt file with those packages.
59
+
60
+ ⸻
61
+
62
+ How to Run
63
+
64
+ Locally:
65
 
 
 
66
  python object_detection.py
 
 
 
 
 
67
 
68
+ Then visit: http://127.0.0.1:7860 in your browser.
69
+
70
+ On Hugging Face:
71
+ Just upload all files (including requirements.txt) to your Space. It’ll launch automatically.
72
+
73
+ ⸻
74
+
75
+ Customizations
76
 
77
+ You can tweak parameters (like host, port, or debug settings) directly in the script if needed. For example:
78
 
79
+ gr.Interface(...).launch(server_name="0.0.0.0", server_port=7860, debug=True)
 
 
80
 
 
81
 
 
82
 
83
+ ⸻
 
 
 
 
84
 
85
+ Credits
86
+ • Hugging Face for the BLIP model
87
+ • Google for gTTS
88
+ • Gradio for simplifying deployment and UI creation
89
 
90
+ ⸻
91
 
92
+ License
93
 
94
+ MIT License – Feel free to use, share, and modify.
 
 
95
 
96
+ ---
97
 
98
+ Let me know if you'd like a version with your name, GitHub link, or any branding!