DemahAlmutairi commited on
Commit
f2d18be
·
verified ·
1 Parent(s): 7091f34

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +48 -0
README.md CHANGED
@@ -10,3 +10,51 @@ pinned: false
10
  ---
11
 
12
  Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
  ---
11
 
12
  Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
13
+
14
+ Here's a README file that covers the required topics for your Gradio interface project:
15
+
16
+ # README
17
+
18
+ ## Project Objectives
19
+ The primary objective of this project is to create an interactive web application that allows users to input images or sketches to generate captions in English and translate those captions into Arabic. This application leverages advanced machine learning models to provide a seamless experience in image captioning and bilingual translation.
20
+
21
+ ## Description of Implemented Pipelines
22
+ The application utilizes two key pipelines from the Hugging Face Transformers library:
23
+
24
+ 1. **Image Captioning Pipeline**:
25
+ - **Model**: `Salesforce/blip-image-captioning-base`
26
+ - This model takes an image as input and generates a descriptive caption in English. The captioning process involves understanding the content of the image and providing a coherent textual representation.
27
+
28
+ 2. **Translation Pipeline**:
29
+ - **Model**: `facebook/nllb-200-distilled-600M`
30
+ - This model translates the generated English caption into Arabic. It ensures that the translation retains the meaning and context of the original text, making it suitable for Arabic-speaking users.
31
+
32
+ ## Instructions for Using the Interface
33
+ 1. **Accessing the Interface**:
34
+ - Run the provided code in a Python environment with the Gradio library installed.
35
+ - The web interface will launch automatically.
36
+
37
+ 2. **Input Options**:
38
+ - **Upload Tab**: Users can upload an image from their device.
39
+ - **Sketch Tab**: Users can draw a sketch of an object using the sketchpad.
40
+
41
+ 3. **Generating Captions**:
42
+ - After inputting an image or sketch, click the "Submit" button.
43
+ - The application will process the input and display the generated English caption and its Arabic translation in the output fields.
44
+
45
+ 4. **Examples**:
46
+ - The "Example Prompts" section provides sample images. Clicking on these examples will populate the upload interface with the selected image.
47
+
48
+ 5. **Clearing Inputs**:
49
+ - Use the "Clear" button to reset the inputs in both tabs.
50
+
51
+ ## Justifications for Model and Pipeline Choices
52
+ - **Image Captioning**: The chosen model (`Salesforce/blip-image-captioning-base`) is known for its efficiency and accuracy in generating descriptive captions for a wide variety of images. This makes it suitable for diverse user inputs.
53
+
54
+ - **Translation**: The `facebook/nllb-200-distilled-600M` model is selected for its capability to handle multiple languages effectively, ensuring that the translation is contextually relevant and accurate for Arabic users.
55
+
56
+ ## Bilingual Implementation
57
+ The application addresses bilingual implementation by utilizing a dedicated translation pipeline that converts English captions into Arabic. This ensures accessibility for Arabic-speaking users, allowing them to understand the content generated by the image captioning model. The choice of model for translation is based on its ability to provide high-quality translations across different contexts.
58
+
59
+ ## Conclusion
60
+ This project demonstrates the integration of image processing and natural language processing to create a user-friendly application that facilitates image captioning and bilingual translation. By leveraging state-of-the-art models, the application aims to enhance user interaction and accessibility in multilingual contexts.