File size: 31,014 Bytes
4666b21
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
"""
World Language Vision Assistant
================================
Final Project Single-File Python Application
Theresa Gomes
ARIN 460

Project Purpose
---------------
This app demonstrates a multimodal AI system using:
1. Computer vision / vision-language modeling for image captioning.
2. Natural language processing for generated explanations.
3. Multilingual language metadata from the Hugging Face dataset:
   lukeslp/world-languages
4. Machine translation for selected supported languages.
5. A Gradio web interface ready for Hugging Face Spaces deployment.

Recommended Hugging Face Space Setup
------------------------------------
Create a Gradio Space and upload this file as app.py.
Also create requirements.txt with:

gradio
transformers
torch
torchvision
datasets
Pillow
sentencepiece
sacremoses
accelerate

The app is designed to be practical for classroom demonstration. It uses the
lightweight Salesforce BLIP image captioning model by default because it is
more realistic to deploy on free or basic Hugging Face Spaces than larger
vision-language models.

Model Used
----------
Vision-language model:
- Salesforce/blip-image-captioning-base

Dataset Used
------------
- lukeslp/world-languages

Translation models:
- Helsinki-NLP/opus-mt-en-es
- Helsinki-NLP/opus-mt-en-fr
- Helsinki-NLP/opus-mt-en-de
- Helsinki-NLP/opus-mt-en-it
- Helsinki-NLP/opus-mt-en-pt
- Helsinki-NLP/opus-mt-en-nl

Notes
-----
This application does not claim perfect translation quality. It is a classroom
project showing how multiple AI capabilities can be integrated into one system.
For languages without an available local translation model in this app, the
system still uses the world language dataset metadata and returns the English
caption with a clear note explaining that translation was not available.
"""

# ============================================================================
# Standard library imports
# ============================================================================

import os
import time
import traceback
from dataclasses import dataclass
from functools import lru_cache
from typing import Any, Dict, List, Optional, Tuple

# ============================================================================
# Third-party imports
# ============================================================================

import gradio as gr
import torch
from PIL import Image
from datasets import load_dataset
from transformers import (
    AutoModelForSeq2SeqLM,
    AutoTokenizer,
    BlipForConditionalGeneration,
    BlipProcessor,
)


# ============================================================================
# Configuration
# ============================================================================

APP_TITLE = "World Language Vision Assistant"
APP_SUBTITLE = (
    "Upload an image, generate an AI caption, and view the result with "
    "world language metadata and supported translations."
)

VISION_MODEL_NAME = "Salesforce/blip-image-captioning-base"
WORLD_LANGUAGES_DATASET = "lukeslp/world-languages"

# Hugging Face Spaces often provides CPU unless GPU is selected. This app checks
# for CUDA automatically and falls back to CPU.
DEVICE = "cuda" if torch.cuda.is_available() else "cpu"
TORCH_DTYPE = torch.float16 if DEVICE == "cuda" else torch.float32

# Translation models included for a manageable classroom deployment. Additional
# languages can be added later if the Space has enough memory.
TRANSLATION_MODEL_MAP: Dict[str, Dict[str, str]] = {
    "Spanish": {
        "iso_639_1": "es",
        "model": "Helsinki-NLP/opus-mt-en-es",
    },
    "French": {
        "iso_639_1": "fr",
        "model": "Helsinki-NLP/opus-mt-en-fr",
    },
    "German": {
        "iso_639_1": "de",
        "model": "Helsinki-NLP/opus-mt-en-de",
    },
    "Italian": {
        "iso_639_1": "it",
        "model": "Helsinki-NLP/opus-mt-en-it",
    },
    "Portuguese": {
        "iso_639_1": "pt",
        "model": "Helsinki-NLP/opus-mt-en-pt",
    },
    "Dutch": {
        "iso_639_1": "nl",
        "model": "Helsinki-NLP/opus-mt-en-nl",
    },
}

DEFAULT_LANGUAGE = "Spanish"
MAX_NEW_TOKENS_CAPTION = 60
MAX_NEW_TOKENS_TRANSLATION = 140


# ============================================================================
# Data classes
# ============================================================================

@dataclass
class LanguageRecord:
    """A safe internal representation of one language metadata record."""

    display_name: str
    iso_639_3: str = "Unknown"
    family: str = "Unknown"
    region: str = "Unknown"
    country: str = "Unknown"
    latitude: Optional[float] = None
    longitude: Optional[float] = None
    population: str = "Unknown"


@dataclass
class TranslationResult:
    """Container for translated text and related diagnostic messages."""

    translated_text: str
    translation_status: str
    model_name: str


@dataclass
class CaptionResult:
    """Container for the generated image caption."""

    caption: str
    model_name: str
    device: str
    elapsed_seconds: float


# ============================================================================
# Utility functions
# ============================================================================


def clean_text(text: str) -> str:
    """Normalize generated text for display.

    Args:
        text: Raw generated text.

    Returns:
        Cleaned string with spacing fixed and first letter capitalized.
    """
    if not text:
        return ""
    cleaned = " ".join(str(text).strip().split())
    if not cleaned:
        return ""
    return cleaned[0].upper() + cleaned[1:]



def safe_get(record: Dict[str, Any], candidate_keys: List[str], default: Any = "Unknown") -> Any:
    """Safely retrieve a value from a dictionary using possible field names.

    The world-languages dataset may evolve over time, so this function protects
    the app if field names vary slightly.
    """
    for key in candidate_keys:
        if key in record and record[key] not in [None, "", "null"]:
            return record[key]
    return default



def format_seconds(seconds: float) -> str:
    """Format latency values for user-friendly display."""
    return f"{seconds:.2f} seconds"



def ensure_rgb_image(image: Image.Image) -> Image.Image:
    """Convert uploaded images to RGB format for the BLIP processor."""
    if image is None:
        raise ValueError("No image was provided. Please upload an image first.")
    if not isinstance(image, Image.Image):
        raise TypeError("The uploaded file could not be read as an image.")
    return image.convert("RGB")



def build_explanation_from_caption(caption: str) -> str:
    """Create a short natural-language explanation from the image caption.

    This step represents a lightweight NLP layer. Instead of overcomplicating
    the app with a second large language model, it turns the caption into a
    simple, human-readable explanation appropriate for a class demonstration.
    """
    caption = clean_text(caption)
    if not caption:
        return "The model could not confidently describe this image."
    explanation = (
        f"The vision-language model describes the uploaded image as: '{caption}'. "
        "In plain language, the system is using visual features from the image "
        "and converting them into a written description that a person can read, "
        "translate, and compare across languages."
    )
    return explanation



def build_metadata_markdown(language: LanguageRecord) -> str:
    """Create markdown text showing selected world language metadata."""
    location = "Unknown"
    if language.latitude is not None and language.longitude is not None:
        location = f"{language.latitude}, {language.longitude}"

    return f"""
### Selected Language Metadata

| Field | Value |
|---|---|
| Language | {language.display_name} |
| ISO 639-3 | {language.iso_639_3} |
| Language Family | {language.family} |
| Region | {language.region} |
| Country / Area | {language.country} |
| Coordinates | {location} |
| Speaker Population | {language.population} |
""".strip()



def build_limitations_markdown(selected_language: str, translation_status: str) -> str:
    """Create a transparent limitations note for responsible AI reporting."""
    return f"""
### Responsible AI Notes

- The image caption is generated by a pretrained model and may be incomplete or incorrect.
- The selected language is **{selected_language}**.
- Translation status: **{translation_status}**.
- The app should not be used for emergency, medical, legal, or safety-critical translation.
- Uploaded images should not contain private, sensitive, or personally identifying information.
""".strip()


# ============================================================================
# Model loading functions
# ============================================================================

@lru_cache(maxsize=1)
def load_vision_components() -> Tuple[BlipProcessor, BlipForConditionalGeneration]:
    """Load the BLIP processor and captioning model once.

    Returns:
        A tuple containing the processor and model.
    """
    processor = BlipProcessor.from_pretrained(VISION_MODEL_NAME)
    model = BlipForConditionalGeneration.from_pretrained(VISION_MODEL_NAME)
    model.to(DEVICE)
    model.eval()
    return processor, model


@lru_cache(maxsize=8)
def load_translation_components(model_name: str) -> Tuple[AutoTokenizer, AutoModelForSeq2SeqLM]:
    """Load a translation tokenizer and model once per language.

    Args:
        model_name: Hugging Face model identifier.

    Returns:
        A tokenizer and sequence-to-sequence model.
    """
    tokenizer = AutoTokenizer.from_pretrained(model_name)
    model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
    model.to(DEVICE)
    model.eval()
    return tokenizer, model


@lru_cache(maxsize=1)
def load_world_languages() -> List[LanguageRecord]:
    """Load the world languages dataset and normalize metadata.

    The dataset is used to satisfy the project requirement and to provide
    language context. The app intentionally handles flexible field names so it
    remains stable if the dataset schema changes slightly.
    """
    try:
        dataset = load_dataset(WORLD_LANGUAGES_DATASET)
        split_name = "train" if "train" in dataset else list(dataset.keys())[0]
        records = dataset[split_name]
    except Exception:
        # Fallback records make the UI usable even if the dataset cannot be
        # downloaded because of a temporary connection issue.
        fallback = [
            LanguageRecord("Spanish", "spa", "Indo-European", "Europe / Americas", "Multiple", None, None, "Large"),
            LanguageRecord("French", "fra", "Indo-European", "Europe / Africa", "Multiple", None, None, "Large"),
            LanguageRecord("German", "deu", "Indo-European", "Europe", "Germany", None, None, "Large"),
            LanguageRecord("Italian", "ita", "Indo-European", "Europe", "Italy", None, None, "Large"),
            LanguageRecord("Portuguese", "por", "Indo-European", "Europe / Americas", "Multiple", None, None, "Large"),
            LanguageRecord("Dutch", "nld", "Indo-European", "Europe", "Netherlands", None, None, "Large"),
        ]
        return fallback

    normalized: List[LanguageRecord] = []

    for item in records:
        item_dict = dict(item)
        name = safe_get(
            item_dict,
            [
                "name",
                "language",
                "language_name",
                "Language",
                "Name",
                "languageName",
                "primary_name",
            ],
            "Unknown Language",
        )
        iso_639_3 = safe_get(
            item_dict,
            ["iso_639_3", "ISO_639_3", "iso", "ISO", "iso_code", "language_code"],
            "Unknown",
        )
        family = safe_get(
            item_dict,
            ["family", "language_family", "Family", "classification"],
            "Unknown",
        )
        region = safe_get(
            item_dict,
            ["region", "macroarea", "continent", "area", "Region"],
            "Unknown",
        )
        country = safe_get(
            item_dict,
            ["country", "countries", "Country", "primary_country"],
            "Unknown",
        )
        latitude = safe_get(item_dict, ["latitude", "lat", "Latitude"], None)
        longitude = safe_get(item_dict, ["longitude", "lon", "lng", "Longitude"], None)
        population = safe_get(
            item_dict,
            ["population", "speakers", "speaker_population", "Population"],
            "Unknown",
        )

        try:
            latitude = float(latitude) if latitude not in [None, "Unknown"] else None
        except Exception:
            latitude = None

        try:
            longitude = float(longitude) if longitude not in [None, "Unknown"] else None
        except Exception:
            longitude = None

        normalized.append(
            LanguageRecord(
                display_name=str(name),
                iso_639_3=str(iso_639_3),
                family=str(family),
                region=str(region),
                country=str(country),
                latitude=latitude,
                longitude=longitude,
                population=str(population),
            )
        )

    # Deduplicate by display name and put common supported translation languages first.
    unique: Dict[str, LanguageRecord] = {}
    for language in normalized:
        if language.display_name not in unique:
            unique[language.display_name] = language

    priority_names = list(TRANSLATION_MODEL_MAP.keys())
    ordered: List[LanguageRecord] = []

    for name in priority_names:
        match = find_language_by_name(list(unique.values()), name)
        if match:
            ordered.append(match)
        else:
            ordered.append(LanguageRecord(name))

    for name in sorted(unique.keys()):
        if name not in priority_names:
            ordered.append(unique[name])

    return ordered


# ============================================================================
# Language lookup functions
# ============================================================================


def find_language_by_name(languages: List[LanguageRecord], name: str) -> Optional[LanguageRecord]:
    """Find a language record by exact or case-insensitive name."""
    if not name:
        return None
    for language in languages:
        if language.display_name == name:
            return language
    lowered = name.lower().strip()
    for language in languages:
        if language.display_name.lower().strip() == lowered:
            return language
    return None



def get_language_choices() -> List[str]:
    """Return language names for the Gradio dropdown.

    To keep the interface usable, common translation-supported languages are
    shown first, followed by all available dataset languages.
    """
    languages = load_world_languages()
    names = [language.display_name for language in languages if language.display_name]
    seen = set()
    clean_names = []
    for name in names:
        if name not in seen:
            clean_names.append(name)
            seen.add(name)
    return clean_names[:1000]



def get_language_record(language_name: str) -> LanguageRecord:
    """Retrieve metadata for the selected language."""
    languages = load_world_languages()
    record = find_language_by_name(languages, language_name)
    if record:
        return record
    return LanguageRecord(display_name=language_name or "Unknown Language")


# ============================================================================
# AI processing functions
# ============================================================================


def generate_caption(image: Image.Image) -> CaptionResult:
    """Generate an English image caption using BLIP.

    Args:
        image: Uploaded PIL image.

    Returns:
        CaptionResult with text, model name, device, and latency.
    """
    start = time.time()
    image = ensure_rgb_image(image)
    processor, model = load_vision_components()

    inputs = processor(images=image, return_tensors="pt")
    inputs = {key: value.to(DEVICE) for key, value in inputs.items()}

    with torch.no_grad():
        output_ids = model.generate(
            **inputs,
            max_new_tokens=MAX_NEW_TOKENS_CAPTION,
            num_beams=3,
        )

    caption = processor.decode(output_ids[0], skip_special_tokens=True)
    caption = clean_text(caption)
    elapsed = time.time() - start

    return CaptionResult(
        caption=caption,
        model_name=VISION_MODEL_NAME,
        device=DEVICE,
        elapsed_seconds=elapsed,
    )



def translate_text(text: str, selected_language: str) -> TranslationResult:
    """Translate English text into a supported selected language.

    Args:
        text: English source text.
        selected_language: Display language selected by the user.

    Returns:
        TranslationResult containing translated text and status.
    """
    text = clean_text(text)
    if not text:
        return TranslationResult(
            translated_text="No text was available to translate.",
            translation_status="No translation performed because source text was empty.",
            model_name="None",
        )

    if selected_language == "English":
        return TranslationResult(
            translated_text=text,
            translation_status="English selected; no translation required.",
            model_name="None",
        )

    config = TRANSLATION_MODEL_MAP.get(selected_language)
    if not config:
        return TranslationResult(
            translated_text=text,
            translation_status=(
                "Translation model not included for this selected language. "
                "The app still displays language metadata from lukeslp/world-languages."
            ),
            model_name="Not available in this demo",
        )

    model_name = config["model"]

    try:
        tokenizer, model = load_translation_components(model_name)
        inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
        inputs = {key: value.to(DEVICE) for key, value in inputs.items()}

        with torch.no_grad():
            output_ids = model.generate(
                **inputs,
                max_new_tokens=MAX_NEW_TOKENS_TRANSLATION,
                num_beams=4,
            )

        translated = tokenizer.decode(output_ids[0], skip_special_tokens=True)
        translated = clean_text(translated)

        return TranslationResult(
            translated_text=translated,
            translation_status="Translation completed using a local Hugging Face model.",
            model_name=model_name,
        )

    except Exception as error:
        return TranslationResult(
            translated_text=text,
            translation_status=(
                "Translation failed, so the English caption is shown instead. "
                f"Error summary: {str(error)[:180]}"
            ),
            model_name=model_name,
        )



def analyze_image(
    image: Image.Image,
    selected_language: str,
    include_explanation: bool,
) -> Tuple[str, str, str, str, str]:
    """Main app pipeline used by the Gradio interface.

    Steps:
    1. Validate the uploaded image.
    2. Generate an English caption using a vision-language model.
    3. Create a plain-language explanation.
    4. Retrieve language metadata from the world languages dataset.
    5. Translate the caption when a supported language is selected.
    6. Return display-ready strings.
    """
    try:
        if image is None:
            raise ValueError("Please upload an image before running the model.")

        language_record = get_language_record(selected_language)
        caption_result = generate_caption(image)
        source_text = caption_result.caption

        translation_result = translate_text(source_text, selected_language)

        explanation = ""
        if include_explanation:
            explanation = build_explanation_from_caption(source_text)
        else:
            explanation = "Explanation hidden based on user selection."

        english_output = f"### English Caption\n\n{source_text}"
        translated_output = (
            f"### Output in {selected_language}\n\n"
            f"{translation_result.translated_text}"
        )
        metadata_output = build_metadata_markdown(language_record)
        system_output = f"""
### System Details

| Component | Value |
|---|---|
| Vision-Language Model | {caption_result.model_name} |
| Translation Model | {translation_result.model_name} |
| Device | {caption_result.device} |
| Caption Latency | {format_seconds(caption_result.elapsed_seconds)} |
| Dataset | {WORLD_LANGUAGES_DATASET} |

{build_limitations_markdown(selected_language, translation_result.translation_status)}
""".strip()

        explanation_output = f"### Plain-Language Explanation\n\n{explanation}"
        return english_output, translated_output, explanation_output, metadata_output, system_output

    except Exception as error:
        error_message = f"""
### Error

The application could not complete the request.

**Reason:** {str(error)}

**Suggested fix:** Check that an image was uploaded, the selected language is valid, and the Space has enough memory to load the model.
""".strip()
        debug_message = f"```text\n{traceback.format_exc()}\n```"
        return error_message, "", "", "", debug_message


# ============================================================================
# Sample content for README and report support
# ============================================================================

README_TEXT = """
# World Language Vision Assistant

## Overview

World Language Vision Assistant is a multimodal AI demonstration app. It allows a user to upload an image, generates an English caption with a pretrained vision-language model, and then displays the result with language metadata from the `lukeslp/world-languages` dataset. For selected supported languages, the app also translates the generated caption using Hugging Face translation models.

This project was created for a final machine learning project focused on deployment, multimodal AI, responsible AI, and Hugging Face Spaces.

## AI Capabilities Demonstrated

1. Computer vision: the app reads visual content from an uploaded image.
2. Vision-language modeling: the app converts image content into natural-language text.
3. Natural language processing: the app cleans and explains the generated caption.
4. Machine translation: the app translates English captions into supported languages.
5. Dataset integration: the app loads and uses the Hugging Face dataset `lukeslp/world-languages`.
6. Deployment: the app is designed for Hugging Face Spaces using Gradio.

## Files Needed for Hugging Face Spaces

Upload these files to a Gradio Space:

- app.py
- requirements.txt

## requirements.txt

```text
gradio
transformers
torch
torchvision
datasets
Pillow
sentencepiece
sacremoses
accelerate
```

## How to Run Locally

1. Install Python 3.10 or newer.
2. Create a virtual environment.
3. Install dependencies:

```bash
pip install -r requirements.txt
```

4. Run the application:

```bash
python app.py
```

5. Open the local Gradio link shown in the terminal.

## How to Deploy on Hugging Face Spaces

1. Sign in to Hugging Face.
2. Click New Space.
3. Choose Gradio as the SDK.
4. Name the Space, for example: `world-language-vision-assistant`.
5. Upload `app.py` and `requirements.txt`.
6. Wait for the Space to build.
7. Open the public Space link and test the app.

## Suggested Test Cases

Use simple, clear images first:

- A coffee mug
- A dog
- A laptop
- A flower
- A bicycle

For each test, record:

- Image used
- Selected language
- English caption
- Translated output
- Whether the caption was accurate
- Whether the translation was understandable
- Any limitations observed

## Ethical and Responsible AI Notes

This system should be treated as a demonstration tool, not a perfect translator or safety-critical image interpreter. It may misunderstand images, generate incomplete captions, or produce awkward translations. Users should avoid uploading private or sensitive images. The system is most appropriate for classroom demonstration, early prototyping, and basic multilingual accessibility experiments.
"""


PROJECT_SUMMARY_TEXT = """
## Project Summary for Report

The World Language Vision Assistant is a deployed multimodal AI application that combines computer vision, natural language processing, multilingual data, and machine translation. The system accepts an image from the user, uses a pretrained BLIP vision-language model to generate an English caption, retrieves language information from the Hugging Face `lukeslp/world-languages` dataset, and translates the caption into selected supported languages. The final interface is built with Gradio and is ready for deployment on Hugging Face Spaces.

The project demonstrates how several AI capabilities can work together in a single practical application. Rather than only training a model in a notebook, the project focuses on creating an interactive system that a nontechnical user can open, test, and understand. This makes the project realistic for a real-world AI deployment scenario.
"""


# ============================================================================
# Gradio interface construction
# ============================================================================


def build_interface() -> gr.Blocks:
    """Build and return the Gradio app interface."""
    language_choices = get_language_choices()
    if DEFAULT_LANGUAGE not in language_choices:
        language_choices.insert(0, DEFAULT_LANGUAGE)

    custom_css = """
    .main-title {
        text-align: center;
        margin-bottom: 0.25rem;
    }
    .subtitle {
        text-align: center;
        color: #555;
        margin-bottom: 1.5rem;
    }
    .small-note {
        font-size: 0.92rem;
        color: #555;
    }
    """

    with gr.Blocks(title=APP_TITLE, css=custom_css) as demo:
        gr.Markdown(f"# {APP_TITLE}", elem_classes=["main-title"])
        gr.Markdown(f"**{APP_SUBTITLE}**", elem_classes=["subtitle"])

        with gr.Row():
            with gr.Column(scale=1):
                image_input = gr.Image(
                    label="Upload Image",
                    type="pil",
                    height=360,
                )
                selected_language = gr.Dropdown(
                    choices=language_choices,
                    value=DEFAULT_LANGUAGE,
                    label="Select Output Language",
                    info=(
                        "Common supported translations include Spanish, French, German, "
                        "Italian, Portuguese, and Dutch. Other languages display metadata."
                    ),
                )
                include_explanation = gr.Checkbox(
                    value=True,
                    label="Include plain-language explanation",
                )
                run_button = gr.Button("Analyze Image", variant="primary")

                gr.Markdown(
                    """
                    <div class='small-note'>
                    Tip: For the best classroom demo, use clear images with one main subject, such as a mug, dog, laptop, flower, or bicycle.
                    </div>
                    """
                )

            with gr.Column(scale=1):
                english_output = gr.Markdown(label="English Caption")
                translated_output = gr.Markdown(label="Translated Output")

        with gr.Row():
            with gr.Column():
                explanation_output = gr.Markdown(label="Plain-Language Explanation")
            with gr.Column():
                metadata_output = gr.Markdown(label="Language Metadata")

        system_output = gr.Markdown(label="System Details and Responsible AI Notes")

        with gr.Accordion("README / Running and Deployment Instructions", open=False):
            gr.Markdown(README_TEXT)

        with gr.Accordion("Project Summary for Report", open=False):
            gr.Markdown(PROJECT_SUMMARY_TEXT)

        run_button.click(
            fn=analyze_image,
            inputs=[image_input, selected_language, include_explanation],
            outputs=[
                english_output,
                translated_output,
                explanation_output,
                metadata_output,
                system_output,
            ],
        )

        gr.Examples(
            examples=[],
            inputs=[image_input, selected_language, include_explanation],
            label="Example inputs can be added after uploading sample images to the Space repository.",
        )

    return demo


# ============================================================================
# Evaluation helper functions for final report
# ============================================================================


def evaluate_caption_basic(expected_keywords: List[str], generated_caption: str) -> Dict[str, Any]:
    """Simple keyword-based evaluation helper.

    This is not a perfect evaluation metric, but it is practical for a classroom
    project. It checks whether expected concepts appear in the generated caption.
    The final report can describe this as a lightweight qualitative/keyword
    evaluation method.
    """
    caption_lower = generated_caption.lower()
    matched = []
    missed = []

    for keyword in expected_keywords:
        if keyword.lower() in caption_lower:
            matched.append(keyword)
        else:
            missed.append(keyword)

    score = len(matched) / len(expected_keywords) if expected_keywords else 0.0

    return {
        "expected_keywords": expected_keywords,
        "matched_keywords": matched,
        "missed_keywords": missed,
        "keyword_match_score": round(score, 3),
    }



def build_test_log_template() -> str:
    """Return a plain text template the student can use for manual testing."""
    return """
Manual Testing Log Template
---------------------------
Test Number:
Image Description:
Selected Language:
Expected Main Object:
English Caption Generated:
Translated Output:
Was the main object correct? Yes/No
Was the translation understandable? Yes/No/Partially
Latency Observed:
Notes / Limitations:
Screenshot Filename:
""".strip()


# ============================================================================
# Main entry point
# ============================================================================

if __name__ == "__main__":
    # Hugging Face Spaces expects the app to launch from app.py. The server_name
    # setting allows the app to listen properly in containerized environments.
    demo = build_interface()
    demo.launch(server_name="0.0.0.0", server_port=int(os.environ.get("PORT", 7860)))