gyoyeonahn
/

MuseEye

@@ -1,73 +1,112 @@
-# Google AI Edge Gallery ✨
-[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)
-[![GitHub release (latest by date)](https://img.shields.io/github/v/release/google-ai-edge/gallery)](https://github.com/google-ai-edge/gallery/releases)
-**Explore, Experience, and Evaluate the Future of On-Device Generative AI with Google AI Edge.**
-The Google AI Edge Gallery is an experimental app that puts the power of cutting-edge Generative AI models directly into your hands, running entirely on your Android *(available now)* and iOS *(coming soon)* devices. Dive into a world of creative and practical AI use cases, all running locally, without needing an internet connection once the model is loaded. Experiment with different models, chat, ask questions with images and audio clip, explore prompts, and more!
-Install the app today from Google Play
-<a href='https://play.google.com/store/apps/details?id=com.google.ai.edge.gallery'><img alt='Get it on Google Play' width="250" src='https://play.google.com/intl/en_us/badges/static/images/badges/en_badge_web_generic.png'/></a>
-For users without Google Play access, install the apk from the [**latest release**](https://github.com/google-ai-edge/gallery/releases/latest/)
-> [!IMPORTANT]
-> You must uninstall all previous versions of the app before installing this one. Past versions will no longer be working and supported.
-<img width="480" alt="01" src="https://github.com/user-attachments/assets/2a60c8d0-ef4e-4040-a948-fa73f6a622b4" />
-<img width="480" alt="02" src="https://github.com/user-attachments/assets/d155d458-b822-415d-9252-7e825fe8c9c0" />
-<img width="480" alt="03" src="https://github.com/user-attachments/assets/1977af6f-ee7e-41b3-aac1-a642c66c0058" />
-<img width="480" alt="04" src="https://github.com/user-attachments/assets/a48be969-f57e-4497-9ecf-8feb35f2ba71" />
-<img width="480" alt="05" src="https://github.com/user-attachments/assets/2a9679ea-f191-4ffd-87db-6726f7c1057d" />
-## ✨ Core Features
-*   **📱 Run Locally, Fully Offline:** Experience the magic of GenAI without an internet connection. All processing happens directly on your device.
-*   **🤖 Choose Your Model:** Easily switch between different models from Hugging Face and compare their performance.
-*   **🖼️ Ask Image:** Upload images and ask questions about them. Get descriptions, solve problems, or identify objects.
-*   **🎙️ Audio Scribe:** Transcribe an uploaded or recorded audio clip into text or translate it into another language.
-*   **✍️ Prompt Lab:** Summarize, rewrite, generate code, or use freeform prompts to explore single-turn LLM use cases.
-*   **💬 AI Chat:** Engage in multi-turn conversations.
-*   **📊 Performance Insights:** Real-time benchmarks (TTFT, decode speed, latency).
-*   **🧩 Bring Your Own Model:** Test your local LiteRT `.litermlm` models.
-*   **🔗 Developer Resources:** Quick links to model cards and source code.
-## 🏁 Get Started in Minutes!
-1. **Check OS Requirement**: Android 12 and up
-2.  **Download the App:**
-    - Install the app from [Google Play](https://play.google.com/store/apps/details?id=com.google.ai.edge.gallery).
-    - For users without Google Play access: install the apk from the [**latest release**](https://github.com/google-ai-edge/gallery/releases/latest/)
-3.  **Install & Explore:** For detailed installation instructions (including for corporate devices) and a full user guide, head over to our [**Project Wiki**](https://github.com/google-ai-edge/gallery/wiki)!
-## 🛠️ Technology Highlights
-*   **Google AI Edge:** Core APIs and tools for on-device ML.
-*   **LiteRT:** Lightweight runtime for optimized model execution.
-*   **LLM Inference API:** Powering on-device Large Language Models.
-*   **Hugging Face Integration:** For model discovery and download.
-## ⌨️ Development
-Check out the [development notes](DEVELOPMENT.md) for instructions about how to build the app locally.
-## 🤝 Feedback
-This is an **experimental Beta release**, and your input is crucial!
-*   🐞 **Found a bug?** [Report it here!](https://github.com/google-ai-edge/gallery/issues/new?assignees=&labels=bug&template=bug_report.md&title=%5BBUG%5D)
-*   💡 **Have an idea?** [Suggest a feature!](https://github.com/google-ai-edge/gallery/issues/new?assignees=&labels=enhancement&template=feature_request.md&title=%5BFEATURE%5D)
-## 📄 License
-Licensed under the Apache License, Version 2.0. See the [LICENSE](LICENSE) file for details.
-## 🔗 Useful Links
-*   [**Project Wiki (Detailed Guides)**](https://github.com/google-ai-edge/gallery/wiki)
-*   [Hugging Face LiteRT Community](https://huggingface.co/litert-community)
-*   [LLM Inference guide for Android](https://ai.google.dev/edge/mediapipe/solutions/genai/llm_inference/android)
-*   [LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM)
-*   [Google AI Edge Documentation](https://ai.google.dev/edge)

+---
+title: "Muse:Eye — On-device Multimodal XR Docent"
+emoji: 🎨
+colorFrom: blue
+colorTo: purple
+sdk: other
+pipeline_tag: other
+tags:
+  - multimodal
+  - XR
+  - on-device
+  - tflite
+  - mobileclip
+  - android
+  - unity
+  - RAG
+  - museum-tech
+---
+# Muse:Eye
+**On-device Multimodal AI + XR Museum Docent for Android**
+Muse:Eye는 **카메라 기반 작품 인식(MobileCLIP)**,
+**온디바이스 LLM(Gemma-3N)**,
+**XR 인터랙션(Unity MR)**
+을 결합한 하이브리드 AI 도슨트 시스템입니다.
+오프라인 환경에서도 빠른 이미지 검색,
+메타데이터 기반 RAG,
+성인/어린이/전문가 모드 설명 제공이 가능합니다.
+---
+# ✨ Features
+### 1) ⚡ On-device AI (No Internet Required)
+- MobileCLIP (TFLite) 이미지 임베딩
+- 작품 유사도 검색 (FAISS → custom `.bin` index)
+- Gemma-3N 온디바이스 LLM
+### 2) 🖼 Artwork Recognition
+- 카메라 인식 → 즉시 embedding 추출
+- RAG 기반 작품·작가 설명 생성
+### 3) 👶🧑‍🦳 Three Docent Modes
+- 어린이 모드
+- 성인 모드
+- 전문가 모드
+### 4) 🥽 XR Integrated
+- Unity + Android Studio 통합
+- MR 모드에서 3D UI 및 음성 안내 제공
+---
+# 🏗 System Architecture
+Muse:Eye는 두 가지 실행 모드를 지원합니다:
+- **Mobile Mode (On-device AI)** — 완전 오프라인
+- **XR/MR Mode (Cloud Multimodal AI)** — Unity 기반 MR + Cloud API
+---
+## 📱 A. Mobile Mode (On-device AI)
+```plaintext
+Mobile App (Android)
+│
+├── Camera Input
+│
+├── MobileCLIP (TFLite)
+│     └─ on-device embedding (512-d)
+│
+├── Embedding Index (.bin)
+│     └─ cosine similarity search (offline)
+│
+├── Local Metadata RAG
+│     └─ artwork metadata lookup
+│
+├── Gemma-3N On-device LLM
+│     └─ adult / child / expert explanation
+│
+└── Android TTS
+```
+---
+## 🥽 B. XR/MR Mode (Cloud Multimodal AI)
+```plaintext
+Unity XR App (MR View)
+│
+├── MR Camera (RenderTexture)
+│
+├── JNI Bridge → Android
+│     └─ frame bytes 전달
+│
+├── Cloud Multimodal API (Gemini)
+│     ├─ artwork analysis
+│     ├─ style / meaning extraction
+│     └─ multimodal reasoning
+│
+├── Cloud PromptManager
+│     ├─ 어린이 모드
+│     ├─ 성인 모드
+│     └─ 전문가 모드
+│
+└── Unity 3D UI + Android TTS
+      ├─ floating info panel
+      ├── 3D guide elements
+      └── audio output
+```