---
title: NanoVLM
emoji: 🤖
colorFrom: blue
colorTo: indigo
sdk: gradio
sdk_version: "5.0.0"
app_file: app.py
pinned: false
license: mit
---

# NanoVLM

NanoVLM is a lightweight Vision-Language Model (VLM) designed for efficient multimodal understanding. It combines image encoding and language generation to answer questions about images while remaining suitable for resource-constrained environments.

## Features

- 🖼️ Image understanding
- 💬 Visual Question Answering (VQA)
- ⚡ Lightweight architecture
- 🤗 Hugging Face compatible
- 📓 Training and inference notebook included

## Model Architecture

The model consists of:

- Vision Encoder
- Projection Layer
- Language Model
- Cross-modal fusion between visual and textual representations

```
Image
   │
Vision Encoder
   │
Projection Layer
   │
Language Model
   │
Generated Answer
```

## Installation

Clone the repository

```bash
git clone https://github.com/your-username/nanovlm.git
cd nanovlm
```

Install dependencies

```bash
pip install -r requirements.txt
```

## Usage

### Inference

```python
from PIL import Image

image = Image.open("example.jpg")

question = "What is in this image?"

answer = model.chat(image, question)

print(answer)
```

### Notebook

Open the notebook:

```
NanoVlm_fixed.ipynb
```

and execute all cells.

## Dataset

The model can be trained on datasets such as:

- COCO Captions
- VQAv2
- Flickr30k
- Custom datasets

## Training

Example:

```bash
python train.py
```

Adjust hyperparameters such as:

- Learning rate
- Batch size
- Number of epochs
- Image resolution

## Results

| Metric | Value |
|---------|------:|
| Accuracy | -- |
| BLEU | -- |
| CIDEr | -- |

*(Replace with your experimental results.)*

## Repository Structure

```
.
├── NanoVlm_fixed.ipynb
├── app.py
├── requirements.txt
├── README.md
├── images/
└── model/
```

## Requirements

- Python 3.10+
- PyTorch
- Transformers
- Pillow
- Torchvision
- Gradio (optional)

Install:

```bash
pip install torch torchvision transformers pillow gradio
```

## Citation

If you use this project, please cite:

```bibtex
@misc{nanovlm2026,
  title={NanoVLM: A Lightweight Vision-Language Model},
  author={Your Name},
  year={2026},
  publisher={Hugging Face}
}
```

## License

MIT License

## Acknowledgements

This project builds upon the excellent work of:

- Hugging Face Transformers
- PyTorch
- Vision Transformer (ViT)
- Large Language Models research

---

⭐ If you find this project useful, consider giving it a star on GitHub or liking it on Hugging Face.