README.md

---
license: apache-2.0
tags:
- remote-sensing
- computer-vision
- swin-transformer
- building-extraction
- change-detection
- foundation-model
datasets:
- remote-sensing-images
model-index:
- name: RSBuilding-Swin-B
  results: []
library_name: transformers
pipeline_tag: feature-extraction
---

# RSBuilding-Swin-B

HuggingFace Transformers version of RSBuilding Swin-Base model, converted from MMDetection/MMSegmentation format.

## Source

- **Source Code**: [https://github.com/Meize0729/RSBuilding](https://github.com/Meize0729/RSBuilding)
- **Original Checkpoint**: [https://huggingface.co/models/BiliSakura/RSBuilding](https://huggingface.co/models/BiliSakura/RSBuilding)

## Model Information

- **Architecture**: Swin Transformer Base
- **Embedding Dimension**: 128
- **Depths**: [2, 2, 18, 2]
- **Number of Heads**: [4, 8, 16, 32]
- **Window Size**: 12
- **Image Size**: 384×384
- **Patch Size**: 4×4

## Important Notes

### Missing Buffer Keys (Expected)

When loading this model, you may see messages about missing buffer keys (typically ~12 keys). **This is expected and normal.**

These missing keys are buffers that are computed dynamically during model initialization:
- `relative_position_index`: Precomputed index mapping for window-based attention
- `relative_coords_table`: Precomputed coordinate table for relative positions
- `relative_position_bias_table`: Precomputed bias table

**Why they're missing:**
- These buffers are recalculated each time the model is instantiated based on `window_size` and other configuration parameters
- They don't need to be saved in checkpoints because they're deterministic and computed from config
- This is standard behavior in HuggingFace Swin transformers

**Action required:** None. The model will work correctly with these buffers computed automatically.

## Quick Start

### Installation

```bash
pip install transformers torch pillow
```

### Inference Example

```python
from transformers import SwinModel, AutoImageProcessor
from PIL import Image
import torch

# Load model and processor
model = SwinModel.from_pretrained("BiliSakura/RSBuilding-Swin-B")
processor = AutoImageProcessor.from_pretrained("BiliSakura/RSBuilding-Swin-B")

# Load and process image
image = Image.open("your_image.jpg")
inputs = processor(image, return_tensors="pt")

# Forward pass
with torch.no_grad():
    outputs = model(**inputs)

# Get features
# outputs.last_hidden_state: (batch_size, num_patches, hidden_size)
# outputs.pooler_output: (batch_size, hidden_size) - pooled representation
features = outputs.last_hidden_state
pooled_features = outputs.pooler_output

print(f"Feature shape: {features.shape}")
print(f"Pooled feature shape: {pooled_features.shape}")
```

### Feature Extraction for Downstream Tasks

```python
from transformers import SwinModel, AutoImageProcessor
import torch

model = SwinModel.from_pretrained("BiliSakura/RSBuilding-Swin-B")
processor = AutoImageProcessor.from_pretrained("BiliSakura/RSBuilding-Swin-B")

# Process image
image = Image.open("your_image.jpg")
inputs = processor(image, return_tensors="pt")

# Extract features
with torch.no_grad():
    outputs = model(**inputs)
    
# Use pooled features for classification/regression
features = outputs.pooler_output  # Shape: (1, 1024)

# Or use last hidden state for dense prediction tasks
spatial_features = outputs.last_hidden_state  # Shape: (1, num_patches, 1024)
```

## Model Configuration

The model uses the following configuration:
- `image_size`: 384
- `patch_size`: 4
- `num_channels`: 3
- `embed_dim`: 128
- `depths`: [2, 2, 18, 2]
- `num_heads`: [4, 8, 16, 32]
- `window_size`: 12
- `mlp_ratio`: 4.0
- `hidden_act`: "gelu"

## Citation

If you use this model, please cite the original RSBuilding paper:

```bibtex
@article{wangRSBuildingGeneralRemote2024a,
  title = {{{RSBuilding}}: {{Toward General Remote Sensing Image Building Extraction}} and {{Change Detection With Foundation Model}}},
  shorttitle = {{{RSBuilding}}},
  author = {Wang, Mingze and Su, Lili and Yan, Cilin and Xu, Sheng and Yuan, Pengcheng and Jiang, Xiaolong and Zhang, Baochang},
  year = {2024},
  journal = {IEEE Transactions on Geoscience and Remote Sensing},
  volume = {62},
  pages = {1--17},
  issn = {1558-0644},
  doi = {10.1109/TGRS.2024.3439395},
  keywords = {Building extraction,Buildings,change detection (CD),Data mining,Feature extraction,federated training,foundation model,Image segmentation,Remote sensing,remote sensing images,Task analysis,Training}
}
```