README.md

---
library_name: craftsman-v1-5
license: creativeml-openrail-m
license_name: creativeml-openrail-m
license_link: https://raw.githubusercontent.com/CompVis/stable-diffusion/refs/heads/main/LICENSE
pipeline_tag: image-to-3d
language:
- en
- zh
---

## **CraftsMan-v1-5**

### <div align="center">CraftsMan: High-fidelity Mesh Generation <br> with 3D Native Generation and Interactive Geometry Refiner<div> 
#####  <p align="center"> [Weiyu Li<sup>*1,2</sup>](https://wyysf-98.github.io/), Jiarui Liu<sup>*1,2</sup>, Hongyu Yan<sup>*1,2</sup>, [Rui Chen<sup>1,2</sup>](https://aruichen.github.io/), [Yixun Liang<sup>2,3</sup>](https://yixunliang.github.io/), [Xuelin Chen<sup>4</sup>](https://xuelin-chen.github.io/), [Ping Tan<sup>1,2</sup>](https://ece.hkust.edu.hk/pingtan), [Xiaoxiao Long<sup>1,2</sup>](https://www.xxlong.site/)</p>
#####  <p align="center"> <sup>1</sup>HKUST, <sup>2</sup>LightIllusions, <sup>3</sup>HKUST(GZ), <sup>4</sup>Tencent AI Lab</p>
<div align="center">
  <a href="https://craftsman3d.github.io/"><img src="https://img.shields.io/static/v1?label=Project%20Page&message=Github&color=blue&logo=github-pages"></a> &ensp;
  <a href="http://algodemo.bj.lightions.top:24926"><img src="https://www.gradio.app/_app/immutable/assets/gradio.CHB5adID.svg" height="25"/></a> &ensp;
  <a href="https://arxiv.org/pdf/2405.14979"><img src="https://img.shields.io/static/v1?label=Paper&message=Arxiv&color=red&logo=arxiv"></a> &ensp;
</div>

# Usage

To use the model, please refer to the [official repository](https://github.com/wyysf-98/CraftsMan) for installation and usage instructions.

```

from craftsman import CraftsManPipeline
import torch

pipeline = CraftsManPipeline.from_pretrained("./ckpts/craftsman-v1-5", device="cuda:0", torch_dtype=torch.float32) # load from local ckpt
mesh = pipeline("https://pub-f9073a756ec645d692ce3d171c2e1232.r2.dev/data/werewolf.png").meshes[0]
mesh.export("werewolf.obj")

```

## 🔥🔥🔥 News!!

* Nov 16, 2024: 💬 We release the CraftsMan-v1-5


## 📑 Open-source Plan

- [x] Inference 
- [x] Checkpoints
- [x] Training
- [ ] ComfyUI

## 🎉 **CraftMan-v1-5 Architecture**

<p align="center">
  <img src="./assets/arch.png"  height=400>
</p>


## Get Started

#### Begin by cloning the repository:

```shell
git clone https://github.com/wyysf-98/CraftsMan
cd CraftsMan
```

#### Installation Guide for Linux

We provide an env_install.sh script file for setting up environment. 

```
# step 1, create conda env
conda create -n CraftsMan python=3.10
conda activate CraftsMan


# step 2. install torch realated package
conda install -c pytorch pytorch=2.3.0 torchvision=0.18.0 cudatoolkit=11.8

# step 3. install other packages
pip install -r docker/requirements.txt
```


#### Using Gradio

We have prepared a gradio demo for you to try out the model. You can run the following command to start the demo.

```shell
# std 
python3 gradio.py
```

Then the demo can be accessed through the output link.


## Citation

If you found this repository helpful, please cite our report:
```bibtex
@misc{li2024craftsman,
title         = {CraftsMan: High-fidelity Mesh Generation with 3D Native Generation and Interactive Geometry Refiner}, 
author        = {Weiyu Li and Jiarui Liu and Rui Chen and Yixun Liang and Xuelin Chen and Ping Tan and Xiaoxiao Long},
year          = {2024},
archivePrefix = {arXiv preprint arXiv:2405.14979},
primaryClass  = {cs.CG}
}
```


# License

[creativeml-openrail-m](https://raw.githubusercontent.com/CompVis/stable-diffusion/refs/heads/main/LICENSE)