inference-endpoints/pr_145/en/engines/custom_container.md

Deploy with your own container

If the available Inference Engines don't meet your requirements, you can deploy your own custom solution as a Docker container and run it on Inference Endpoints. You can use public images like tensorflow/serving:2.7.3 or private images hosted on Docker Hub, AWS ECR, Azure ACR, or Google GCR.

The creation flow of your image artifacts from a custom image is the same as the base image. This means Inference Endpoints will create a unique image artifact derived from your provided image, including all model artifacts.

The model artifacts (weights) are stored under /repository. For example, if you use tensorflow/serving as your custom image, then you have to set `model_base_path="/repository":

tensorflow_model_server \
  --rest_api_port=5000 \
  --model_name=my_model \
  --model_base_path="/repository"

HF_DOC_BODY_END