google-cloud/pr_98/en/containers/available.html

<meta charset="utf-8" /><meta name="hf:doc:metadata" content="{&quot;title&quot;:&quot;DLCs on Google Cloud&quot;,&quot;local&quot;:&quot;dlcs-on-google-cloud&quot;,&quot;sections&quot;:[{&quot;title&quot;:&quot;Text Generation Inference (TGI)&quot;,&quot;local&quot;:&quot;text-generation-inference-tgi&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2},{&quot;title&quot;:&quot;Text Embeddings Inference (TEI)&quot;,&quot;local&quot;:&quot;text-embeddings-inference-tei&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2},{&quot;title&quot;:&quot;PyTorch Inference&quot;,&quot;local&quot;:&quot;pytorch-inference&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2},{&quot;title&quot;:&quot;PyTorch Training&quot;,&quot;local&quot;:&quot;pytorch-training&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2}],&quot;depth&quot;:1}">
		<link href="/docs/google-cloud/pr_98/en/_app/immutable/assets/0.e3b0c442.css" rel="modulepreload">
		<link rel="modulepreload" href="/docs/google-cloud/pr_98/en/_app/immutable/entry/start.f33eecf6.js">
		<link rel="modulepreload" href="/docs/google-cloud/pr_98/en/_app/immutable/chunks/scheduler.8b74b908.js">
		<link rel="modulepreload" href="/docs/google-cloud/pr_98/en/_app/immutable/chunks/singletons.02266636.js">
		<link rel="modulepreload" href="/docs/google-cloud/pr_98/en/_app/immutable/chunks/paths.ed5b02d0.js">
		<link rel="modulepreload" href="/docs/google-cloud/pr_98/en/_app/immutable/entry/app.3791794a.js">
		<link rel="modulepreload" href="/docs/google-cloud/pr_98/en/_app/immutable/chunks/index.0ed2a570.js">
		<link rel="modulepreload" href="/docs/google-cloud/pr_98/en/_app/immutable/nodes/0.29c824a2.js">
		<link rel="modulepreload" href="/docs/google-cloud/pr_98/en/_app/immutable/nodes/2.83bd6bd4.js">
		<link rel="modulepreload" href="/docs/google-cloud/pr_98/en/_app/immutable/chunks/Tip.07d3ac1e.js">
		<link rel="modulepreload" href="/docs/google-cloud/pr_98/en/_app/immutable/chunks/EditOnGithub.d2d81eda.js"><!-- HEAD_svelte-u9bgzb_START --><meta name="hf:doc:metadata" content="{&quot;title&quot;:&quot;DLCs on Google Cloud&quot;,&quot;local&quot;:&quot;dlcs-on-google-cloud&quot;,&quot;sections&quot;:[{&quot;title&quot;:&quot;Text Generation Inference (TGI)&quot;,&quot;local&quot;:&quot;text-generation-inference-tgi&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2},{&quot;title&quot;:&quot;Text Embeddings Inference (TEI)&quot;,&quot;local&quot;:&quot;text-embeddings-inference-tei&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2},{&quot;title&quot;:&quot;PyTorch Inference&quot;,&quot;local&quot;:&quot;pytorch-inference&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2},{&quot;title&quot;:&quot;PyTorch Training&quot;,&quot;local&quot;:&quot;pytorch-training&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2}],&quot;depth&quot;:1}"><!-- HEAD_svelte-u9bgzb_END -->      <p></p>  <h1 class="relative group"><a id="dlcs-on-google-cloud" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#dlcs-on-google-cloud"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>DLCs on Google Cloud</span></h1> <p data-svelte-h="svelte-cddt1o">Below you can find a listing of all the Deep Learning Containers (DLCs) available on Google Cloud. Containers are created for each supported combination of use-case (training, inference), accelerator type (CPU, GPU, TPU), and framework (PyTorch, TGI, TEI).</p>  <div class="course-tip  bg-gradient-to-br dark:bg-gradient-to-r before:border-green-500 dark:before:border-green-800 from-green-50 dark:from-gray-900 to-white dark:to-gray-950 border border-green-50 text-green-700 dark:text-gray-400"><p data-svelte-h="svelte-lkacfy">The listing below only contains the latest version of each one of the Hugging Face DLCs, the full listing of the available published containers in Google Cloud can be found either in the <a href="https://cloud.google.com/deep-learning-containers/docs/choosing-container#hugging-face" rel="nofollow">Google Cloud Deep Learning Containers Documentation</a>, in the <a href="https://console.cloud.google.com/artifacts/docker/deeplearning-platform-release/us/gcr.io" rel="nofollow">Google Cloud Artifact Registry</a> or via the <code>gcloud container images list --repository=&quot;us-docker.pkg.dev/deeplearning-platform-release/gcr.io&quot; | grep &quot;huggingface-&quot;</code> command.</p></div>  <h2 class="relative group"><a id="text-generation-inference-tgi" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#text-generation-inference-tgi"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>Text Generation Inference (TGI)</span></h2> <p data-svelte-h="svelte-1xju71k">Text Generation Inference (TGI) DLC is available for high-performance text generation of Large Language Models on both GPU and TPU (soon). The TGI DLC enables you to deploy <a href="https://huggingface.co/models?other=text-generation-inference&sort=trending" rel="nofollow">any of the +140,000 text generation inference supported models from the Hugging Face Hub</a>, or any custom model as long as <a href="https://huggingface.co/docs/text-generation-inference/supported_models" rel="nofollow">its architecture is supported within TGI</a>.</p> <table data-svelte-h="svelte-16qrj82"><thead><tr><th>Container URI</th> <th>Path</th> <th>Accelerator</th></tr></thead> <tbody><tr><td>us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-text-generation-inference-cu121.2-2.ubuntu2204.py310</td> <td><a href="./containers/tgi/gpu/2.2.0/Dockerfile">text-generation-inference-gpu.2.2.0</a></td> <td>GPU</td></tr></tbody></table>  <h2 class="relative group"><a id="text-embeddings-inference-tei" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#text-embeddings-inference-tei"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>Text Embeddings Inference (TEI)</span></h2> <p data-svelte-h="svelte-13c4h5t">Text Embeddings Inference (TEI) DLC is available for high-performance serving of embedding models on both GPU and GPU. The TEI DLC enables you to deploy <a href="https://huggingface.co/models?other=text-embeddings-inference&sort=trending" rel="nofollow">any of the +10,000 embedding, re-ranking or sequence classification supported models from the Hugging Face Hub</a>, or any custom model as long as <a href="https://huggingface.co/docs/text-embeddings-inference/en/supported_models" rel="nofollow">its architecture is supported within TEI</a>.</p> <table data-svelte-h="svelte-18rzbr7"><thead><tr><th>Container URI</th> <th>Path</th> <th>Accelerator</th></tr></thead> <tbody><tr><td>us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-text-embeddings-inference-cu122.1-4.ubuntu2204</td> <td><a href="./containers/tei/gpu/1.4.0/Dockerfile">text-embeddings-inference-gpu.1.4.0</a></td> <td>GPU</td></tr> <tr><td>us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-text-embeddings-inference-cpu.1-4</td> <td><a href="./containers/tei/cpu/1.4.0/Dockerfile">text-embeddings-inference-cpu.1.4.0</a></td> <td>CPU</td></tr></tbody></table>  <h2 class="relative group"><a id="pytorch-inference" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#pytorch-inference"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>PyTorch Inference</span></h2> <p data-svelte-h="svelte-122ovbm">Pytorch Inference DLC is available for Pytorch via 🤗 Transformers, for serving models trained with 🤗 TRL, Sentence Transformers or 🧨 Diffusers, on both CPU and GPU.</p> <table data-svelte-h="svelte-p98suh"><thead><tr><th>Container URI</th> <th>Path</th> <th>Accelerator</th></tr></thead> <tbody><tr><td>us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-inference-cu121.2-2.transformers.4-44.ubuntu2204.py311</td> <td><a href="./containers/pytorch/inference/gpu/2.2.2/transformers/4.44.0/py311/Dockerfile">huggingface-pytorch-inference-gpu.2.2.2.transformers.4.44.0.py311</a></td> <td>GPU</td></tr> <tr><td>us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-inference-cpu.2-2.transformers.4-44.ubuntu2204.py311</td> <td><a href="./containers/pytorch/inference/cpu/2.2.2/transformers/4.44.0/py311/Dockerfile">huggingface-pytorch-inference-cpu.2.2.2.transformers.4.44.0.py311</a></td> <td>CPU</td></tr></tbody></table>  <h2 class="relative group"><a id="pytorch-training" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#pytorch-training"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>PyTorch Training</span></h2> <p data-svelte-h="svelte-novjsm">Pytorch Training DLC is available for PyTorch via 🤗 Transformers. It includes support for training with libraries such as 🤗 TRL, Sentence Transformers, or 🧨 Diffusers, on both GPUs and TPUs (soon).</p> <table data-svelte-h="svelte-15qyvwm"><thead><tr><th>Container URI</th> <th>Path</th> <th>Accelerator</th></tr></thead> <tbody><tr><td>us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-training-cu121.2-3.transformers.4-42.ubuntu2204.py310</td> <td><a href="./containers/pytorch/training/gpu/2.3.0/transformers/4.42.3/py310/Dockerfile">huggingface-pytorch-training-gpu.2.3.0.transformers.4.42.3.py310</a></td> <td>GPU</td></tr></tbody></table> <a class="!text-gray-400 !no-underline text-sm flex items-center not-prose mt-4" href="https://github.com/huggingface/Google-Cloud-Containers/blob/main/docs/source/containers/available.mdx" target="_blank"><span data-svelte-h="svelte-1kd6by1">&lt;</span> <span data-svelte-h="svelte-x0xyl0">&gt;</span> <span data-svelte-h="svelte-1dajgef"><span class="underline ml-1.5">Update</span> on GitHub</span></a>  <p></p> 
			
			<script>
				{
					__sveltekit_o3ffq7 = {
						assets: "/docs/google-cloud/pr_98/en",
						base: "/docs/google-cloud/pr_98/en",
						env: {}
					};

					const element = document.currentScript.parentElement;

					const data = [null,null];

					Promise.all([
						import("/docs/google-cloud/pr_98/en/_app/immutable/entry/start.f33eecf6.js"),
						import("/docs/google-cloud/pr_98/en/_app/immutable/entry/app.3791794a.js")
					]).then(([kit, app]) => {
						kit.start(app, element, {
							node_ids: [0, 2],
							data,
							form: null,
							error: null
						});
					});
				}
			</script>