inference-endpoints/pr_113/en/guides/custom_dependencies.html

<meta charset="utf-8" /><meta name="hf:doc:metadata" content="{&quot;title&quot;:&quot;Add custom Dependencies&quot;,&quot;local&quot;:&quot;add-custom-dependencies&quot;,&quot;sections&quot;:[],&quot;depth&quot;:1}">
		<link href="/docs/inference-endpoints/pr_113/en/_app/immutable/assets/0.e3b0c442.css" rel="modulepreload">
		<link rel="modulepreload" href="/docs/inference-endpoints/pr_113/en/_app/immutable/entry/start.d1c14968.js">
		<link rel="modulepreload" href="/docs/inference-endpoints/pr_113/en/_app/immutable/chunks/scheduler.389d799c.js">
		<link rel="modulepreload" href="/docs/inference-endpoints/pr_113/en/_app/immutable/chunks/singletons.16c9b508.js">
		<link rel="modulepreload" href="/docs/inference-endpoints/pr_113/en/_app/immutable/chunks/paths.58d119e0.js">
		<link rel="modulepreload" href="/docs/inference-endpoints/pr_113/en/_app/immutable/entry/app.18050d92.js">
		<link rel="modulepreload" href="/docs/inference-endpoints/pr_113/en/_app/immutable/chunks/index.8f81d18f.js">
		<link rel="modulepreload" href="/docs/inference-endpoints/pr_113/en/_app/immutable/nodes/0.ce016c16.js">
		<link rel="modulepreload" href="/docs/inference-endpoints/pr_113/en/_app/immutable/nodes/10.fc2f459f.js">
		<link rel="modulepreload" href="/docs/inference-endpoints/pr_113/en/_app/immutable/chunks/CodeBlock.c0898180.js">
		<link rel="modulepreload" href="/docs/inference-endpoints/pr_113/en/_app/immutable/chunks/getInferenceSnippets.8efa8e08.js"><!-- HEAD_svelte-u9bgzb_START --><meta name="hf:doc:metadata" content="{&quot;title&quot;:&quot;Add custom Dependencies&quot;,&quot;local&quot;:&quot;add-custom-dependencies&quot;,&quot;sections&quot;:[],&quot;depth&quot;:1}"><!-- HEAD_svelte-u9bgzb_END -->      <p></p>  <h1 class="relative group"><a id="add-custom-dependencies" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#add-custom-dependencies"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>Add custom Dependencies</span></h1> <p data-svelte-h="svelte-1otnpic">Inference Endpoints’ base image includes all required libraries to run inference on Transformers models, but it also supports custom dependencies. This is useful if you want to:</p> <ul data-svelte-h="svelte-uzwec3"><li><a href="/docs/inference-endpoints/guides/custom_handler">customize your inference pipeline</a> and need additional Python dependencies</li> <li>run a model which requires special dependencies like the newest or a fixed version of a library (for example, <code>tapas</code> (<code>torch-scatter</code>)).</li></ul> <p data-svelte-h="svelte-10pf3qn">To add custom dependencies, add a <code>requirements.txt</code> <a href="https://huggingface.co/philschmid/distilbert-onnx-banking77/blob/main/requirements.txt" rel="nofollow">file</a> with the Python dependencies you want to install in your model repository on the Hugging Face Hub. When your Endpoint and Image artifacts are created, Inference Endpoints checks if the model repository contains a <code>requirements.txt</code> file and installs the dependencies listed within.</p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5   text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START -->optimum[onnxruntime]==1.2.3
mkl-include
mkl<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-d1f339">Check out the <code>requirements.txt</code> files in the following model repositories for examples:</p> <ul data-svelte-h="svelte-1x07diw"><li><a href="https://huggingface.co/philschmid/distilbert-onnx-banking77/blob/main/requirements.txt" rel="nofollow">Optimum and onnxruntime</a></li> <li><a href="https://huggingface.co/philschmid/stable-diffusion-v1-4-endpoints/blob/main/requirements.txt" rel="nofollow">diffusers</a></li></ul> <p data-svelte-h="svelte-19c75i9">For more information, take a look at how you can create and install dependencies when you <a href="/docs/inference-endpoints/guides/custom_container">use your own custom container</a> for inference.</p> <a class="!text-gray-400 !no-underline text-sm flex items-center not-prose mt-4" href="https://github.com/huggingface/hf-endpoints-documentation/blob/main/docs/source/guides/custom_dependencies.mdx" target="_blank"><span data-svelte-h="svelte-1kd6by1">&lt;</span> <span data-svelte-h="svelte-x0xyl0">&gt;</span> <span data-svelte-h="svelte-1dajgef"><span class="underline ml-1.5">Update</span> on GitHub</span></a>  <p></p> 
			
			<script>
				{
					__sveltekit_87vzq7 = {
						assets: "/docs/inference-endpoints/pr_113/en",
						base: "/docs/inference-endpoints/pr_113/en",
						env: {}
					};

					const element = document.currentScript.parentElement;

					const data = [null,null];

					Promise.all([
						import("/docs/inference-endpoints/pr_113/en/_app/immutable/entry/start.d1c14968.js"),
						import("/docs/inference-endpoints/pr_113/en/_app/immutable/entry/app.18050d92.js")
					]).then(([kit, app]) => {
						kit.start(app, element, {
							node_ids: [0, 10],
							data,
							form: null,
							error: null
						});
					});
				}
			</script>