Hugging Face's logo

Buckets:
hf-doc-build
/
doc-dev

Files
xet
hf-doc-build/doc-dev / kernels /pr_653 /en /_app /immutable /nodes /30.6bd5d2b3.js
HuggingFaceDocBuilder's picture
HuggingFaceDocBuilder
download
raw
45.1 kB
import{s as Wt,n as Gt,o as St}from"../chunks/scheduler.f3b1e791.js";import{S as Qt,i as Ht,e as a,s as n,c as y,h as Lt,a as i,d as t,b as M,f as Zt,g as o,j as r,k as Dl,l as Vt,m as s,n as p,t as T,o as U,p as J}from"../chunks/index.023a9934.js";import{C as xt}from"../chunks/CopyLLMTxtMenu.260ef33c.js";import{C as w}from"../chunks/CodeBlock.f5a2404f.js";import{H as c,E as Xt}from"../chunks/MermaidChart.svelte_svelte_type_style_lang.07a4ec62.js";function zt(et){let m,Be,ve,Ze,d,We,j,Ge,C,lt=`A kernel can provide layers in addition to kernel functions. A layer from
the Hub can replace the <code>forward</code> method of an existing layer for a certain
device type. This makes it possible to provide more performant kernels for
existing layers.`,Se,I,tt=`See <a href="kernel-requirements">Kernel requirements</a> for more information on the
requirements of Hub layers.`,Qe,f,He,h,Le,A,st=`A layer can be made extensible with the <a href="/docs/kernels/pr_653/en/api/layers#kernels.use_kernel_forward_from_hub">use_kernel_forward_from_hub()</a>
decorator. For example:`,Ve,b,xe,k,nt=`The decorator does not change the behavior of the class — it annotates
the class with the given name (here <code>SiluAndMul</code>). The <a href="/docs/kernels/pr_653/en/api/layers#kernels.kernelize">kernelize()</a> function
described below uses this name to look up kernels for the layer.`,Xe,g,ze,$,Mt=`An existing layer that does not (yet) have the <a href="/docs/kernels/pr_653/en/api/layers#kernels.use_kernel_forward_from_hub">use_kernel_forward_from_hub()</a>
decorator can be made extensible using the <a href="/docs/kernels/pr_653/en/api/layers#kernels.replace_kernel_forward_from_hub">replace_kernel_forward_from_hub()</a>
function:`,Fe,_,qe,E,at=`<strong>Warning:</strong> we strongly recommend using layers with a decorator, since
it signifies that the maintainer intends to keep the <code>forward</code> signature
compatible with layers from the hub.`,Ye,R,Pe,v,it=`Sometimes it can be useful to make a function extensible, for example
because the function cannot be replaced by a layer. In such cases, you
can annotate the function with the <a href="/docs/kernels/pr_653/en/api/layers#kernels.use_kernel_func_from_hub">use_kernel_func_from_hub()</a> decorator:`,Ke,N,Oe,B,rt=`This will replace the function by an instantiated <code>torch.nn.Module</code>
(singleton) that calls the function itself in its forward method.`,De,Z,yt=`<strong>Note:</strong> for kernelization to see the function, it must be a member of
another <code>torch.nn.Module</code> that is part of the model. For example:`,el,W,ll,G,tl,S,ot=`A model will not use Hub kernels by default, even if it contains extensible
layers. To enable the use of Hub kernels in the model, it needs to be
‘kernelized’ using the <a href="/docs/kernels/pr_653/en/api/layers#kernels.kernelize">kernelize()</a> function. This function traverses the
model graph and replaces the <code>forward</code> methods of extensible layers for which
Hub kernels are registered. <a href="/docs/kernels/pr_653/en/api/layers#kernels.kernelize">kernelize()</a> can be used as follows:`,sl,Q,nl,H,pt=`The <a href="/docs/kernels/pr_653/en/api/layers#kernels.kernelize">kernelize()</a> function modifies the model in-place, the model itself is
returned as a convenience. The <code>mode</code> specifies that the model will be used
in inference. Similarly, you can ask <a href="/docs/kernels/pr_653/en/api/layers#kernels.kernelize">kernelize()</a> to prepare the model for
training:`,Ml,L,al,V,Tt=`A model that is kernelized for training can also be used for inference, but
not the other way around. If you want to change the mode of the kernelized
model, you can just run <a href="/docs/kernels/pr_653/en/api/layers#kernels.kernelize">kernelize()</a> on the model again with the new mode.`,il,x,Ut=`If you want to compile a model with <code>torch.compile</code>, this should be indicated
in the mode as well. You can do this by combining <code>Mode.INFERENCE</code> or
<code>Mode.TRAINING</code> with <code>Mode.TORCH_COMPILE</code> using the set union (<code>|</code>) operator:`,rl,X,yl,z,ol,F,Jt=`Kernels can be registered per device type. For instance, separate <code>cuda</code> and
<code>metal</code> kernels could be registered for the name <code>SiluAndMul</code>. By default,
<a href="/docs/kernels/pr_653/en/api/layers#kernels.kernelize">kernelize()</a> will try to infer the device type from the model’s parameters.
You can pass the device type to <a href="/docs/kernels/pr_653/en/api/layers#kernels.kernelize">kernelize()</a> if the device type cannot be
inferred (e.g. because the model has no parameters):`,pl,q,Tl,Y,Ul,P,wt=`If the <code>TRAINING</code> and/or <code>TORCH_COMPILE</code> modes are used, but a registered
kernel does not support backward passes or <code>torch.compile</code> respectively,
<a href="/docs/kernels/pr_653/en/api/layers#kernels.kernelize">kernelize()</a> will fall back to the original, non-kernelized, layer. You
can let <a href="/docs/kernels/pr_653/en/api/layers#kernels.kernelize">kernelize()</a> raise an exception instead by using <code>use_fallback=False</code>:`,Jl,K,wl,O,ct="This can be useful if you want to guarantee that Hub kernels are used.",cl,D,ml,ee,mt=`The kernels that are used are logged at the <code>INFO</code> level by <a href="/docs/kernels/pr_653/en/api/layers#kernels.kernelize">kernelize()</a>.
See the <a href="https://docs.python.org/3/library/logging.html" rel="nofollow">Python logging</a>
documentation for information on how to configure logging.`,ul,le,dl,te,ut=`<a href="/docs/kernels/pr_653/en/api/layers#kernels.kernelize">kernelize()</a> relies on kernel mappings to find Hub kernels for layers.
Kernel mappings map a kernel name such as <code>SiluAndMul</code> to a kernel on
the Hub. For example:`,jl,se,Cl,ne,dt=`This uses version <code>1</code> of the <code>SiluAndMul</code> kernel layer from
<code>kernels-community/activation</code> for the <code>cuda</code> and <code>rocm</code> backends. Kernel
layers are versioned using a major version number. Using <code>version=1</code>
will get the latest kernel build from the <code>v1</code> branch. Kernel layers
within a version branch must never break the API or remove builds for
older PyTorch versions. This ensures that your code will continue to
work.
Hub-backed <a href="/docs/kernels/pr_653/en/api/layers#kernels.LayerRepository">LayerRepository</a> and <a href="/docs/kernels/pr_653/en/api/layers#kernels.FuncRepository">FuncRepository</a> entries must specify
either a <code>version</code> or an explicit <code>revision</code>.`,Il,u,jt=`<p>Version <code>0</code> kernels are excluded from the API compatibility requirement,
since it is used for alpha/beta-quality kernels that may still have
rapidly changing APIs.</p>`,fl,Me,Ct='You can register a mapping, like the one above, using <a href="/docs/kernels/pr_653/en/api/layers#kernels.register_kernel_mapping">register_kernel_mapping()</a>:',hl,ae,Al,ie,It=`This will register the kernel mapping in the current context, which is
normally global. It is recommended to scope the mapping to where it is
used with the <a href="/docs/kernels/pr_653/en/api/layers#kernels.use_kernel_mapping">use_kernel_mapping()</a> context manager:`,bl,re,kl,ye,ft=`This ensures that the mapping is not active anymore outside the
<code>with</code>-scope.`,gl,oe,ht=`If the layer is stateless (it does not use member variables in its forward <em>or</em> it was
originally a function that was converted into a kernel layer with
<a href="/docs/kernels/pr_653/en/api/layers#kernels.use_kernel_func_from_hub">use_kernel_func_from_hub()</a>), it can also be mapped to a kernel function:`,$l,pe,_l,Te,El,Ue,At=`You might want to register two different kernels for a particular layer,
where one kernel is optimized for a specific mode. You can do so by
registering layer repositories for specific modes. For example:`,Rl,Je,vl,we,bt=`The <a href="/docs/kernels/pr_653/en/api/layers#kernels.kernelize">kernelize()</a> function will attempt to use the following registered
kernels for a given mode:`,Nl,ce,kt=`<li><code>INFERENCE</code>: <code>INFERENCE</code> → <code>INFERENCE | TORCH_COMPILE</code> → <code>TRAINING</code> →
<code>TRAINING | TORCH_COMPILE</code> → <code>FALLBACK</code></li> <li><code>INFERENCE | TORCH_COMPILE</code>: <code>INFERENCE | TORCH_COMPILE</code> →
<code>TRAINING | TORCH_COMPILE</code> → <code>FALLBACK</code></li> <li><code>TRAINING</code>: <code>TRAINING</code> → <code>TRAINING | TORCH_COMPILE</code> → <code>FALLBACK</code></li> <li><code>TRAINING | TORCH_COMPILE</code>: <code>TRAINING | TORCH_COMPILE</code> → <code>FALLBACK</code></li>`,Bl,me,gt=`<code>Mode.FALLBACK</code> is a special mode that is used when no other mode matches. It
is also used when a kernel is registered without a mode, as described in the
previous section.`,Zl,ue,Wl,de,$t=`In this case, both <code>Mode.INFERENCE | Mode.TORCH_COMPILE</code> and
<code>Mode.TRAINING | Mode.TORCH_COMPILE</code> will use the <code>Mode.FALLBACK</code> kernel,
since the other kernels do not support <code>torch.compile</code>.`,Gl,je,Sl,Ce,_t=`Some kernels only work with newer CUDA architectures. For instance, some
kernels require capability 9.0 for the TMA unit on Hopper GPUs. <code>kernels</code>
supports registering layers for a range of CUDA capabilities. To do so,
you need to register the layer for a <a href="/docs/kernels/pr_653/en/api/layers#kernels.Device">Device</a> with type <code>cuda</code> and
set the supported range of CUDA capabilities with using <code>CUDAProperties</code>:`,Ql,Ie,Hl,fe,Et="Capabilities behave as follows:",Ll,he,Rt=`<li><p>The minimum and maximum capabilities are inclusive.</p></li> <li><p>When a new kernel is registered with the same min/max capabilities as
an existing kernel, the new kernel will replace the old kernel.</p></li> <li><p>When there are multiple kernels that support a capability, the kernel
with the smaller capability interval will be used. E.g. given:</p> <ul><li><code>KernelA</code> with <code>min_capability=80</code> and <code>max_capability=89</code>;</li> <li><code>KernelB</code> with <code>min_capability=75</code> and <code>max_capability=89</code>;</li> <li><a href="/docs/kernels/pr_653/en/api/layers#kernels.kernelize">kernelize()</a> runs on a system with capability 8.6.</li></ul> <p>Then <code>KernelA</code> will be used because the interval 80..89 is smaller
than 75..89. The motivation is that kernels with smaller ranges
tend to be more optimized for a specific set of GPUs. <strong>This behavior
might still change in the future.</strong></p></li>`,Vl,Ae,xl,be,vt=`Registering kernels for the ROCm architecture follows the exact same
pattern as CUDA kernels, using <code>min_capability</code> and <code>max_capability</code> to restrict
a kernel to a range of ROCm capabilities.`,Xl,ke,zl,ge,Nt=`The <a href="/docs/kernels/pr_653/en/api/layers#kernels.LocalLayerRepository">LocalLayerRepository</a> class is provided to load a repository from
a local directory. For example:`,Fl,$e,ql,_e,Bt=`Similarly, the <a href="/docs/kernels/pr_653/en/api/layers#kernels.LocalFuncRepository">LocalFuncRepository</a> class can be used to load a kernel
function from a local directory:`,Yl,Ee,Pl,Re,Kl,Ne,Ol;return d=new xt({props:{containerStyle:"float: right; margin-left: 10px; display: inline-flex; position: relative; z-index: 10;"}}),j=new c({props:{title:"Layers",local:"layers",headingTag:"h1"}}),f=new c({props:{title:"Making a layer extensible with kernels from the hub",local:"making-a-layer-extensible-with-kernels-from-the-hub",headingTag:"h2"}}),h=new c({props:{title:"Using a decorator",local:"using-a-decorator",headingTag:"h3"}}),b=new w({props:{code:"JTQwdXNlX2tlcm5lbF9mb3J3YXJkX2Zyb21faHViKCUyMlNpbHVBbmRNdWwlMjIpJTBBY2xhc3MlMjBTaWx1QW5kTXVsKG5uLk1vZHVsZSklM0ElMEElMjAlMjAlMjAlMjBkZWYlMjBmb3J3YXJkKHNlbGYlMkMlMjBpbnB1dCUzQSUyMHRvcmNoLlRlbnNvciklMjAtJTNFJTIwdG9yY2guVGVuc29yJTNBJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwZCUyMCUzRCUyMGlucHV0LnNoYXBlJTVCLTElNUQlMjAlMkYlMkYlMjAyJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwcmV0dXJuJTIwRi5zaWx1KGlucHV0JTVCLi4uJTJDJTIwJTNBZCU1RCklMjAqJTIwaW5wdXQlNUIuLi4lMkMlMjBkJTNBJTVE",highlighted:`<span class="hljs-meta">@use_kernel_forward_from_hub(<span class="hljs-params"><span class="hljs-string">&quot;SiluAndMul&quot;</span></span>)</span>
<span class="hljs-keyword">class</span> <span class="hljs-title class_">SiluAndMul</span>(nn.Module):
 <span class="hljs-keyword">def</span> <span class="hljs-title function_">forward</span>(<span class="hljs-params">self, <span class="hljs-built_in">input</span>: torch.Tensor</span>) -&gt; torch.Tensor:
 d = <span class="hljs-built_in">input</span>.shape[-<span class="hljs-number">1</span>] // <span class="hljs-number">2</span>
 <span class="hljs-keyword">return</span> F.silu(<span class="hljs-built_in">input</span>[..., :d]) * <span class="hljs-built_in">input</span>[..., d:]`,lang:"python",wrap:!1}}),g=new c({props:{title:"External layers",local:"external-layers",headingTag:"h3"}}),_=new w({props:{code:"ZnJvbSUyMHNvbWVsaWJyYXJ5JTIwaW1wb3J0JTIwU2lsdUFuZE11bCUwQSUwQXJlcGxhY2Vfa2VybmVsX2ZvcndhcmRfZnJvbV9odWIoU2lsdUFuZE11bCUyQyUyMCUyMlNpbHVBbmRNdWwlMjIp",highlighted:`<span class="hljs-keyword">from</span> somelibrary <span class="hljs-keyword">import</span> SiluAndMul
replace_kernel_forward_from_hub(SiluAndMul, <span class="hljs-string">&quot;SiluAndMul&quot;</span>)`,lang:"python",wrap:!1}}),R=new c({props:{title:"Using a function as a layer",local:"using-a-function-as-a-layer",headingTag:"h3"}}),N=new w({props:{code:"JTQwdXNlX2tlcm5lbF9mdW5jX2Zyb21faHViKCUyMnNpbHVfYW5kX211bCUyMiklMEFkZWYlMjBzaWx1X2FuZF9tdWwoeCUzQSUyMHRvcmNoLlRlbnNvciklMjAtJTNFJTIwdG9yY2guVGVuc29yJTNBJTBBJTIwJTIwJTIwJTIwZCUyMCUzRCUyMHguc2hhcGUlNUItMSU1RCUyMCUyRiUyRiUyMDIlMEElMjAlMjAlMjAlMjByZXR1cm4lMjBGLnNpbHUoeCU1Qi4uLiUyQyUyMCUzQWQlNUQpJTIwKiUyMHglNUIuLi4lMkMlMjBkJTNBJTVE",highlighted:`<span class="hljs-meta">@use_kernel_func_from_hub(<span class="hljs-params"><span class="hljs-string">&quot;silu_and_mul&quot;</span></span>)</span>
<span class="hljs-keyword">def</span> <span class="hljs-title function_">silu_and_mul</span>(<span class="hljs-params">x: torch.Tensor</span>) -&gt; torch.Tensor:
 d = x.shape[-<span class="hljs-number">1</span>] // <span class="hljs-number">2</span>
 <span class="hljs-keyword">return</span> F.silu(x[..., :d]) * x[..., d:]`,lang:"python",wrap:!1}}),W=new w({props:{code:"Y2xhc3MlMjBGZWVkRm9yd2FyZChubi5Nb2R1bGUpJTNBJTBBJTIwJTIwZGVmJTIwX19pbml0X18oc2VsZiUyQyUyMGluX2ZlYXR1cmVzJTNBJTIwaW50JTJDJTIwb3V0X2ZlYXR1cmVzJTNBJTIwaW50KSUzQSUwQSUyMCUyMCUyMCUyMCUyMCUyMHNlbGYubGluZWFyJTIwJTNEJTIwbm4uTGluZWFyKGluX2ZlYXR1cmVzJTJDJTIwb3V0X2ZlYXR1cmVzKSUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMyUyME5vdGUlM0ElMjBzaWx1X2FuZF9tdWwlMjBpcyUyMGElMjBUb3JjaCUyMG1vZHVsZS4lMEElMjAlMjAlMjAlMjAlMjAlMjBzZWxmLnNpbHVfYW5kX211bCUyMCUzRCUyMHNpbHVfYW5kX211bCUwQSUwQSUyMCUyMGRlZiUyMGZvcndhcmQoc2VsZiUyQyUyMHglM0ElMjB0b3JjaC5UZW5zb3IpJTIwLSUzRSUyMHRvcmNoLlRlbnNvciUzQSUwQSUyMCUyMCUyMCUyMCUyMCUyMHJldHVybiUyMHNlbGYuc2lsdV9hbmRfbXVsKHNlbGYubGluZWFyKHgpKQ==",highlighted:`<span class="hljs-keyword">class</span> <span class="hljs-title class_">FeedForward</span>(nn.Module):
 <span class="hljs-keyword">def</span> <span class="hljs-title function_">__init__</span>(<span class="hljs-params">self, in_features: <span class="hljs-built_in">int</span>, out_features: <span class="hljs-built_in">int</span></span>):
 self.linear = nn.Linear(in_features, out_features)
 <span class="hljs-comment"># Note: silu_and_mul is a Torch module.</span>
 self.silu_and_mul = silu_and_mul
 <span class="hljs-keyword">def</span> <span class="hljs-title function_">forward</span>(<span class="hljs-params">self, x: torch.Tensor</span>) -&gt; torch.Tensor:
 <span class="hljs-keyword">return</span> self.silu_and_mul(self.linear(x))`,lang:"python",wrap:!1}}),G=new c({props:{title:"Kernelizing a model",local:"kernelizing-a-model",headingTag:"h2"}}),Q=new w({props:{code:"bW9kZWwlMjAlM0QlMjBNeU1vZGVsKC4uLiklMEFtb2RlbCUyMCUzRCUyMGtlcm5lbGl6ZShtb2RlbCUyQyUyMG1vZGUlM0RNb2RlLklORkVSRU5DRSk=",highlighted:`model = MyModel(...)
model = kernelize(model, mode=Mode.INFERENCE)`,lang:"python",wrap:!1}}),L=new w({props:{code:"bW9kZWwlMjAlM0QlMjBNeU1vZGVsKC4uLiklMEFtb2RlbCUyMCUzRCUyMGtlcm5lbGl6ZShtb2RlbCUyQyUyMG1vZGUlM0RNb2RlLlRSQUlOSU5HKQ==",highlighted:`model = MyModel(...)
model = kernelize(model, mode=Mode.TRAINING)`,lang:"python",wrap:!1}}),X=new w({props:{code:"bW9kZWwlMjAlM0QlMjBNeU1vZGVsKC4uLiklMEElMEElMjMlMjBJbmZlcmVuY2UlMEFtb2RlbCUyMCUzRCUyMGtlcm5lbGl6ZShtb2RlbCUyQyUyMG1vZGUlM0RNb2RlLklORkVSRU5DRSUyMCU3QyUyME1vZGUuVE9SQ0hfQ09NUElMRSklMEElMEElMjMlMjBUcmFpbmluZyUwQW1vZGVsJTIwJTNEJTIwa2VybmVsaXplKG1vZGVsJTJDJTIwbW9kZSUzRE1vZGUuVFJBSU5JTkclMjAlN0MlMjBNb2RlLlRPUkNIX0NPTVBJTEUp",highlighted:`model = MyModel(...)
<span class="hljs-comment"># Inference</span>
model = kernelize(model, mode=Mode.INFERENCE | Mode.TORCH_COMPILE)
<span class="hljs-comment"># Training</span>
model = kernelize(model, mode=Mode.TRAINING | Mode.TORCH_COMPILE)`,lang:"python",wrap:!1}}),z=new c({props:{title:"Kernel device",local:"kernel-device",headingTag:"h3"}}),q=new w({props:{code:"bW9kZWwlMjAlM0QlMjBNeU1vZGVsKC4uLiklMEFtb2RlbCUyMCUzRCUyMGtlcm5lbGl6ZShtb2RlbCUyQyUyMGRldmljZSUzRCUyMmN1ZGElMjIlMkMlMjBtb2RlJTNETW9kZS5JTkZFUkVOQ0Up",highlighted:`model = MyModel(...)
model = kernelize(model, device=<span class="hljs-string">&quot;cuda&quot;</span>, mode=Mode.INFERENCE)`,lang:"python",wrap:!1}}),Y=new c({props:{title:"Fallback forward",local:"fallback-forward",headingTag:"h3"}}),K=new w({props:{code:"bW9kZWwlMjAlM0QlMjBNeU1vZGVsKC4uLiklMEFtb2RlbCUyMCUzRCUyMGtlcm5lbGl6ZShtb2RlbCUyQyUyMG1vZGUlM0RNb2RlLklORkVSRU5DRSUyMCU3QyUyME1vZGUuVE9SQ0hfQ09NUElMRSUyQyUyMHVzZV9mYWxsYmFjayUzREZhbHNlKQ==",highlighted:`model = MyModel(...)
model = kernelize(model, mode=Mode.INFERENCE | Mode.TORCH_COMPILE, use_fallback=<span class="hljs-literal">False</span>)`,lang:"python",wrap:!1}}),D=new c({props:{title:"Inspecting which kernels are used",local:"inspecting-which-kernels-are-used",headingTag:"h3"}}),le=new c({props:{title:"Registering a hub kernel for a layer",local:"registering-a-hub-kernel-for-a-layer",headingTag:"h2"}}),se=new w({props:{code:"a2VybmVsX2xheWVyX21hcHBpbmclMjAlM0QlMjAlN0IlMEElMjAlMjAlMjAlMjAlMjJTaWx1QW5kTXVsJTIyJTNBJTIwJTdCJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIyY3VkYSUyMiUzQSUyMExheWVyUmVwb3NpdG9yeSglMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjByZXBvX2lkJTNEJTIya2VybmVscy1jb21tdW5pdHklMkZhY3RpdmF0aW9uJTIyJTJDJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwbGF5ZXJfbmFtZSUzRCUyMlNpbHVBbmRNdWwlMjIlMkMlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjB2ZXJzaW9uJTNEMSUyQyUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCklMkMlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjJyb2NtJTIyJTNBJTIwTGF5ZXJSZXBvc2l0b3J5KCUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMHJlcG9faWQlM0QlMjJrZXJuZWxzLWNvbW11bml0eSUyRmFjdGl2YXRpb24lMjIlMkMlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjBsYXllcl9uYW1lJTNEJTIyU2lsdUFuZE11bCUyMiUyQyUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMHZlcnNpb24lM0QxJTJDJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwKSUwQSUyMCUyMCUyMCUyMCU3RCUwQSU3RA==",highlighted:`kernel_layer_mapping = {
 <span class="hljs-string">&quot;SiluAndMul&quot;</span>: {
 <span class="hljs-string">&quot;cuda&quot;</span>: LayerRepository(
 repo_id=<span class="hljs-string">&quot;kernels-community/activation&quot;</span>,
 layer_name=<span class="hljs-string">&quot;SiluAndMul&quot;</span>,
 version=<span class="hljs-number">1</span>,
 ),
 <span class="hljs-string">&quot;rocm&quot;</span>: LayerRepository(
 repo_id=<span class="hljs-string">&quot;kernels-community/activation&quot;</span>,
 layer_name=<span class="hljs-string">&quot;SiluAndMul&quot;</span>,
 version=<span class="hljs-number">1</span>,
 )
 }
}`,lang:"python",wrap:!1}}),ae=new w({props:{code:"cmVnaXN0ZXJfa2VybmVsX21hcHBpbmcoa2VybmVsX2xheWVyX21hcHBpbmcp",highlighted:"register_kernel_mapping(kernel_layer_mapping)",lang:"python",wrap:!1}}),re=new w({props:{code:"d2l0aCUyMHVzZV9rZXJuZWxfbWFwcGluZyhrZXJuZWxfbGF5ZXJfbWFwcGluZyklM0ElMEElMjAlMjAlMjAlMjAlMjMlMjBVc2UlMjB0aGUlMjBsYXllciUyMGZvciUyMHdoaWNoJTIwdGhlJTIwbWFwcGluZyUyMGlzJTIwYXBwbGllZC4lMEElMjAlMjAlMjAlMjBtb2RlbCUyMCUzRCUyMGtlcm5lbGl6ZShtb2RlbCUyQyUyMG1vZGUlM0RNb2RlLlRSQUlOSU5HJTIwJTdDJTIwTW9kZS5UT1JDSF9DT01QSUxFKQ==",highlighted:`<span class="hljs-keyword">with</span> use_kernel_mapping(kernel_layer_mapping):
 <span class="hljs-comment"># Use the layer for which the mapping is applied.</span>
 model = kernelize(model, mode=Mode.TRAINING | Mode.TORCH_COMPILE)`,lang:"python",wrap:!1}}),pe=new w({props:{code:"a2VybmVsX2xheWVyX21hcHBpbmclMjAlM0QlMjAlN0IlMEElMjAlMjAlMjAlMjAlMjJTaWx1QW5kTXVsJTIyJTNBJTIwJTdCJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIyY3VkYSUyMiUzQSUyMEZ1bmNSZXBvc2l0b3J5KCUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMHJlcG9faWQlM0QlMjJrZXJuZWxzLWNvbW11bml0eSUyRmFjdGl2YXRpb24lMjIlMkMlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjBmdW5jX25hbWUlM0QlMjJzaWx1X2FuZF9tdWwlMjIlMkMlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjB2ZXJzaW9uJTNEMSUyQyUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCklMkMlMEElMjAlMjAlMjAlMjAlN0QlMEElN0Q=",highlighted:`kernel_layer_mapping = {
 <span class="hljs-string">&quot;SiluAndMul&quot;</span>: {
 <span class="hljs-string">&quot;cuda&quot;</span>: FuncRepository(
 repo_id=<span class="hljs-string">&quot;kernels-community/activation&quot;</span>,
 func_name=<span class="hljs-string">&quot;silu_and_mul&quot;</span>,
 version=<span class="hljs-number">1</span>,
 ),
 }
}`,lang:"python",wrap:!1}}),Te=new c({props:{title:"Registering kernels for specific modes",local:"registering-kernels-for-specific-modes",headingTag:"h3"}}),Je=new w({props:{code:"a2VybmVsX2xheWVyX21hcHBpbmclMjAlM0QlMjAlN0IlMEElMjAlMjAlMjAlMjAlMjJTaWx1QW5kTXVsJTIyJTNBJTIwJTdCJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIyY3VkYSUyMiUzQSUyMCU3QiUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyME1vZGUuSU5GRVJFTkNFJTNBJTIwTGF5ZXJSZXBvc2l0b3J5KCUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMHJlcG9faWQlM0QlMjJrZXJuZWxzLWNvbW11bml0eSUyRmFjdGl2YXRpb24taW5mZXJlbmNlLW9wdGltaXplZCUyMiUyQyUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMGxheWVyX25hbWUlM0QlMjJTaWx1QW5kTXVsJTIyJTJDJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwdmVyc2lvbiUzRDElMkMlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjApJTJDJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwTW9kZS5UUkFJTklORyUyMCU3QyUyME1vZGUuVE9SQ0hfQ09NUElMRSUzQSUyMExheWVyUmVwb3NpdG9yeSglMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjByZXBvX2lkJTNEJTIya2VybmVscy1jb21tdW5pdHklMkZhY3RpdmF0aW9uLXRyYWluaW5nLW9wdGltaXplZCUyMiUyQyUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMGxheWVyX25hbWUlM0QlMjJTaWx1QW5kTXVsJTIyJTJDJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwdmVyc2lvbiUzRDElMkMlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjApJTJDJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTdEJTBBJTIwJTIwJTIwJTIwJTdEJTBBJTdE",highlighted:`kernel_layer_mapping = {
 <span class="hljs-string">&quot;SiluAndMul&quot;</span>: {
 <span class="hljs-string">&quot;cuda&quot;</span>: {
 Mode.INFERENCE: LayerRepository(
 repo_id=<span class="hljs-string">&quot;kernels-community/activation-inference-optimized&quot;</span>,
 layer_name=<span class="hljs-string">&quot;SiluAndMul&quot;</span>,
 version=<span class="hljs-number">1</span>,
 ),
 Mode.TRAINING | Mode.TORCH_COMPILE: LayerRepository(
 repo_id=<span class="hljs-string">&quot;kernels-community/activation-training-optimized&quot;</span>,
 layer_name=<span class="hljs-string">&quot;SiluAndMul&quot;</span>,
 version=<span class="hljs-number">1</span>,
 ),
 }
 }
}`,lang:"python",wrap:!1}}),ue=new w({props:{code:"a2VybmVsX2xheWVyX21hcHBpbmclMjAlM0QlMjAlN0IlMEElMjAlMjAlMjAlMjAlMjJTaWx1QW5kTXVsJTIyJTNBJTIwJTdCJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIyY3VkYSUyMiUzQSUyMCU3QiUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyME1vZGUuRkFMTEJBQ0slM0ElMjBMYXllclJlcG9zaXRvcnkoJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwcmVwb19pZCUzRCUyMmtlcm5lbHMtY29tbXVuaXR5JTJGYWN0aXZhdGlvbiUyMiUyQyUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMGxheWVyX25hbWUlM0QlMjJTaWx1QW5kTXVsJTIyJTJDJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwdmVyc2lvbiUzRDElMkMlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjApJTJDJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwTW9kZS5JTkZFUkVOQ0UlM0ElMjBMYXllclJlcG9zaXRvcnkoJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwcmVwb19pZCUzRCUyMmtlcm5lbHMtY29tbXVuaXR5JTJGYWN0aXZhdGlvbi1pbmZlcmVuY2Utb3B0aW1pemVkJTIyJTJDJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwbGF5ZXJfbmFtZSUzRCUyMlNpbHVBbmRNdWwlMjIlMkMlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjB2ZXJzaW9uJTNEMSUyQyUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCklMkMlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjBNb2RlLlRSQUlOSU5HJTNBJTIwTGF5ZXJSZXBvc2l0b3J5KCUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMHJlcG9faWQlM0QlMjJrZXJuZWxzLWNvbW11bml0eSUyRmFjdGl2YXRpb24tdHJhaW5pbmctb3B0aW1pemVkJTIyJTJDJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwbGF5ZXJfbmFtZSUzRCUyMlNpbHVBbmRNdWwlMjIlMkMlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjB2ZXJzaW9uJTNEMSUyQyUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCklMkMlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlN0QlMEElMjAlMjAlMjAlMjAlN0QlMEElN0Q=",highlighted:`kernel_layer_mapping = {
 <span class="hljs-string">&quot;SiluAndMul&quot;</span>: {
 <span class="hljs-string">&quot;cuda&quot;</span>: {
 Mode.FALLBACK: LayerRepository(
 repo_id=<span class="hljs-string">&quot;kernels-community/activation&quot;</span>,
 layer_name=<span class="hljs-string">&quot;SiluAndMul&quot;</span>,
 version=<span class="hljs-number">1</span>,
 ),
 Mode.INFERENCE: LayerRepository(
 repo_id=<span class="hljs-string">&quot;kernels-community/activation-inference-optimized&quot;</span>,
 layer_name=<span class="hljs-string">&quot;SiluAndMul&quot;</span>,
 version=<span class="hljs-number">1</span>,
 ),
 Mode.TRAINING: LayerRepository(
 repo_id=<span class="hljs-string">&quot;kernels-community/activation-training-optimized&quot;</span>,
 layer_name=<span class="hljs-string">&quot;SiluAndMul&quot;</span>,
 version=<span class="hljs-number">1</span>,
 ),
 }
 }
}`,lang:"python",wrap:!1}}),je=new c({props:{title:"Registering kernels for specific CUDA capabilities",local:"registering-kernels-for-specific-cuda-capabilities",headingTag:"h3"}}),Ie=new w({props:{code:"a2VybmVsX2xheWVyX21hcHBpbmclMjAlM0QlMjAlN0IlMEElMjAlMjAlMjAlMjAlMjJTaWx1QW5kTXVsJTIyJTNBJTIwJTdCJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwRGV2aWNlKCUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMHR5cGUlM0QlMjJjdWRhJTIyJTJDJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwcHJvcGVydGllcyUzRENVREFQcm9wZXJ0aWVzKCUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMG1pbl9jYXBhYmlsaXR5JTNENzUlMkMlMjBtYXhfY2FwYWJpbGl0eSUzRDg5JTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwKSUyQyUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCklM0ElMjBMYXllclJlcG9zaXRvcnkoJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwcmVwb19pZCUzRCUyMmtlcm5lbHMtY29tbXVuaXR5JTJGYWN0aXZhdGlvbiUyMiUyQyUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMGxheWVyX25hbWUlM0QlMjJTaWx1QW5kTXVsJTIyJTJDJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwdmVyc2lvbiUzRDElMkMlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjApJTJDJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwRGV2aWNlKCUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMHR5cGUlM0QlMjJjdWRhJTIyJTJDJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwcHJvcGVydGllcyUzRENVREFQcm9wZXJ0aWVzKCUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMG1pbl9jYXBhYmlsaXR5JTNEOTAlMkMlMjBtYXhfY2FwYWJpbGl0eSUzRHN5cy5tYXhzaXplJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwKSUyQyUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCklM0ElMjBMYXllclJlcG9zaXRvcnkoJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwcmVwb19pZCUzRCUyMmtlcm5lbHMtY29tbXVuaXR5JTJGYWN0aXZhdGlvbi1ob3BwZXIlMjIlMkMlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjBsYXllcl9uYW1lJTNEJTIyU2lsdUFuZE11bCUyMiUyQyUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMHZlcnNpb24lM0QxJTJDJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwKSUyQyUwQSUyMCUyMCUyMCUyMCU3RCUwQSU3RA==",highlighted:`kernel_layer_mapping = {
 <span class="hljs-string">&quot;SiluAndMul&quot;</span>: {
 Device(
 <span class="hljs-built_in">type</span>=<span class="hljs-string">&quot;cuda&quot;</span>,
 properties=CUDAProperties(
 min_capability=<span class="hljs-number">75</span>, max_capability=<span class="hljs-number">89</span>
 ),
 ): LayerRepository(
 repo_id=<span class="hljs-string">&quot;kernels-community/activation&quot;</span>,
 layer_name=<span class="hljs-string">&quot;SiluAndMul&quot;</span>,
 version=<span class="hljs-number">1</span>,
 ),
 Device(
 <span class="hljs-built_in">type</span>=<span class="hljs-string">&quot;cuda&quot;</span>,
 properties=CUDAProperties(
 min_capability=<span class="hljs-number">90</span>, max_capability=sys.maxsize
 ),
 ): LayerRepository(
 repo_id=<span class="hljs-string">&quot;kernels-community/activation-hopper&quot;</span>,
 layer_name=<span class="hljs-string">&quot;SiluAndMul&quot;</span>,
 version=<span class="hljs-number">1</span>,
 ),
 }
}`,lang:"python",wrap:!1}}),Ae=new c({props:{title:"Registering kernels for specific ROCm capabilities",local:"registering-kernels-for-specific-rocm-capabilities",headingTag:"h3"}}),ke=new c({props:{title:"Loading from a local repository for testing",local:"loading-from-a-local-repository-for-testing",headingTag:"h3"}}),$e=new w({props:{code:"d2l0aCUyMHVzZV9rZXJuZWxfbWFwcGluZyglMEElMjAlMjAlMjAlMjAlN0IlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjJTaWx1QW5kTXVsJTIyJTNBJTIwJTdCJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIyY3VkYSUyMiUzQSUyMExvY2FsTGF5ZXJSZXBvc2l0b3J5KCUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMHJlcG9fcGF0aCUzRCUyMiUyRmhvbWUlMkZkYW5pZWwlMkZrZXJuZWxzJTJGYWN0aXZhdGlvbiUyMiUyQyUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMHBhY2thZ2VfbmFtZSUzRCUyMmFjdGl2YXRpb24lMjIlMkMlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjBsYXllcl9uYW1lJTNEJTIyU2lsdUFuZE11bCUyMiUyQyUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCklMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlN0QlMEElMjAlMjAlMjAlMjAlN0QlMkMlMEElMjAlMjAlMjAlMjBpbmhlcml0X21hcHBpbmclM0RGYWxzZSUyQyUwQSklM0ElMEElMjAlMjAlMjAlMjBrZXJuZWxpemUobGluZWFyJTJDJTIwbW9kZSUzRE1vZGUuSU5GRVJFTkNFKQ==",highlighted:`<span class="hljs-keyword">with</span> use_kernel_mapping(
 {
 <span class="hljs-string">&quot;SiluAndMul&quot;</span>: {
 <span class="hljs-string">&quot;cuda&quot;</span>: LocalLayerRepository(
 repo_path=<span class="hljs-string">&quot;/home/daniel/kernels/activation&quot;</span>,
 package_name=<span class="hljs-string">&quot;activation&quot;</span>,
 layer_name=<span class="hljs-string">&quot;SiluAndMul&quot;</span>,
 )
 }
 },
 inherit_mapping=<span class="hljs-literal">False</span>,
):
 kernelize(linear, mode=Mode.INFERENCE)`,lang:"python",wrap:!1}}),Ee=new w({props:{code:"d2l0aCUyMHVzZV9rZXJuZWxfbWFwcGluZyglMEElMjAlMjAlMjAlMjAlN0IlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjJzaWx1X2FuZF9tdWwlMjIlM0ElMjAlN0IlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjJjdWRhJTIyJTNBJTIwTG9jYWxGdW5jUmVwb3NpdG9yeSglMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjByZXBvX3BhdGglM0QlMjIlMkZob21lJTJGZGFuaWVsJTJGa2VybmVscyUyRmFjdGl2YXRpb24lMjIlMkMlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjBwYWNrYWdlX25hbWUlM0QlMjJhY3RpdmF0aW9uJTIyJTJDJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwZnVuY19uYW1lJTNEJTIyc2lsdV9hbmRfbXVsJTIyJTJDJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwKSUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCU3RCUwQSUyMCUyMCUyMCUyMCU3RCUyQyUwQSUyMCUyMCUyMCUyMGluaGVyaXRfbWFwcGluZyUzREZhbHNlJTJDJTBBKSUzQSUwQSUyMCUyMCUyMCUyMGtlcm5lbGl6ZShtb2RlbCUyQyUyMG1vZGUlM0RNb2RlLklORkVSRU5DRSk=",highlighted:`<span class="hljs-keyword">with</span> use_kernel_mapping(
 {
 <span class="hljs-string">&quot;silu_and_mul&quot;</span>: {
 <span class="hljs-string">&quot;cuda&quot;</span>: LocalFuncRepository(
 repo_path=<span class="hljs-string">&quot;/home/daniel/kernels/activation&quot;</span>,
 package_name=<span class="hljs-string">&quot;activation&quot;</span>,
 func_name=<span class="hljs-string">&quot;silu_and_mul&quot;</span>,
 )
 }
 },
 inherit_mapping=<span class="hljs-literal">False</span>,
):
 kernelize(model, mode=Mode.INFERENCE)`,lang:"python",wrap:!1}}),Re=new Xt({props:{source:"https://github.com/huggingface/kernels/blob/main/docs/source/layers.md"}}),{c(){m=a("meta"),Be=n(),ve=a("p"),Ze=n(),y(d.$$.fragment),We=n(),y(j.$$.fragment),Ge=n(),C=a("p"),C.innerHTML=lt,Se=n(),I=a("p"),I.innerHTML=tt,Qe=n(),y(f.$$.fragment),He=n(),y(h.$$.fragment),Le=n(),A=a("p"),A.innerHTML=st,Ve=n(),y(b.$$.fragment),xe=n(),k=a("p"),k.innerHTML=nt,Xe=n(),y(g.$$.fragment),ze=n(),$=a("p"),$.innerHTML=Mt,Fe=n(),y(_.$$.fragment),qe=n(),E=a("p"),E.innerHTML=at,Ye=n(),y(R.$$.fragment),Pe=n(),v=a("p"),v.innerHTML=it,Ke=n(),y(N.$$.fragment),Oe=n(),B=a("p"),B.innerHTML=rt,De=n(),Z=a("p"),Z.innerHTML=yt,el=n(),y(W.$$.fragment),ll=n(),y(G.$$.fragment),tl=n(),S=a("p"),S.innerHTML=ot,sl=n(),y(Q.$$.fragment),nl=n(),H=a("p"),H.innerHTML=pt,Ml=n(),y(L.$$.fragment),al=n(),V=a("p"),V.innerHTML=Tt,il=n(),x=a("p"),x.innerHTML=Ut,rl=n(),y(X.$$.fragment),yl=n(),y(z.$$.fragment),ol=n(),F=a("p"),F.innerHTML=Jt,pl=n(),y(q.$$.fragment),Tl=n(),y(Y.$$.fragment),Ul=n(),P=a("p"),P.innerHTML=wt,Jl=n(),y(K.$$.fragment),wl=n(),O=a("p"),O.textContent=ct,cl=n(),y(D.$$.fragment),ml=n(),ee=a("p"),ee.innerHTML=mt,ul=n(),y(le.$$.fragment),dl=n(),te=a("p"),te.innerHTML=ut,jl=n(),y(se.$$.fragment),Cl=n(),ne=a("p"),ne.innerHTML=dt,Il=n(),u=a("blockquote"),u.innerHTML=jt,fl=n(),Me=a("p"),Me.innerHTML=Ct,hl=n(),y(ae.$$.fragment),Al=n(),ie=a("p"),ie.innerHTML=It,bl=n(),y(re.$$.fragment),kl=n(),ye=a("p"),ye.innerHTML=ft,gl=n(),oe=a("p"),oe.innerHTML=ht,$l=n(),y(pe.$$.fragment),_l=n(),y(Te.$$.fragment),El=n(),Ue=a("p"),Ue.textContent=At,Rl=n(),y(Je.$$.fragment),vl=n(),we=a("p"),we.innerHTML=bt,Nl=n(),ce=a("ul"),ce.innerHTML=kt,Bl=n(),me=a("p"),me.innerHTML=gt,Zl=n(),y(ue.$$.fragment),Wl=n(),de=a("p"),de.innerHTML=$t,Gl=n(),y(je.$$.fragment),Sl=n(),Ce=a("p"),Ce.innerHTML=_t,Ql=n(),y(Ie.$$.fragment),Hl=n(),fe=a("p"),fe.textContent=Et,Ll=n(),he=a("ul"),he.innerHTML=Rt,Vl=n(),y(Ae.$$.fragment),xl=n(),be=a("p"),be.innerHTML=vt,Xl=n(),y(ke.$$.fragment),zl=n(),ge=a("p"),ge.innerHTML=Nt,Fl=n(),y($e.$$.fragment),ql=n(),_e=a("p"),_e.innerHTML=Bt,Yl=n(),y(Ee.$$.fragment),Pl=n(),y(Re.$$.fragment),Kl=n(),Ne=a("p"),this.h()},l(e){const l=Lt("svelte-u9bgzb",document.head);m=i(l,"META",{name:!0,content:!0}),l.forEach(t),Be=M(e),ve=i(e,"P",{}),Zt(ve).forEach(t),Ze=M(e),o(d.$$.fragment,e),We=M(e),o(j.$$.fragment,e),Ge=M(e),C=i(e,"P",{"data-svelte-h":!0}),r(C)!=="svelte-178x673"&&(C.innerHTML=lt),Se=M(e),I=i(e,"P",{"data-svelte-h":!0}),r(I)!=="svelte-35znf1"&&(I.innerHTML=tt),Qe=M(e),o(f.$$.fragment,e),He=M(e),o(h.$$.fragment,e),Le=M(e),A=i(e,"P",{"data-svelte-h":!0}),r(A)!=="svelte-1smlos"&&(A.innerHTML=st),Ve=M(e),o(b.$$.fragment,e),xe=M(e),k=i(e,"P",{"data-svelte-h":!0}),r(k)!=="svelte-1lus29p"&&(k.innerHTML=nt),Xe=M(e),o(g.$$.fragment,e),ze=M(e),$=i(e,"P",{"data-svelte-h":!0}),r($)!=="svelte-y7857b"&&($.innerHTML=Mt),Fe=M(e),o(_.$$.fragment,e),qe=M(e),E=i(e,"P",{"data-svelte-h":!0}),r(E)!=="svelte-13coke1"&&(E.innerHTML=at),Ye=M(e),o(R.$$.fragment,e),Pe=M(e),v=i(e,"P",{"data-svelte-h":!0}),r(v)!=="svelte-dlegsn"&&(v.innerHTML=it),Ke=M(e),o(N.$$.fragment,e),Oe=M(e),B=i(e,"P",{"data-svelte-h":!0}),r(B)!=="svelte-1w74xdp"&&(B.innerHTML=rt),De=M(e),Z=i(e,"P",{"data-svelte-h":!0}),r(Z)!=="svelte-t3eq4x"&&(Z.innerHTML=yt),el=M(e),o(W.$$.fragment,e),ll=M(e),o(G.$$.fragment,e),tl=M(e),S=i(e,"P",{"data-svelte-h":!0}),r(S)!=="svelte-xugj28"&&(S.innerHTML=ot),sl=M(e),o(Q.$$.fragment,e),nl=M(e),H=i(e,"P",{"data-svelte-h":!0}),r(H)!=="svelte-1s9fyfs"&&(H.innerHTML=pt),Ml=M(e),o(L.$$.fragment,e),al=M(e),V=i(e,"P",{"data-svelte-h":!0}),r(V)!=="svelte-14s5eb3"&&(V.innerHTML=Tt),il=M(e),x=i(e,"P",{"data-svelte-h":!0}),r(x)!=="svelte-195gdzl"&&(x.innerHTML=Ut),rl=M(e),o(X.$$.fragment,e),yl=M(e),o(z.$$.fragment,e),ol=M(e),F=i(e,"P",{"data-svelte-h":!0}),r(F)!=="svelte-1dxuye4"&&(F.innerHTML=Jt),pl=M(e),o(q.$$.fragment,e),Tl=M(e),o(Y.$$.fragment,e),Ul=M(e),P=i(e,"P",{"data-svelte-h":!0}),r(P)!=="svelte-18neid7"&&(P.innerHTML=wt),Jl=M(e),o(K.$$.fragment,e),wl=M(e),O=i(e,"P",{"data-svelte-h":!0}),r(O)!=="svelte-1lkusth"&&(O.textContent=ct),cl=M(e),o(D.$$.fragment,e),ml=M(e),ee=i(e,"P",{"data-svelte-h":!0}),r(ee)!=="svelte-1mrgf3g"&&(ee.innerHTML=mt),ul=M(e),o(le.$$.fragment,e),dl=M(e),te=i(e,"P",{"data-svelte-h":!0}),r(te)!=="svelte-d1d1zb"&&(te.innerHTML=ut),jl=M(e),o(se.$$.fragment,e),Cl=M(e),ne=i(e,"P",{"data-svelte-h":!0}),r(ne)!=="svelte-gdm9xx"&&(ne.innerHTML=dt),Il=M(e),u=i(e,"BLOCKQUOTE",{class:!0,"data-svelte-h":!0}),r(u)!=="svelte-n51n6u"&&(u.innerHTML=jt),fl=M(e),Me=i(e,"P",{"data-svelte-h":!0}),r(Me)!=="svelte-1m8fbna"&&(Me.innerHTML=Ct),hl=M(e),o(ae.$$.fragment,e),Al=M(e),ie=i(e,"P",{"data-svelte-h":!0}),r(ie)!=="svelte-1ugm4f0"&&(ie.innerHTML=It),bl=M(e),o(re.$$.fragment,e),kl=M(e),ye=i(e,"P",{"data-svelte-h":!0}),r(ye)!=="svelte-z3xuuf"&&(ye.innerHTML=ft),gl=M(e),oe=i(e,"P",{"data-svelte-h":!0}),r(oe)!=="svelte-2jodrc"&&(oe.innerHTML=ht),$l=M(e),o(pe.$$.fragment,e),_l=M(e),o(Te.$$.fragment,e),El=M(e),Ue=i(e,"P",{"data-svelte-h":!0}),r(Ue)!=="svelte-1ed3b6y"&&(Ue.textContent=At),Rl=M(e),o(Je.$$.fragment,e),vl=M(e),we=i(e,"P",{"data-svelte-h":!0}),r(we)!=="svelte-16b9bxy"&&(we.innerHTML=bt),Nl=M(e),ce=i(e,"UL",{"data-svelte-h":!0}),r(ce)!=="svelte-ewhtf4"&&(ce.innerHTML=kt),Bl=M(e),me=i(e,"P",{"data-svelte-h":!0}),r(me)!=="svelte-m4furw"&&(me.innerHTML=gt),Zl=M(e),o(ue.$$.fragment,e),Wl=M(e),de=i(e,"P",{"data-svelte-h":!0}),r(de)!=="svelte-1oi4pcx"&&(de.innerHTML=$t),Gl=M(e),o(je.$$.fragment,e),Sl=M(e),Ce=i(e,"P",{"data-svelte-h":!0}),r(Ce)!=="svelte-1jhx6s0"&&(Ce.innerHTML=_t),Ql=M(e),o(Ie.$$.fragment,e),Hl=M(e),fe=i(e,"P",{"data-svelte-h":!0}),r(fe)!=="svelte-1k7nbkd"&&(fe.textContent=Et),Ll=M(e),he=i(e,"UL",{"data-svelte-h":!0}),r(he)!=="svelte-10ah83g"&&(he.innerHTML=Rt),Vl=M(e),o(Ae.$$.fragment,e),xl=M(e),be=i(e,"P",{"data-svelte-h":!0}),r(be)!=="svelte-1q9vv7t"&&(be.innerHTML=vt),Xl=M(e),o(ke.$$.fragment,e),zl=M(e),ge=i(e,"P",{"data-svelte-h":!0}),r(ge)!=="svelte-12ivbsf"&&(ge.innerHTML=Nt),Fl=M(e),o($e.$$.fragment,e),ql=M(e),_e=i(e,"P",{"data-svelte-h":!0}),r(_e)!=="svelte-nf6jny"&&(_e.innerHTML=Bt),Yl=M(e),o(Ee.$$.fragment,e),Pl=M(e),o(Re.$$.fragment,e),Kl=M(e),Ne=i(e,"P",{}),Zt(Ne).forEach(t),this.h()},h(){Dl(m,"name","hf:doc:metadata"),Dl(m,"content",Ft),Dl(u,"class","note")},m(e,l){Vt(document.head,m),s(e,Be,l),s(e,ve,l),s(e,Ze,l),p(d,e,l),s(e,We,l),p(j,e,l),s(e,Ge,l),s(e,C,l),s(e,Se,l),s(e,I,l),s(e,Qe,l),p(f,e,l),s(e,He,l),p(h,e,l),s(e,Le,l),s(e,A,l),s(e,Ve,l),p(b,e,l),s(e,xe,l),s(e,k,l),s(e,Xe,l),p(g,e,l),s(e,ze,l),s(e,$,l),s(e,Fe,l),p(_,e,l),s(e,qe,l),s(e,E,l),s(e,Ye,l),p(R,e,l),s(e,Pe,l),s(e,v,l),s(e,Ke,l),p(N,e,l),s(e,Oe,l),s(e,B,l),s(e,De,l),s(e,Z,l),s(e,el,l),p(W,e,l),s(e,ll,l),p(G,e,l),s(e,tl,l),s(e,S,l),s(e,sl,l),p(Q,e,l),s(e,nl,l),s(e,H,l),s(e,Ml,l),p(L,e,l),s(e,al,l),s(e,V,l),s(e,il,l),s(e,x,l),s(e,rl,l),p(X,e,l),s(e,yl,l),p(z,e,l),s(e,ol,l),s(e,F,l),s(e,pl,l),p(q,e,l),s(e,Tl,l),p(Y,e,l),s(e,Ul,l),s(e,P,l),s(e,Jl,l),p(K,e,l),s(e,wl,l),s(e,O,l),s(e,cl,l),p(D,e,l),s(e,ml,l),s(e,ee,l),s(e,ul,l),p(le,e,l),s(e,dl,l),s(e,te,l),s(e,jl,l),p(se,e,l),s(e,Cl,l),s(e,ne,l),s(e,Il,l),s(e,u,l),s(e,fl,l),s(e,Me,l),s(e,hl,l),p(ae,e,l),s(e,Al,l),s(e,ie,l),s(e,bl,l),p(re,e,l),s(e,kl,l),s(e,ye,l),s(e,gl,l),s(e,oe,l),s(e,$l,l),p(pe,e,l),s(e,_l,l),p(Te,e,l),s(e,El,l),s(e,Ue,l),s(e,Rl,l),p(Je,e,l),s(e,vl,l),s(e,we,l),s(e,Nl,l),s(e,ce,l),s(e,Bl,l),s(e,me,l),s(e,Zl,l),p(ue,e,l),s(e,Wl,l),s(e,de,l),s(e,Gl,l),p(je,e,l),s(e,Sl,l),s(e,Ce,l),s(e,Ql,l),p(Ie,e,l),s(e,Hl,l),s(e,fe,l),s(e,Ll,l),s(e,he,l),s(e,Vl,l),p(Ae,e,l),s(e,xl,l),s(e,be,l),s(e,Xl,l),p(ke,e,l),s(e,zl,l),s(e,ge,l),s(e,Fl,l),p($e,e,l),s(e,ql,l),s(e,_e,l),s(e,Yl,l),p(Ee,e,l),s(e,Pl,l),p(Re,e,l),s(e,Kl,l),s(e,Ne,l),Ol=!0},p:Gt,i(e){Ol||(T(d.$$.fragment,e),T(j.$$.fragment,e),T(f.$$.fragment,e),T(h.$$.fragment,e),T(b.$$.fragment,e),T(g.$$.fragment,e),T(_.$$.fragment,e),T(R.$$.fragment,e),T(N.$$.fragment,e),T(W.$$.fragment,e),T(G.$$.fragment,e),T(Q.$$.fragment,e),T(L.$$.fragment,e),T(X.$$.fragment,e),T(z.$$.fragment,e),T(q.$$.fragment,e),T(Y.$$.fragment,e),T(K.$$.fragment,e),T(D.$$.fragment,e),T(le.$$.fragment,e),T(se.$$.fragment,e),T(ae.$$.fragment,e),T(re.$$.fragment,e),T(pe.$$.fragment,e),T(Te.$$.fragment,e),T(Je.$$.fragment,e),T(ue.$$.fragment,e),T(je.$$.fragment,e),T(Ie.$$.fragment,e),T(Ae.$$.fragment,e),T(ke.$$.fragment,e),T($e.$$.fragment,e),T(Ee.$$.fragment,e),T(Re.$$.fragment,e),Ol=!0)},o(e){U(d.$$.fragment,e),U(j.$$.fragment,e),U(f.$$.fragment,e),U(h.$$.fragment,e),U(b.$$.fragment,e),U(g.$$.fragment,e),U(_.$$.fragment,e),U(R.$$.fragment,e),U(N.$$.fragment,e),U(W.$$.fragment,e),U(G.$$.fragment,e),U(Q.$$.fragment,e),U(L.$$.fragment,e),U(X.$$.fragment,e),U(z.$$.fragment,e),U(q.$$.fragment,e),U(Y.$$.fragment,e),U(K.$$.fragment,e),U(D.$$.fragment,e),U(le.$$.fragment,e),U(se.$$.fragment,e),U(ae.$$.fragment,e),U(re.$$.fragment,e),U(pe.$$.fragment,e),U(Te.$$.fragment,e),U(Je.$$.fragment,e),U(ue.$$.fragment,e),U(je.$$.fragment,e),U(Ie.$$.fragment,e),U(Ae.$$.fragment,e),U(ke.$$.fragment,e),U($e.$$.fragment,e),U(Ee.$$.fragment,e),U(Re.$$.fragment,e),Ol=!1},d(e){e&&(t(Be),t(ve),t(Ze),t(We),t(Ge),t(C),t(Se),t(I),t(Qe),t(He),t(Le),t(A),t(Ve),t(xe),t(k),t(Xe),t(ze),t($),t(Fe),t(qe),t(E),t(Ye),t(Pe),t(v),t(Ke),t(Oe),t(B),t(De),t(Z),t(el),t(ll),t(tl),t(S),t(sl),t(nl),t(H),t(Ml),t(al),t(V),t(il),t(x),t(rl),t(yl),t(ol),t(F),t(pl),t(Tl),t(Ul),t(P),t(Jl),t(wl),t(O),t(cl),t(ml),t(ee),t(ul),t(dl),t(te),t(jl),t(Cl),t(ne),t(Il),t(u),t(fl),t(Me),t(hl),t(Al),t(ie),t(bl),t(kl),t(ye),t(gl),t(oe),t($l),t(_l),t(El),t(Ue),t(Rl),t(vl),t(we),t(Nl),t(ce),t(Bl),t(me),t(Zl),t(Wl),t(de),t(Gl),t(Sl),t(Ce),t(Ql),t(Hl),t(fe),t(Ll),t(he),t(Vl),t(xl),t(be),t(Xl),t(zl),t(ge),t(Fl),t(ql),t(_e),t(Yl),t(Pl),t(Kl),t(Ne)),t(m),J(d,e),J(j,e),J(f,e),J(h,e),J(b,e),J(g,e),J(_,e),J(R,e),J(N,e),J(W,e),J(G,e),J(Q,e),J(L,e),J(X,e),J(z,e),J(q,e),J(Y,e),J(K,e),J(D,e),J(le,e),J(se,e),J(ae,e),J(re,e),J(pe,e),J(Te,e),J(Je,e),J(ue,e),J(je,e),J(Ie,e),J(Ae,e),J(ke,e),J($e,e),J(Ee,e),J(Re,e)}}}const Ft='{"title":"Layers","local":"layers","sections":[{"title":"Making a layer extensible with kernels from the hub","local":"making-a-layer-extensible-with-kernels-from-the-hub","sections":[{"title":"Using a decorator","local":"using-a-decorator","sections":[],"depth":3},{"title":"External layers","local":"external-layers","sections":[],"depth":3},{"title":"Using a function as a layer","local":"using-a-function-as-a-layer","sections":[],"depth":3}],"depth":2},{"title":"Kernelizing a model","local":"kernelizing-a-model","sections":[{"title":"Kernel device","local":"kernel-device","sections":[],"depth":3},{"title":"Fallback forward","local":"fallback-forward","sections":[],"depth":3},{"title":"Inspecting which kernels are used","local":"inspecting-which-kernels-are-used","sections":[],"depth":3}],"depth":2},{"title":"Registering a hub kernel for a layer","local":"registering-a-hub-kernel-for-a-layer","sections":[{"title":"Registering kernels for specific modes","local":"registering-kernels-for-specific-modes","sections":[],"depth":3},{"title":"Registering kernels for specific CUDA capabilities","local":"registering-kernels-for-specific-cuda-capabilities","sections":[],"depth":3},{"title":"Registering kernels for specific ROCm capabilities","local":"registering-kernels-for-specific-rocm-capabilities","sections":[],"depth":3},{"title":"Loading from a local repository for testing","local":"loading-from-a-local-repository-for-testing","sections":[],"depth":3}],"depth":2}],"depth":1}';function qt(et){return St(()=>{new URLSearchParams(window.location.search).get("fw")}),[]}class es extends Qt{constructor(m){super(),Ht(this,m,qt,zt,Wt,{})}}export{es as component};

Xet Storage Details

Size:
45.1 kB
·
Xet hash:
fbaa03ad984d7441779f074680cea937d30d2d9cbbd25d1b9ff0a3fb0e6d941e

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.