Buckets:
| <meta charset="utf-8" /><meta name="hf:doc:metadata" content="{"title":"Come creare una pipeline personalizzata?","local":"come-creare-una-pipeline-personalizzata","sections":[{"title":"Aggiungilo alla lista dei tasks supportati","local":"aggiungilo-alla-lista-dei-tasks-supportati","sections":[],"depth":2},{"title":"Condividi la tua pipeline sull’Hub","local":"condividi-la-tua-pipeline-sullhub","sections":[],"depth":2},{"title":"Aggiungere la pipeline a Transformers","local":"aggiungere-la-pipeline-a-transformers","sections":[],"depth":2}],"depth":1}"> | |
| <link href="/docs/transformers/main/it/_app/immutable/assets/0.e3b0c442.css" rel="modulepreload"> | |
| <link rel="modulepreload" href="/docs/transformers/main/it/_app/immutable/entry/start.dac10038.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/it/_app/immutable/chunks/scheduler.36a0863c.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/it/_app/immutable/chunks/singletons.d5d832bc.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/it/_app/immutable/chunks/index.733708bb.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/it/_app/immutable/chunks/paths.d03c7e87.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/it/_app/immutable/entry/app.354f2351.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/it/_app/immutable/chunks/index.9c13489a.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/it/_app/immutable/nodes/0.b7cdb3fe.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/it/_app/immutable/chunks/each.e59479a4.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/it/_app/immutable/nodes/4.55f666f0.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/it/_app/immutable/chunks/CodeBlock.05d8ec32.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/it/_app/immutable/chunks/EditOnGithub.e88f2b7b.js"><!-- HEAD_svelte-u9bgzb_START --><meta name="hf:doc:metadata" content="{"title":"Come creare una pipeline personalizzata?","local":"come-creare-una-pipeline-personalizzata","sections":[{"title":"Aggiungilo alla lista dei tasks supportati","local":"aggiungilo-alla-lista-dei-tasks-supportati","sections":[],"depth":2},{"title":"Condividi la tua pipeline sull’Hub","local":"condividi-la-tua-pipeline-sullhub","sections":[],"depth":2},{"title":"Aggiungere la pipeline a Transformers","local":"aggiungere-la-pipeline-a-transformers","sections":[],"depth":2}],"depth":1}"><!-- HEAD_svelte-u9bgzb_END --> <p></p> <h1 class="relative group"><a id="come-creare-una-pipeline-personalizzata" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#come-creare-una-pipeline-personalizzata"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>Come creare una pipeline personalizzata?</span></h1> <p data-svelte-h="svelte-11810ad">In questa guida, scopriremo come creare una pipeline personalizzata e condividerla sull’ <a href="https://hf.co/models" rel="nofollow">Hub</a> o aggiungerla nella libreria | |
| Transformers.</p> <p data-svelte-h="svelte-1iatkga">Innanzitutto, è necessario decidere gli input grezzi che la pipeline sarà in grado di accettare. Possono essere strings, raw bytes, | |
| dictionaries o qualsiasi cosa sia l’input desiderato più probabile. Cerca di mantenere questi input il più possibile in Python | |
| in quanto facilita la compatibilità (anche con altri linguaggi tramite JSON). Questi saranno gli <code>inputs</code> della | |
| pipeline (<code>preprocess</code>).</p> <p data-svelte-h="svelte-i2xp1j">Poi definire gli <code>outputs</code>. Stessa strategia degli <code>inputs</code>. Più è seplice e meglio è. Questi saranno gli output del metodo | |
| <code>postprocess</code>.</p> <p data-svelte-h="svelte-eef5a9">Si parte ereditando la classe base <code>Pipeline</code>. con i 4 metodi che bisogna implementare <code>preprocess</code>, | |
| <code>_forward</code>, <code>postprocess</code> e <code>_sanitize_parameters</code>.</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">from</span> transformers <span class="hljs-keyword">import</span> Pipeline | |
| <span class="hljs-keyword">class</span> <span class="hljs-title class_">MyPipeline</span>(<span class="hljs-title class_ inherited__">Pipeline</span>): | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">_sanitize_parameters</span>(<span class="hljs-params">self, **kwargs</span>): | |
| preprocess_kwargs = {} | |
| <span class="hljs-keyword">if</span> <span class="hljs-string">"maybe_arg"</span> <span class="hljs-keyword">in</span> kwargs: | |
| preprocess_kwargs[<span class="hljs-string">"maybe_arg"</span>] = kwargs[<span class="hljs-string">"maybe_arg"</span>] | |
| <span class="hljs-keyword">return</span> preprocess_kwargs, {}, {} | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">preprocess</span>(<span class="hljs-params">self, inputs, maybe_arg=<span class="hljs-number">2</span></span>): | |
| model_input = Tensor(inputs[<span class="hljs-string">"input_ids"</span>]) | |
| <span class="hljs-keyword">return</span> {<span class="hljs-string">"model_input"</span>: model_input} | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">_forward</span>(<span class="hljs-params">self, model_inputs</span>): | |
| <span class="hljs-comment"># model_inputs == {"model_input": model_input}</span> | |
| outputs = self.model(**model_inputs) | |
| <span class="hljs-comment"># Maybe {"logits": Tensor(...)}</span> | |
| <span class="hljs-keyword">return</span> outputs | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">postprocess</span>(<span class="hljs-params">self, model_outputs</span>): | |
| best_class = model_outputs[<span class="hljs-string">"logits"</span>].softmax(-<span class="hljs-number">1</span>) | |
| <span class="hljs-keyword">return</span> best_class<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-1lfi6tr">La struttura di questa suddivisione consiste nel supportare in modo relativamente continuo CPU/GPU, supportando allo stesso tempo l’esecuzione di | |
| pre/postelaborazione sulla CPU su thread diversi.</p> <p data-svelte-h="svelte-m2oyf8"><code>preprocess</code> prenderà gli input originariamente definiti e li trasformerà in qualcosa di alimentabile dal modello. Potrebbe | |
| contenere più informazioni e di solito è un <code>Dict</code>.</p> <p data-svelte-h="svelte-xk0co8"><code>_forward</code> è il dettaglio dell’implementazione e non è destinato a essere chiamato direttamente. <code>forward</code> è il metodo preferito per assicurarsi che tutto funzioni correttamente perchè contiene delle slavaguardie. Se qualcosa è | |
| è collegato a un modello reale, appartiene al metodo <code>_forward</code>, tutto il resto è nel preprocess/postprocess.</p> <p data-svelte-h="svelte-wbceks"><code>postprocess</code> prende l’otput di <code>_forward</code> e lo trasforma nell’output finale che era stato deciso in precedenza.</p> <p data-svelte-h="svelte-1w8676p"><code>_sanitize_parameters</code> esiste per consentire agli utenti di passare i parametri ogni volta che desiderano sia a inizialization time <code>pipeline(...., maybe_arg=4)</code> che al call time <code>pipe = pipeline(...); output = pipe(...., maybe_arg=4)</code>.</p> <p data-svelte-h="svelte-wn1l5e"><code>_sanitize_parameters</code> ritorna 3 dicts di kwargs che vengono passati direttamente a <code>preprocess</code>, | |
| <code>_forward</code> e <code>postprocess</code>. Non riempire nulla se il chiamante non ha chiamato con alcun parametro aggiuntivo. Questo | |
| consente di mantenere gli argomenti predefiniti nella definizione della funzione, che è sempre più “naturale”.</p> <p data-svelte-h="svelte-hymnzr">Un esempio classico potrebbe essere l’argomento <code>top_k</code> nel post processing dei classification tasks.</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-meta">>>> </span>pipe = pipeline(<span class="hljs-string">"my-new-task"</span>) | |
| <span class="hljs-meta">>>> </span>pipe(<span class="hljs-string">"This is a test"</span>) | |
| [{<span class="hljs-string">"label"</span>: <span class="hljs-string">"1-star"</span>, <span class="hljs-string">"score"</span>: <span class="hljs-number">0.8</span>}, {<span class="hljs-string">"label"</span>: <span class="hljs-string">"2-star"</span>, <span class="hljs-string">"score"</span>: <span class="hljs-number">0.1</span>}, {<span class="hljs-string">"label"</span>: <span class="hljs-string">"3-star"</span>, <span class="hljs-string">"score"</span>: <span class="hljs-number">0.05</span>} | |
| {<span class="hljs-string">"label"</span>: <span class="hljs-string">"4-star"</span>, <span class="hljs-string">"score"</span>: <span class="hljs-number">0.025</span>}, {<span class="hljs-string">"label"</span>: <span class="hljs-string">"5-star"</span>, <span class="hljs-string">"score"</span>: <span class="hljs-number">0.025</span>}] | |
| <span class="hljs-meta">>>> </span>pipe(<span class="hljs-string">"This is a test"</span>, top_k=<span class="hljs-number">2</span>) | |
| [{<span class="hljs-string">"label"</span>: <span class="hljs-string">"1-star"</span>, <span class="hljs-string">"score"</span>: <span class="hljs-number">0.8</span>}, {<span class="hljs-string">"label"</span>: <span class="hljs-string">"2-star"</span>, <span class="hljs-string">"score"</span>: <span class="hljs-number">0.1</span>}]<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-nsc0of">In order to achieve that, we’ll update our <code>postprocess</code> method with a default parameter to <code>5</code>. and edit | |
| <code>_sanitize_parameters</code> to allow this new parameter.</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">def</span> <span class="hljs-title function_">postprocess</span>(<span class="hljs-params">self, model_outputs, top_k=<span class="hljs-number">5</span></span>): | |
| best_class = model_outputs[<span class="hljs-string">"logits"</span>].softmax(-<span class="hljs-number">1</span>) | |
| <span class="hljs-comment"># Add logic to handle top_k</span> | |
| <span class="hljs-keyword">return</span> best_class | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">_sanitize_parameters</span>(<span class="hljs-params">self, **kwargs</span>): | |
| preprocess_kwargs = {} | |
| <span class="hljs-keyword">if</span> <span class="hljs-string">"maybe_arg"</span> <span class="hljs-keyword">in</span> kwargs: | |
| preprocess_kwargs[<span class="hljs-string">"maybe_arg"</span>] = kwargs[<span class="hljs-string">"maybe_arg"</span>] | |
| postprocess_kwargs = {} | |
| <span class="hljs-keyword">if</span> <span class="hljs-string">"top_k"</span> <span class="hljs-keyword">in</span> kwargs: | |
| postprocess_kwargs[<span class="hljs-string">"top_k"</span>] = kwargs[<span class="hljs-string">"top_k"</span>] | |
| <span class="hljs-keyword">return</span> preprocess_kwargs, {}, postprocess_kwargs<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-1fkgamo">Cercare di mantenere gli input/output molto semplici e idealmente serializzabili in JSON, in quanto ciò rende l’uso della pipeline molto facile | |
| senza richiedere agli utenti di comprendere nuovi tipi di oggetti. È anche relativamente comune supportare molti tipi di argomenti | |
| per facilitarne l’uso (ad esempio file audio, possono essere nomi di file, URL o byte puri).</p> <h2 class="relative group"><a id="aggiungilo-alla-lista-dei-tasks-supportati" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#aggiungilo-alla-lista-dei-tasks-supportati"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>Aggiungilo alla lista dei tasks supportati</span></h2> <p data-svelte-h="svelte-4zkgvo">Per registrar il tuo <code>new-task</code> alla lista dei tasks supportati, devi aggiungerlo al <code>PIPELINE_REGISTRY</code>:</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">from</span> transformers.pipelines <span class="hljs-keyword">import</span> PIPELINE_REGISTRY | |
| PIPELINE_REGISTRY.register_pipeline( | |
| <span class="hljs-string">"new-task"</span>, | |
| pipeline_class=MyPipeline, | |
| pt_model=AutoModelForSequenceClassification, | |
| )<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-u38mjv">Puoi specificare il modello di default che desideri, in questo caso dovrebbe essere accompagnato da una revisione specifica (che può essere il nome di un branch o l’hash di un commit, in questo caso abbiamo preso <code>"abcdef"</code>) e anche dal type:</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START -->PIPELINE_REGISTRY.register_pipeline( | |
| <span class="hljs-string">"new-task"</span>, | |
| pipeline_class=MyPipeline, | |
| pt_model=AutoModelForSequenceClassification, | |
| default={<span class="hljs-string">"pt"</span>: (<span class="hljs-string">"user/awesome_model"</span>, <span class="hljs-string">"abcdef"</span>)}, | |
| <span class="hljs-built_in">type</span>=<span class="hljs-string">"text"</span>, <span class="hljs-comment"># current support type: text, audio, image, multimodal</span> | |
| )<!-- HTML_TAG_END --></pre></div> <h2 class="relative group"><a id="condividi-la-tua-pipeline-sullhub" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#condividi-la-tua-pipeline-sullhub"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>Condividi la tua pipeline sull’Hub</span></h2> <p data-svelte-h="svelte-16cgcsm">Per condividere la tua pipeline personalizzata sull’Hub, devi solo salvare il codice della tua sottoclasse <code>Pipeline</code> in un file | |
| python. Per esempio, supponiamo di voler utilizzare una pipeline personalizzata per la classificazione delle coppie di frasi come la seguente:</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">import</span> numpy <span class="hljs-keyword">as</span> np | |
| <span class="hljs-keyword">from</span> transformers <span class="hljs-keyword">import</span> Pipeline | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">softmax</span>(<span class="hljs-params">outputs</span>): | |
| maxes = np.<span class="hljs-built_in">max</span>(outputs, axis=-<span class="hljs-number">1</span>, keepdims=<span class="hljs-literal">True</span>) | |
| shifted_exp = np.exp(outputs - maxes) | |
| <span class="hljs-keyword">return</span> shifted_exp / shifted_exp.<span class="hljs-built_in">sum</span>(axis=-<span class="hljs-number">1</span>, keepdims=<span class="hljs-literal">True</span>) | |
| <span class="hljs-keyword">class</span> <span class="hljs-title class_">PairClassificationPipeline</span>(<span class="hljs-title class_ inherited__">Pipeline</span>): | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">_sanitize_parameters</span>(<span class="hljs-params">self, **kwargs</span>): | |
| preprocess_kwargs = {} | |
| <span class="hljs-keyword">if</span> <span class="hljs-string">"second_text"</span> <span class="hljs-keyword">in</span> kwargs: | |
| preprocess_kwargs[<span class="hljs-string">"second_text"</span>] = kwargs[<span class="hljs-string">"second_text"</span>] | |
| <span class="hljs-keyword">return</span> preprocess_kwargs, {}, {} | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">preprocess</span>(<span class="hljs-params">self, text, second_text=<span class="hljs-literal">None</span></span>): | |
| <span class="hljs-keyword">return</span> self.tokenizer(text, text_pair=second_text, return_tensors=self.framework) | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">_forward</span>(<span class="hljs-params">self, model_inputs</span>): | |
| <span class="hljs-keyword">return</span> self.model(**model_inputs) | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">postprocess</span>(<span class="hljs-params">self, model_outputs</span>): | |
| logits = model_outputs.logits[<span class="hljs-number">0</span>].numpy() | |
| probabilities = softmax(logits) | |
| best_class = np.argmax(probabilities) | |
| label = self.model.config.id2label[best_class] | |
| score = probabilities[best_class].item() | |
| logits = logits.tolist() | |
| <span class="hljs-keyword">return</span> {<span class="hljs-string">"label"</span>: label, <span class="hljs-string">"score"</span>: score, <span class="hljs-string">"logits"</span>: logits}<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-libyk9">L’implementazione è agnostica al framework, e lavorerà sia con modelli PyTorch che con TensorFlow. Se l’abbiamo salvato in un file chiamato <code>pair_classification.py</code>, può essere successivamente importato e registrato in questo modo:</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">from</span> pair_classification <span class="hljs-keyword">import</span> PairClassificationPipeline | |
| <span class="hljs-keyword">from</span> transformers.pipelines <span class="hljs-keyword">import</span> PIPELINE_REGISTRY | |
| <span class="hljs-keyword">from</span> transformers <span class="hljs-keyword">import</span> AutoModelForSequenceClassification, TFAutoModelForSequenceClassification | |
| PIPELINE_REGISTRY.register_pipeline( | |
| <span class="hljs-string">"pair-classification"</span>, | |
| pipeline_class=PairClassificationPipeline, | |
| pt_model=AutoModelForSequenceClassification, | |
| tf_model=TFAutoModelForSequenceClassification, | |
| )<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-vhle5u">Una volta fatto, possiamo usarla con un modello pretrained. L’istanza <code>sgugger/finetuned-bert-mrpc</code> è stata | |
| fine-tuned sul dataset MRPC, che classifica le coppie di frasi come parafrasi o no.</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">from</span> transformers <span class="hljs-keyword">import</span> pipeline | |
| classifier = pipeline(<span class="hljs-string">"pair-classification"</span>, model=<span class="hljs-string">"sgugger/finetuned-bert-mrpc"</span>)<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-g3f5h">Successivamente possiamo condividerlo sull’Hub usando il metodo <code>push_to_hub</code></p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START -->classifier.push_to_hub(<span class="hljs-string">"test-dynamic-pipeline"</span>)<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-1jl4eog">Questo codice copierà il file dove è stato definitp <code>PairClassificationPipeline</code> all’interno della cartella <code>"test-dynamic-pipeline"</code>, | |
| insieme al salvataggio del modello e del tokenizer della pipeline, prima di pushare il tutto nel repository | |
| <code>{your_username}/test-dynamic-pipeline</code>. Dopodiché chiunque potrà utilizzarlo, purché fornisca l’opzione | |
| <code>trust_remote_code=True</code>:</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">from</span> transformers <span class="hljs-keyword">import</span> pipeline | |
| classifier = pipeline(model=<span class="hljs-string">"{your_username}/test-dynamic-pipeline"</span>, trust_remote_code=<span class="hljs-literal">True</span>)<!-- HTML_TAG_END --></pre></div> <h2 class="relative group"><a id="aggiungere-la-pipeline-a-transformers" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#aggiungere-la-pipeline-a-transformers"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>Aggiungere la pipeline a Transformers</span></h2> <p data-svelte-h="svelte-3oo5s6">Se vuoi contribuire con la tua pipeline a Transformers, dovrai aggiungere un modulo nel sottomodulo <code>pipelines</code> | |
| con il codice della tua pipeline, quindi aggiungilo all’elenco dei tasks definiti in <code>pipelines/__init__.py</code>.</p> <p data-svelte-h="svelte-17yxzlw">Poi hai bisogno di aggiungere i test. Crea un nuovo file <code>tests/test_pipelines_MY_PIPELINE.py</code> con esempi ed altri test.</p> <p data-svelte-h="svelte-agzokg">La funzione <code>run_pipeline_test</code> sarà molto generica e su piccoli modelli casuali su ogni possibile | |
| architettura, come definito da <code>model_mapping</code> e <code>tf_model_mapping</code>.</p> <p data-svelte-h="svelte-td3vn9">Questo è molto importante per testare la compatibilità futura, nel senso che se qualcuno aggiunge un nuovo modello di | |
| <code>XXXForQuestionAnswering</code> allora il test della pipeline tenterà di essere eseguito su di esso. Poiché i modelli sono casuali, è | |
| è impossibile controllare i valori effettivi, per questo esiste un aiuto <code>ANY</code> che tenterà solamente di far corrispondere l’output della pipeline TYPE.</p> <p data-svelte-h="svelte-dschfg">Hai anche <em>bisogno</em> di implementare 2 (idealmente 4) test.</p> <ul data-svelte-h="svelte-1c56iwk"><li><code>test_small_model_pt</code> : Definire 1 piccolo modello per questa pipeline (non importa se i risultati non hanno senso) | |
| e testare i risultati della pipeline. I risultati dovrebbero essere gli stessi di <code>test_small_model_tf</code>.</li> <li><code>test_small_model_tf</code> : Definire 1 piccolo modello per questa pipeline (non importa se i risultati non hanno senso) | |
| e testare i risultati della pipeline. I risultati dovrebbero essere gli stessi di <code>test_small_model_pt</code>.</li> <li><code>test_large_model_pt</code> (<code>optional</code>): Testare la pipeline su una pipeline reale in cui i risultati dovrebbero avere | |
| senso. Questi test sono lenti e dovrebbero essere contrassegnati come tali. In questo caso l’obiettivo è mostrare la pipeline e assicurarsi che non ci siano derive nelle versioni future</li> <li><code>test_large_model_tf</code> (<code>optional</code>): Testare la pipeline su una pipeline reale in cui i risultati dovrebbero avere | |
| senso. Questi test sono lenti e dovrebbero essere contrassegnati come tali. In questo caso l’obiettivo è mostrare la pipeline e assicurarsi | |
| che non ci siano derive nelle versioni future</li></ul> <a class="!text-gray-400 !no-underline text-sm flex items-center not-prose mt-4" href="https://github.com/huggingface/transformers/blob/main/docs/source/it/add_new_pipeline.md" target="_blank"><span data-svelte-h="svelte-1kd6by1"><</span> <span data-svelte-h="svelte-x0xyl0">></span> <span data-svelte-h="svelte-1dajgef"><span class="underline ml-1.5">Update</span> on GitHub</span></a> <p></p> | |
| <script> | |
| { | |
| __sveltekit_1h8xam7 = { | |
| assets: "/docs/transformers/main/it", | |
| base: "/docs/transformers/main/it", | |
| env: {} | |
| }; | |
| const element = document.currentScript.parentElement; | |
| const data = [null,null]; | |
| Promise.all([ | |
| import("/docs/transformers/main/it/_app/immutable/entry/start.dac10038.js"), | |
| import("/docs/transformers/main/it/_app/immutable/entry/app.354f2351.js") | |
| ]).then(([kit, app]) => { | |
| kit.start(app, element, { | |
| node_ids: [0, 4], | |
| data, | |
| form: null, | |
| error: null | |
| }); | |
| }); | |
| } | |
| </script> | |
Xet Storage Details
- Size:
- 38.9 kB
- Xet hash:
- 1c25ebbf5718ac02da320493a679033f850b2fd67fa7d1c2299f8dfbeae6acbf
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.