Buckets:
| <meta charset="utf-8" /><meta name="hf:doc:metadata" content="{"title":"어떻게 사용자 정의 파이프라인을 생성하나요?","local":"how-to-create-a-custom-pipeline","sections":[{"title":"지원되는 작업 목록에 추가하기","local":"adding-it-to-the-list-of-supported-tasks","sections":[],"depth":2},{"title":"Hub에 파이프라인 공유하기","local":"share-your-pipeline-on-the-hub","sections":[],"depth":2},{"title":"🤗 Transformers에 파이프라인 추가하기","local":"add-the-pipeline-to-transformers","sections":[],"depth":2}],"depth":1}"> | |
| <link href="/docs/transformers/main/ko/_app/immutable/assets/0.e3b0c442.css" rel="modulepreload"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ko/_app/immutable/entry/start.9aa88961.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ko/_app/immutable/chunks/scheduler.9bc65507.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ko/_app/immutable/chunks/singletons.9eec45c3.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ko/_app/immutable/chunks/index.3b203c72.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ko/_app/immutable/chunks/paths.566078f7.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ko/_app/immutable/entry/app.84fb67c3.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ko/_app/immutable/chunks/index.707bf1b6.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ko/_app/immutable/nodes/0.1c99376b.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ko/_app/immutable/chunks/each.e59479a4.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ko/_app/immutable/nodes/4.ae5e582c.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ko/_app/immutable/chunks/CodeBlock.54a9f38d.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ko/_app/immutable/chunks/EditOnGithub.922df6ba.js"><!-- HEAD_svelte-u9bgzb_START --><meta name="hf:doc:metadata" content="{"title":"어떻게 사용자 정의 파이프라인을 생성하나요?","local":"how-to-create-a-custom-pipeline","sections":[{"title":"지원되는 작업 목록에 추가하기","local":"adding-it-to-the-list-of-supported-tasks","sections":[],"depth":2},{"title":"Hub에 파이프라인 공유하기","local":"share-your-pipeline-on-the-hub","sections":[],"depth":2},{"title":"🤗 Transformers에 파이프라인 추가하기","local":"add-the-pipeline-to-transformers","sections":[],"depth":2}],"depth":1}"><!-- HEAD_svelte-u9bgzb_END --> <p></p> <h1 class="relative group"><a id="how-to-create-a-custom-pipeline" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#how-to-create-a-custom-pipeline"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>어떻게 사용자 정의 파이프라인을 생성하나요?</span></h1> <p data-svelte-h="svelte-1fzm5o3">이 가이드에서는 사용자 정의 파이프라인을 어떻게 생성하고 <a href="https://hf.co/models" rel="nofollow">허브</a>에 공유하거나 🤗 Transformers 라이브러리에 추가하는 방법을 살펴보겠습니다.</p> <p data-svelte-h="svelte-6ogr6z">먼저 파이프라인이 수용할 수 있는 원시 입력을 결정해야 합니다. | |
| 문자열, 원시 바이트, 딕셔너리 또는 가장 원하는 입력일 가능성이 높은 것이면 무엇이든 가능합니다. | |
| 이 입력을 가능한 한 순수한 Python 형식으로 유지해야 (JSON을 통해 다른 언어와도) 호환성이 좋아집니다. | |
| 이것이 전처리(<code>preprocess</code>) 파이프라인의 입력(<code>inputs</code>)이 될 것입니다.</p> <p data-svelte-h="svelte-1r1v4v7">그런 다음 <code>outputs</code>를 정의하세요. | |
| <code>inputs</code>와 같은 정책을 따르고, 간단할수록 좋습니다. | |
| 이것이 후처리(<code>postprocess</code>) 메소드의 출력이 될 것입니다.</p> <p data-svelte-h="svelte-1u9gwbf">먼저 4개의 메소드(<code>preprocess</code>, <code>_forward</code>, <code>postprocess</code> 및 <code>_sanitize_parameters</code>)를 구현하기 위해 기본 클래스 <code>Pipeline</code>을 상속하여 시작합니다.</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">from</span> transformers <span class="hljs-keyword">import</span> Pipeline | |
| <span class="hljs-keyword">class</span> <span class="hljs-title class_">MyPipeline</span>(<span class="hljs-title class_ inherited__">Pipeline</span>): | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">_sanitize_parameters</span>(<span class="hljs-params">self, **kwargs</span>): | |
| preprocess_kwargs = {} | |
| <span class="hljs-keyword">if</span> <span class="hljs-string">"maybe_arg"</span> <span class="hljs-keyword">in</span> kwargs: | |
| preprocess_kwargs[<span class="hljs-string">"maybe_arg"</span>] = kwargs[<span class="hljs-string">"maybe_arg"</span>] | |
| <span class="hljs-keyword">return</span> preprocess_kwargs, {}, {} | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">preprocess</span>(<span class="hljs-params">self, inputs, maybe_arg=<span class="hljs-number">2</span></span>): | |
| model_input = Tensor(inputs[<span class="hljs-string">"input_ids"</span>]) | |
| <span class="hljs-keyword">return</span> {<span class="hljs-string">"model_input"</span>: model_input} | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">_forward</span>(<span class="hljs-params">self, model_inputs</span>): | |
| <span class="hljs-comment"># model_inputs == {"model_input": model_input}</span> | |
| outputs = self.model(**model_inputs) | |
| <span class="hljs-comment"># Maybe {"logits": Tensor(...)}</span> | |
| <span class="hljs-keyword">return</span> outputs | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">postprocess</span>(<span class="hljs-params">self, model_outputs</span>): | |
| best_class = model_outputs[<span class="hljs-string">"logits"</span>].softmax(-<span class="hljs-number">1</span>) | |
| <span class="hljs-keyword">return</span> best_class<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-rv0lg4">이 분할 구조는 CPU/GPU에 대한 비교적 원활한 지원을 제공하는 동시에, 다른 스레드에서 CPU에 대한 사전/사후 처리를 수행할 수 있게 지원하는 것입니다.</p> <p data-svelte-h="svelte-fwn4og"><code>preprocess</code>는 원래 정의된 입력을 가져와 모델에 공급할 수 있는 형식으로 변환합니다. | |
| 더 많은 정보를 포함할 수 있으며 일반적으로 <code>Dict</code> 형태입니다.</p> <p data-svelte-h="svelte-tcs7ux"><code>_forward</code>는 구현 세부 사항이며 직접 호출할 수 없습니다. | |
| <code>forward</code>는 예상 장치에서 모든 것이 작동하는지 확인하기 위한 안전장치가 포함되어 있어 선호되는 호출 메소드입니다. | |
| 실제 모델과 관련된 것은 <code>_forward</code> 메소드에 속하며, 나머지는 전처리/후처리 과정에 있습니다.</p> <p data-svelte-h="svelte-1a54d54"><code>postprocess</code> 메소드는 <code>_forward</code>의 출력을 가져와 이전에 결정한 최종 출력 형식으로 변환합니다.</p> <p data-svelte-h="svelte-ufagsr"><code>_sanitize_parameters</code>는 초기화 시간에 <code>pipeline(...., maybe_arg=4)</code>이나 호출 시간에 <code>pipe = pipeline(...); output = pipe(...., maybe_arg=4)</code>과 같이, 사용자가 원하는 경우 언제든지 매개변수를 전달할 수 있도록 허용합니다.</p> <p data-svelte-h="svelte-j02rys"><code>_sanitize_parameters</code>의 반환 값은 <code>preprocess</code>, <code>_forward</code>, <code>postprocess</code>에 직접 전달되는 3개의 kwargs 딕셔너리입니다. | |
| 호출자가 추가 매개변수로 호출하지 않았다면 아무것도 채우지 마십시오. | |
| 이렇게 하면 항상 더 “자연스러운” 함수 정의의 기본 인수를 유지할 수 있습니다.</p> <p data-svelte-h="svelte-1n2acw">분류 작업에서 <code>top_k</code> 매개변수가 대표적인 예입니다.</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-meta">>>> </span>pipe = pipeline(<span class="hljs-string">"my-new-task"</span>) | |
| <span class="hljs-meta">>>> </span>pipe(<span class="hljs-string">"This is a test"</span>) | |
| [{<span class="hljs-string">"label"</span>: <span class="hljs-string">"1-star"</span>, <span class="hljs-string">"score"</span>: <span class="hljs-number">0.8</span>}, {<span class="hljs-string">"label"</span>: <span class="hljs-string">"2-star"</span>, <span class="hljs-string">"score"</span>: <span class="hljs-number">0.1</span>}, {<span class="hljs-string">"label"</span>: <span class="hljs-string">"3-star"</span>, <span class="hljs-string">"score"</span>: <span class="hljs-number">0.05</span>} | |
| {<span class="hljs-string">"label"</span>: <span class="hljs-string">"4-star"</span>, <span class="hljs-string">"score"</span>: <span class="hljs-number">0.025</span>}, {<span class="hljs-string">"label"</span>: <span class="hljs-string">"5-star"</span>, <span class="hljs-string">"score"</span>: <span class="hljs-number">0.025</span>}] | |
| <span class="hljs-meta">>>> </span>pipe(<span class="hljs-string">"This is a test"</span>, top_k=<span class="hljs-number">2</span>) | |
| [{<span class="hljs-string">"label"</span>: <span class="hljs-string">"1-star"</span>, <span class="hljs-string">"score"</span>: <span class="hljs-number">0.8</span>}, {<span class="hljs-string">"label"</span>: <span class="hljs-string">"2-star"</span>, <span class="hljs-string">"score"</span>: <span class="hljs-number">0.1</span>}]<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-1d0sxb1">이를 달성하기 위해 우리는 <code>postprocess</code> 메소드를 기본 매개변수인 <code>5</code>로 업데이트하고 <code>_sanitize_parameters</code>를 수정하여 이 새 매개변수를 허용합니다.</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">def</span> <span class="hljs-title function_">postprocess</span>(<span class="hljs-params">self, model_outputs, top_k=<span class="hljs-number">5</span></span>): | |
| best_class = model_outputs[<span class="hljs-string">"logits"</span>].softmax(-<span class="hljs-number">1</span>) | |
| <span class="hljs-comment"># top_k를 처리하는 로직 추가</span> | |
| <span class="hljs-keyword">return</span> best_class | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">_sanitize_parameters</span>(<span class="hljs-params">self, **kwargs</span>): | |
| preprocess_kwargs = {} | |
| <span class="hljs-keyword">if</span> <span class="hljs-string">"maybe_arg"</span> <span class="hljs-keyword">in</span> kwargs: | |
| preprocess_kwargs[<span class="hljs-string">"maybe_arg"</span>] = kwargs[<span class="hljs-string">"maybe_arg"</span>] | |
| postprocess_kwargs = {} | |
| <span class="hljs-keyword">if</span> <span class="hljs-string">"top_k"</span> <span class="hljs-keyword">in</span> kwargs: | |
| postprocess_kwargs[<span class="hljs-string">"top_k"</span>] = kwargs[<span class="hljs-string">"top_k"</span>] | |
| <span class="hljs-keyword">return</span> preprocess_kwargs, {}, postprocess_kwargs<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-1k45o3o">입/출력을 가능한 한 간단하고 완전히 JSON 직렬화 가능한 형식으로 유지하려고 노력하십시오. | |
| 이렇게 하면 사용자가 새로운 종류의 개체를 이해하지 않고도 파이프라인을 쉽게 사용할 수 있습니다. | |
| 또한 사용 용이성을 위해 여러 가지 유형의 인수(오디오 파일은 파일 이름, URL 또는 순수한 바이트일 수 있음)를 지원하는 것이 비교적 일반적입니다.</p> <h2 class="relative group"><a id="adding-it-to-the-list-of-supported-tasks" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#adding-it-to-the-list-of-supported-tasks"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>지원되는 작업 목록에 추가하기</span></h2> <p data-svelte-h="svelte-1ty0isp"><code>new-task</code>를 지원되는 작업 목록에 등록하려면 <code>PIPELINE_REGISTRY</code>에 추가해야 합니다:</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">from</span> transformers.pipelines <span class="hljs-keyword">import</span> PIPELINE_REGISTRY | |
| PIPELINE_REGISTRY.register_pipeline( | |
| <span class="hljs-string">"new-task"</span>, | |
| pipeline_class=MyPipeline, | |
| pt_model=AutoModelForSequenceClassification, | |
| )<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-ct19sw">원하는 경우 기본 모델을 지정할 수 있으며, 이 경우 특정 개정(분기 이름 또는 커밋 해시일 수 있음, 여기서는 “abcdef”)과 타입을 함께 가져와야 합니다:</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START -->PIPELINE_REGISTRY.register_pipeline( | |
| <span class="hljs-string">"new-task"</span>, | |
| pipeline_class=MyPipeline, | |
| pt_model=AutoModelForSequenceClassification, | |
| default={<span class="hljs-string">"pt"</span>: (<span class="hljs-string">"user/awesome_model"</span>, <span class="hljs-string">"abcdef"</span>)}, | |
| <span class="hljs-built_in">type</span>=<span class="hljs-string">"text"</span>, <span class="hljs-comment"># 현재 지원 유형: text, audio, image, multimodal</span> | |
| )<!-- HTML_TAG_END --></pre></div> <h2 class="relative group"><a id="share-your-pipeline-on-the-hub" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#share-your-pipeline-on-the-hub"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>Hub에 파이프라인 공유하기</span></h2> <p data-svelte-h="svelte-56khl3">Hub에 사용자 정의 파이프라인을 공유하려면 <code>Pipeline</code> 하위 클래스의 사용자 정의 코드를 Python 파일에 저장하기만 하면 됩니다. | |
| 예를 들어, 다음과 같이 문장 쌍 분류를 위한 사용자 정의 파이프라인을 사용한다고 가정해 보겠습니다:</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">import</span> numpy <span class="hljs-keyword">as</span> np | |
| <span class="hljs-keyword">from</span> transformers <span class="hljs-keyword">import</span> Pipeline | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">softmax</span>(<span class="hljs-params">outputs</span>): | |
| maxes = np.<span class="hljs-built_in">max</span>(outputs, axis=-<span class="hljs-number">1</span>, keepdims=<span class="hljs-literal">True</span>) | |
| shifted_exp = np.exp(outputs - maxes) | |
| <span class="hljs-keyword">return</span> shifted_exp / shifted_exp.<span class="hljs-built_in">sum</span>(axis=-<span class="hljs-number">1</span>, keepdims=<span class="hljs-literal">True</span>) | |
| <span class="hljs-keyword">class</span> <span class="hljs-title class_">PairClassificationPipeline</span>(<span class="hljs-title class_ inherited__">Pipeline</span>): | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">_sanitize_parameters</span>(<span class="hljs-params">self, **kwargs</span>): | |
| preprocess_kwargs = {} | |
| <span class="hljs-keyword">if</span> <span class="hljs-string">"second_text"</span> <span class="hljs-keyword">in</span> kwargs: | |
| preprocess_kwargs[<span class="hljs-string">"second_text"</span>] = kwargs[<span class="hljs-string">"second_text"</span>] | |
| <span class="hljs-keyword">return</span> preprocess_kwargs, {}, {} | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">preprocess</span>(<span class="hljs-params">self, text, second_text=<span class="hljs-literal">None</span></span>): | |
| <span class="hljs-keyword">return</span> self.tokenizer(text, text_pair=second_text, return_tensors=self.framework) | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">_forward</span>(<span class="hljs-params">self, model_inputs</span>): | |
| <span class="hljs-keyword">return</span> self.model(**model_inputs) | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">postprocess</span>(<span class="hljs-params">self, model_outputs</span>): | |
| logits = model_outputs.logits[<span class="hljs-number">0</span>].numpy() | |
| probabilities = softmax(logits) | |
| best_class = np.argmax(probabilities) | |
| label = self.model.config.id2label[best_class] | |
| score = probabilities[best_class].item() | |
| logits = logits.tolist() | |
| <span class="hljs-keyword">return</span> {<span class="hljs-string">"label"</span>: label, <span class="hljs-string">"score"</span>: score, <span class="hljs-string">"logits"</span>: logits}<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-1d1jvf5">구현은 프레임워크에 구애받지 않으며, PyTorch와 TensorFlow 모델에 대해 작동합니다. | |
| 이를 <code>pair_classification.py</code>라는 파일에 저장한 경우, 다음과 같이 가져오고 등록할 수 있습니다:</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">from</span> pair_classification <span class="hljs-keyword">import</span> PairClassificationPipeline | |
| <span class="hljs-keyword">from</span> transformers.pipelines <span class="hljs-keyword">import</span> PIPELINE_REGISTRY | |
| <span class="hljs-keyword">from</span> transformers <span class="hljs-keyword">import</span> AutoModelForSequenceClassification, TFAutoModelForSequenceClassification | |
| PIPELINE_REGISTRY.register_pipeline( | |
| <span class="hljs-string">"pair-classification"</span>, | |
| pipeline_class=PairClassificationPipeline, | |
| pt_model=AutoModelForSequenceClassification, | |
| tf_model=TFAutoModelForSequenceClassification, | |
| )<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-1vdr0z9">이 작업이 완료되면 사전훈련된 모델과 함께 사용할 수 있습니다. | |
| 예를 들어, <code>sgugger/finetuned-bert-mrpc</code>은 MRPC 데이터 세트에서 미세 조정되어 문장 쌍을 패러프레이즈인지 아닌지를 분류합니다.</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">from</span> transformers <span class="hljs-keyword">import</span> pipeline | |
| classifier = pipeline(<span class="hljs-string">"pair-classification"</span>, model=<span class="hljs-string">"sgugger/finetuned-bert-mrpc"</span>)<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-dyh6pw">그런 다음 <code>push_to_hub</code> 메소드를 사용하여 허브에 공유할 수 있습니다:</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START -->classifier.push_to_hub(<span class="hljs-string">"test-dynamic-pipeline"</span>)<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-1yjhf56">이렇게 하면 “test-dynamic-pipeline” 폴더 내에 <code>PairClassificationPipeline</code>을 정의한 파일이 복사되며, 파이프라인의 모델과 토크나이저도 저장한 후, <code>{your_username}/test-dynamic-pipeline</code> 저장소에 있는 모든 것을 푸시합니다. | |
| 이후에는 <code>trust_remote_code=True</code> 옵션만 제공하면 누구나 사용할 수 있습니다.</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">from</span> transformers <span class="hljs-keyword">import</span> pipeline | |
| classifier = pipeline(model=<span class="hljs-string">"{your_username}/test-dynamic-pipeline"</span>, trust_remote_code=<span class="hljs-literal">True</span>)<!-- HTML_TAG_END --></pre></div> <h2 class="relative group"><a id="add-the-pipeline-to-transformers" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#add-the-pipeline-to-transformers"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>🤗 Transformers에 파이프라인 추가하기</span></h2> <p data-svelte-h="svelte-52sri7">🤗 Transformers에 사용자 정의 파이프라인을 기여하려면, <code>pipelines</code> 하위 모듈에 사용자 정의 파이프라인 코드와 함께 새 모듈을 추가한 다음, <code>pipelines/__init__.py</code>에서 정의된 작업 목록에 추가해야 합니다.</p> <p data-svelte-h="svelte-5f0sxq">그런 다음 테스트를 추가해야 합니다. | |
| <code>tests/test_pipelines_MY_PIPELINE.py</code>라는 새 파일을 만들고 다른 테스트와 예제를 함께 작성합니다.</p> <p data-svelte-h="svelte-yv6zf4"><code>run_pipeline_test</code> 함수는 매우 일반적이며, <code>model_mapping</code> 및 <code>tf_model_mapping</code>에서 정의된 가능한 모든 아키텍처의 작은 무작위 모델에서 실행됩니다.</p> <p data-svelte-h="svelte-15fh04w">이는 향후 호환성을 테스트하는 데 매우 중요하며, 누군가 <code>XXXForQuestionAnswering</code>을 위한 새 모델을 추가하면 파이프라인 테스트가 해당 모델에서 실행을 시도한다는 의미입니다. | |
| 모델이 무작위이기 때문에 실제 값을 확인하는 것은 불가능하므로, 단순히 파이프라인 출력 <code>TYPE</code>과 일치시키기 위한 도우미 <code>ANY</code>가 있습니다.</p> <p data-svelte-h="svelte-e68emk">또한 2개(이상적으로는 4개)의 테스트를 구현해야 합니다.</p> <ul data-svelte-h="svelte-1wfarmi"><li><code>test_small_model_pt</code>: 이 파이프라인에 대한 작은 모델 1개를 정의(결과가 의미 없어도 상관없음)하고 파이프라인 출력을 테스트합니다. | |
| 결과는 <code>test_small_model_tf</code>와 동일해야 합니다.</li> <li><code>test_small_model_tf</code>: 이 파이프라인에 대한 작은 모델 1개를 정의(결과가 의미 없어도 상관없음)하고 파이프라인 출력을 테스트합니다. | |
| 결과는 <code>test_small_model_pt</code>와 동일해야 합니다.</li> <li><code>test_large_model_pt</code>(<code>선택사항</code>): 결과가 의미 있을 것으로 예상되는 실제 파이프라인에서 파이프라인을 테스트합니다. | |
| 이러한 테스트는 속도가 느리므로 이를 표시해야 합니다. | |
| 여기서의 목표는 파이프라인을 보여주고 향후 릴리즈에서의 변화가 없는지 확인하는 것입니다.</li> <li><code>test_large_model_tf</code>(<code>선택사항</code>): 결과가 의미 있을 것으로 예상되는 실제 파이프라인에서 파이프라인을 테스트합니다. | |
| 이러한 테스트는 속도가 느리므로 이를 표시해야 합니다. | |
| 여기서의 목표는 파이프라인을 보여주고 향후 릴리즈에서의 변화가 없는지 확인하는 것입니다.</li></ul> <a class="!text-gray-400 !no-underline text-sm flex items-center not-prose mt-4" href="https://github.com/huggingface/transformers/blob/main/docs/source/ko/add_new_pipeline.md" target="_blank"><span data-svelte-h="svelte-1kd6by1"><</span> <span data-svelte-h="svelte-x0xyl0">></span> <span data-svelte-h="svelte-1dajgef"><span class="underline ml-1.5">Update</span> on GitHub</span></a> <p></p> | |
| <script> | |
| { | |
| __sveltekit_1hrx8 = { | |
| assets: "/docs/transformers/main/ko", | |
| base: "/docs/transformers/main/ko", | |
| env: {} | |
| }; | |
| const element = document.currentScript.parentElement; | |
| const data = [null,null]; | |
| Promise.all([ | |
| import("/docs/transformers/main/ko/_app/immutable/entry/start.9aa88961.js"), | |
| import("/docs/transformers/main/ko/_app/immutable/entry/app.84fb67c3.js") | |
| ]).then(([kit, app]) => { | |
| kit.start(app, element, { | |
| node_ids: [0, 4], | |
| data, | |
| form: null, | |
| error: null | |
| }); | |
| }); | |
| } | |
| </script> | |
Xet Storage Details
- Size:
- 39.6 kB
- Xet hash:
- b1f91679a551853e111c176bd49d41a6aa2cf6c401ed2114e227c9c2dc05d21a
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.