Buckets:
| <meta charset="utf-8" /><meta name="hf:doc:metadata" content="{"title":"XLA Integration for TensorFlow Models","local":"xla-integration-for-tensorflow-models","sections":[{"title":"Running TF functions with XLA","local":"running-tf-functions-with-xla","sections":[],"depth":2},{"title":"Running a TF text generation model with XLA from 🤗 Transformers","local":"running-a-tf-text-generation-model-with-xla-from--transformers","sections":[],"depth":2},{"title":"Gotchas to be aware of","local":"gotchas-to-be-aware-of","sections":[],"depth":2},{"title":"Additional Resources","local":"additional-resources","sections":[],"depth":2}],"depth":1}"> | |
| <link href="/docs/transformers/main/ja/_app/immutable/assets/0.e3b0c442.css" rel="modulepreload"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ja/_app/immutable/entry/start.1486e459.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ja/_app/immutable/chunks/scheduler.9bc65507.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ja/_app/immutable/chunks/singletons.eee55cbf.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ja/_app/immutable/chunks/index.3b203c72.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ja/_app/immutable/chunks/paths.59da1547.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ja/_app/immutable/entry/app.d9ae818f.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ja/_app/immutable/chunks/index.707bf1b6.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ja/_app/immutable/nodes/0.c06aa070.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ja/_app/immutable/chunks/each.e59479a4.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ja/_app/immutable/nodes/162.c16f073c.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ja/_app/immutable/chunks/CodeBlock.54a9f38d.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ja/_app/immutable/chunks/DocNotebookDropdown.41f65cb5.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ja/_app/immutable/chunks/globals.7f7f1b26.js"> | |
| <link rel="modulepreload" href="/docs/transformers/main/ja/_app/immutable/chunks/EditOnGithub.922df6ba.js"><!-- HEAD_svelte-u9bgzb_START --><meta name="hf:doc:metadata" content="{"title":"XLA Integration for TensorFlow Models","local":"xla-integration-for-tensorflow-models","sections":[{"title":"Running TF functions with XLA","local":"running-tf-functions-with-xla","sections":[],"depth":2},{"title":"Running a TF text generation model with XLA from 🤗 Transformers","local":"running-a-tf-text-generation-model-with-xla-from--transformers","sections":[],"depth":2},{"title":"Gotchas to be aware of","local":"gotchas-to-be-aware-of","sections":[],"depth":2},{"title":"Additional Resources","local":"additional-resources","sections":[],"depth":2}],"depth":1}"><!-- HEAD_svelte-u9bgzb_END --> <p></p> <h1 class="relative group"><a id="xla-integration-for-tensorflow-models" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#xla-integration-for-tensorflow-models"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>XLA Integration for TensorFlow Models</span></h1> <div class="flex space-x-1 absolute z-10 right-0 top-0"> <div class="relative colab-dropdown "> <button class=" " type="button"> <img alt="Open In Colab" class="!m-0" src="https://colab.research.google.com/assets/colab-badge.svg"> </button> </div> <div class="relative colab-dropdown "> <button class=" " type="button"> <img alt="Open In Studio Lab" class="!m-0" src="https://studiolab.sagemaker.aws/studiolab.svg"> </button> </div></div> <p data-svelte-h="svelte-zzdk25">加速線形代数（Accelerated Linear Algebra）、通称XLAは、TensorFlowモデルのランタイムを高速化するためのコンパイラです。<a href="https://www.tensorflow.org/xla" rel="nofollow">公式ドキュメント</a>によれば、XLA（Accelerated Linear Algebra）は線形代数のためのドメイン固有のコンパイラで、TensorFlowモデルを潜在的にソースコードの変更なしで高速化できます。</p> <p data-svelte-h="svelte-17c4ps">TensorFlowでXLAを使用するのは簡単です。XLAは<code>tensorflow</code>ライブラリ内にパッケージ化されており、<a href="https://www.tensorflow.org/guide/intro_to_graphs" rel="nofollow"><code>tf.function</code></a>などのグラフを作成する関数内で<code>jit_compile</code>引数を使用してトリガーできます。<code>fit()</code>や<code>predict()</code>などのKerasメソッドを使用する場合、<code>model.compile()</code>に<code>jit_compile</code>引数を渡すだけでXLAを有効にできます。ただし、XLAはこれらのメソッドに限定されているわけではありません。任意の<code>tf.function</code>を高速化するためにも使用できます。</p> <p data-svelte-h="svelte-jmz72m">🤗 Transformers内のいくつかのTensorFlowメソッドは、XLAと互換性があるように書き直されています。これには、<a href="https://huggingface.co/docs/transformers/model_doc/gpt2" rel="nofollow">GPT2</a>、<a href="https://huggingface.co/docs/transformers/model_doc/t5" rel="nofollow">T5</a>、<a href="https://huggingface.co/docs/transformers/model_doc/opt" rel="nofollow">OPT</a>などのテキスト生成モデルや、<a href="https://huggingface.co/docs/transformers/model_doc/whisper" rel="nofollow">Whisper</a>などの音声処理モデルも含まれます。</p> <p data-svelte-h="svelte-163nszz">速度向上の具体的な量はモデルに非常に依存しますが、🤗 Transformers内のTensorFlowテキスト生成モデルでは、約100倍の速度向上を確認しています。このドキュメントでは、これらのモデルにXLAを使用して最大のパフォーマンスを得る方法を説明します。また、ベンチマークとXLA統合のデザイン哲学について詳しく学びたい場合の追加リソースへのリンクも提供します。</p> <h2 class="relative group"><a id="running-tf-functions-with-xla" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#running-tf-functions-with-xla"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>Running TF functions with XLA</span></h2> <p data-svelte-h="svelte-i2jpr9">以下のTensorFlowモデルを考えてみましょう：</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">import</span> tensorflow <span class="hljs-keyword">as</span> tf | |
| model = tf.keras.Sequential( | |
| [tf.keras.layers.Dense(<span class="hljs-number">10</span>, input_shape=(<span class="hljs-number">10</span>,), activation=<span class="hljs-string">"relu"</span>), tf.keras.layers.Dense(<span class="hljs-number">5</span>, activation=<span class="hljs-string">"softmax"</span>)] | |
| )<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-xhjq9d">上記のモデルは、次元が<code>(10, )</code>の入力を受け入れます。このモデルをフォワードパスで実行するには、次のようにします：</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-comment"># Generate random inputs for the model.</span> | |
| batch_size = <span class="hljs-number">16</span> | |
| input_vector_dim = <span class="hljs-number">10</span> | |
| random_inputs = tf.random.normal((batch_size, input_vector_dim)) | |
| <span class="hljs-comment"># Run a forward pass.</span> | |
| _ = model(random_inputs)<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-1xlk5bx">XLAでコンパイルされた関数を使用してフォワードパスを実行するには、以下のようにします：</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START -->xla_fn = tf.function(model, jit_compile=<span class="hljs-literal">True</span>) | |
| _ = xla_fn(random_inputs)<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-bfwqnu"><code>model</code>のデフォルトの <code>call()</code> 関数はXLAグラフをコンパイルするために使用されます。ただし、XLAにコンパイルしたい他のモデル関数がある場合、それも可能です。以下はその方法です：</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START -->my_xla_fn = tf.function(model.my_xla_fn, jit_compile=<span class="hljs-literal">True</span>)<!-- HTML_TAG_END --></pre></div> <h2 class="relative group"><a id="running-a-tf-text-generation-model-with-xla-from--transformers" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#running-a-tf-text-generation-model-with-xla-from--transformers"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>Running a TF text generation model with XLA from 🤗 Transformers</span></h2> <p data-svelte-h="svelte-1rj7w64">🤗 Transformers内でXLAでの高速化された生成を有効にするには、最新バージョンの<code>transformers</code>がインストールされている必要があります。次のコマンドを実行してインストールできます：</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START -->pip install transformers --upgrade<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-kzcg3w">次に、次のコードを実行できます：</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">import</span> tensorflow <span class="hljs-keyword">as</span> tf | |
| <span class="hljs-keyword">from</span> transformers <span class="hljs-keyword">import</span> AutoTokenizer, TFAutoModelForCausalLM | |
| <span class="hljs-comment"># Will error if the minimal version of Transformers is not installed.</span> | |
| <span class="hljs-keyword">from</span> transformers.utils <span class="hljs-keyword">import</span> check_min_version | |
| check_min_version(<span class="hljs-string">"4.21.0"</span>) | |
| tokenizer = AutoTokenizer.from_pretrained(<span class="hljs-string">"openai-community/gpt2"</span>, padding_side=<span class="hljs-string">"left"</span>, pad_token=<span class="hljs-string">"</s>"</span>) | |
| model = TFAutoModelForCausalLM.from_pretrained(<span class="hljs-string">"openai-community/gpt2"</span>) | |
| input_string = [<span class="hljs-string">"TensorFlow is"</span>] | |
| <span class="hljs-comment"># One line to create an XLA generation function</span> | |
| xla_generate = tf.function(model.generate, jit_compile=<span class="hljs-literal">True</span>) | |
| tokenized_input = tokenizer(input_string, return_tensors=<span class="hljs-string">"tf"</span>) | |
| generated_tokens = xla_generate(**tokenized_input, num_beams=<span class="hljs-number">2</span>) | |
| decoded_text = tokenizer.decode(generated_tokens[<span class="hljs-number">0</span>], skip_special_tokens=<span class="hljs-literal">True</span>) | |
| <span class="hljs-built_in">print</span>(<span class="hljs-string">f"Generated -- <span class="hljs-subst">{decoded_text}</span>"</span>) | |
| <span class="hljs-comment"># Generated -- TensorFlow is an open-source, open-source, distributed-source application # framework for the</span><!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-1gq39rt"><code>generate()</code>でXLAを有効にするのは、たった一行のコードです。コードの残り部分は変更されていません。ただし、XLA固有のいくつかの注意点が上記のコードスニペットにあります。これらに注意する必要があり、XLAがもたらす速度向上を実現するためにそれらを把握することが重要です。次のセクションでこれらについて詳しく説明します。</p> <h2 class="relative group"><a id="gotchas-to-be-aware-of" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#gotchas-to-be-aware-of"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>Gotchas to be aware of</span></h2> <p data-svelte-h="svelte-1u18xwm">XLAを有効にした関数（上記の<code>xla_generate()</code>など）を初めて実行すると、内部で計算グラフを推論しようとしますが、これは時間がかかります。このプロセスは<a href="https://www.tensorflow.org/guide/intro_to_graphs#when_is_a_function_tracing" rel="nofollow">“トレーシング”（tracing）</a>として知られています。</p> <p data-svelte-h="svelte-vj7dap">生成時間が高速ではないことに気付くかもしれません。<code>xla_generate()</code>（または他のXLA対応関数）の連続呼び出しでは、関数への入力が最初に計算グラフが構築されたときと同じ形状に従っている場合、計算グラフを推論する必要はありません。これは、入力形状が固定されているモダリティ（例：画像）には問題ありませんが、変数の入力形状モダリティ（例：テキスト）を扱う場合には注意が必要です。</p> <p data-svelte-h="svelte-10cmu1q"><code>xla_generate()</code>が常に同じ入力形状で動作するようにするには、トークナイザを呼び出す際に<code>padding</code>引数を指定できます。</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">import</span> tensorflow <span class="hljs-keyword">as</span> tf | |
| <span class="hljs-keyword">from</span> transformers <span class="hljs-keyword">import</span> AutoTokenizer, TFAutoModelForCausalLM | |
| tokenizer = AutoTokenizer.from_pretrained(<span class="hljs-string">"openai-community/gpt2"</span>, padding_side=<span class="hljs-string">"left"</span>, pad_token=<span class="hljs-string">"</s>"</span>) | |
| model = TFAutoModelForCausalLM.from_pretrained(<span class="hljs-string">"openai-community/gpt2"</span>) | |
| input_string = [<span class="hljs-string">"TensorFlow is"</span>] | |
| xla_generate = tf.function(model.generate, jit_compile=<span class="hljs-literal">True</span>) | |
| <span class="hljs-comment"># Here, we call the tokenizer with padding options.</span> | |
| tokenized_input = tokenizer(input_string, pad_to_multiple_of=<span class="hljs-number">8</span>, padding=<span class="hljs-literal">True</span>, return_tensors=<span class="hljs-string">"tf"</span>) | |
| generated_tokens = xla_generate(**tokenized_input, num_beams=<span class="hljs-number">2</span>) | |
| decoded_text = tokenizer.decode(generated_tokens[<span class="hljs-number">0</span>], skip_special_tokens=<span class="hljs-literal">True</span>) | |
| <span class="hljs-built_in">print</span>(<span class="hljs-string">f"Generated -- <span class="hljs-subst">{decoded_text}</span>"</span>)<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-q4vr22">これにより、<code>xla_generate()</code>への入力が常にトレースされた形状の入力を受け取ることを確認し、生成時間の高速化を実現できます。以下のコードでこれを確認できます：</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">import</span> time | |
| <span class="hljs-keyword">import</span> tensorflow <span class="hljs-keyword">as</span> tf | |
| <span class="hljs-keyword">from</span> transformers <span class="hljs-keyword">import</span> AutoTokenizer, TFAutoModelForCausalLM | |
| tokenizer = AutoTokenizer.from_pretrained(<span class="hljs-string">"openai-community/gpt2"</span>, padding_side=<span class="hljs-string">"left"</span>, pad_token=<span class="hljs-string">"</s>"</span>) | |
| model = TFAutoModelForCausalLM.from_pretrained(<span class="hljs-string">"openai-community/gpt2"</span>) | |
| xla_generate = tf.function(model.generate, jit_compile=<span class="hljs-literal">True</span>) | |
| <span class="hljs-keyword">for</span> input_string <span class="hljs-keyword">in</span> [<span class="hljs-string">"TensorFlow is"</span>, <span class="hljs-string">"TensorFlow is a"</span>, <span class="hljs-string">"TFLite is a"</span>]: | |
| tokenized_input = tokenizer(input_string, pad_to_multiple_of=<span class="hljs-number">8</span>, padding=<span class="hljs-literal">True</span>, return_tensors=<span class="hljs-string">"tf"</span>) | |
| start = time.time_ns() | |
| generated_tokens = xla_generate(**tokenized_input, num_beams=<span class="hljs-number">2</span>) | |
| end = time.time_ns() | |
| <span class="hljs-built_in">print</span>(<span class="hljs-string">f"Execution time -- <span class="hljs-subst">{(end - start) / <span class="hljs-number">1e6</span>:<span class="hljs-number">.1</span>f}</span> ms\n"</span>)<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-1pv1wyj">Tesla T4 GPUを使用すると、次のような出力が期待されます：</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START -->Execution time -- 30819.6 ms | |
| Execution time -- 79.0 ms | |
| Execution time -- 78.9 ms<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-1ja7r83">最初の<code>xla_generate()</code>呼び出しはトレーシングのために時間がかかりますが、連続する呼び出しは桁違いに高速です。生成オプションのいかなる変更も、再トレーシングを引き起こし、生成時間の遅延を引き起こすことに注意してください。</p> <p data-svelte-h="svelte-xpg5d5">このドキュメントでは、🤗 Transformersが提供するテキスト生成オプションをすべて網羅していません。高度なユースケースについてはドキュメンテーションを参照することをお勧めします。</p> <h2 class="relative group"><a id="additional-resources" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#additional-resources"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>Additional Resources</span></h2> <p data-svelte-h="svelte-ctf09q">ここでは、🤗 Transformersと一般的なXLAについてさらに詳しく学びたい場合のいくつかの追加リソースを提供します。</p> <ul data-svelte-h="svelte-ycl3uy"><li><a href="https://colab.research.google.com/github/huggingface/blog/blob/main/notebooks/91_tf_xla_generate.ipynb" rel="nofollow">このColab Notebook</a>では、XLA対応のエンコーダーデコーダー（<a href="https://huggingface.co/docs/transformers/model_doc/t5" rel="nofollow">T5</a>など）およびデコーダー専用（<a href="https://huggingface.co/docs/transformers/model_doc/gpt2" rel="nofollow">GPT2</a>など）テキスト生成モデルを試すための対話型デモが提供されています。</li> <li><a href="https://huggingface.co/blog/tf-xla-generate" rel="nofollow">このブログ記事</a>では、XLA対応モデルの比較ベンチマークの概要と、TensorFlowでのXLAについての友好的な紹介が提供されています。</li> <li><a href="https://blog.tensorflow.org/2022/11/how-hugging-face-improved-text-generation-performance-with-xla.html" rel="nofollow">このブログ記事</a>では、🤗 TransformersのTensorFlowモデルにXLAサポートを追加する際の設計哲学について説明しています。</li> <li>一般的なXLAとTensorFlowグラフについて詳しく学ぶためのおすすめの投稿：<ul><li><a href="https://www.tensorflow.org/xla" rel="nofollow">XLA: 機械学習用の最適化コンパイラ</a></li> <li><a href="https://www.tensorflow.org/guide/intro_to_graphs" rel="nofollow">グラフと<code>tf.function</code>の紹介</a></li> <li><a href="https://www.tensorflow.org/guide/function" rel="nofollow"><code>tf.function</code>を使用したパフォーマンス向上</a></li></ul></li></ul> <a class="!text-gray-400 !no-underline text-sm flex items-center not-prose mt-4" href="https://github.com/huggingface/transformers/blob/main/docs/source/ja/tf_xla.md" target="_blank"><span data-svelte-h="svelte-1kd6by1"><</span> <span data-svelte-h="svelte-x0xyl0">></span> <span data-svelte-h="svelte-1dajgef"><span class="underline ml-1.5">Update</span> on GitHub</span></a> <p></p> | |
| <script> | |
| { | |
| __sveltekit_jement = { | |
| assets: "/docs/transformers/main/ja", | |
| base: "/docs/transformers/main/ja", | |
| env: {} | |
| }; | |
| const element = document.currentScript.parentElement; | |
| const data = [null,null]; | |
| Promise.all([ | |
| import("/docs/transformers/main/ja/_app/immutable/entry/start.1486e459.js"), | |
| import("/docs/transformers/main/ja/_app/immutable/entry/app.d9ae818f.js") | |
| ]).then(([kit, app]) => { | |
| kit.start(app, element, { | |
| node_ids: [0, 162], | |
| data, | |
| form: null, | |
| error: null | |
| }); | |
| }); | |
| } | |
| </script> | |
Xet Storage Details
- Size:
- 35.8 kB
- Xet hash:
- 5d17b7d09735e787f389a906c5bd62da9bdc630fce15d5706d1108f1e6ff0522
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.