Buckets:
| <meta charset="utf-8" /><meta name="hf:doc:metadata" content="{"title":"Adding a Custom Task","local":"adding-a-custom-task","sections":[{"title":"Step by step creation of a custom task","local":"step-by-step-creation-of-a-custom-task","sections":[],"depth":2}],"depth":1}"> | |
| <link href="/docs/lighteval/pr_706/en/_app/immutable/assets/0.e3b0c442.css" rel="modulepreload"> | |
| <link rel="modulepreload" href="/docs/lighteval/pr_706/en/_app/immutable/entry/start.282ba36b.js"> | |
| <link rel="modulepreload" href="/docs/lighteval/pr_706/en/_app/immutable/chunks/scheduler.7da89386.js"> | |
| <link rel="modulepreload" href="/docs/lighteval/pr_706/en/_app/immutable/chunks/singletons.95bcf9b7.js"> | |
| <link rel="modulepreload" href="/docs/lighteval/pr_706/en/_app/immutable/chunks/paths.757e4909.js"> | |
| <link rel="modulepreload" href="/docs/lighteval/pr_706/en/_app/immutable/entry/app.aa546cdc.js"> | |
| <link rel="modulepreload" href="/docs/lighteval/pr_706/en/_app/immutable/chunks/index.20910acc.js"> | |
| <link rel="modulepreload" href="/docs/lighteval/pr_706/en/_app/immutable/nodes/0.1774b32f.js"> | |
| <link rel="modulepreload" href="/docs/lighteval/pr_706/en/_app/immutable/chunks/each.e59479a4.js"> | |
| <link rel="modulepreload" href="/docs/lighteval/pr_706/en/_app/immutable/nodes/2.bcc40225.js"> | |
| <link rel="modulepreload" href="/docs/lighteval/pr_706/en/_app/immutable/chunks/Tip.53e22153.js"> | |
| <link rel="modulepreload" href="/docs/lighteval/pr_706/en/_app/immutable/chunks/CodeBlock.143bd81e.js"> | |
| <link rel="modulepreload" href="/docs/lighteval/pr_706/en/_app/immutable/chunks/index.3af8f81c.js"><!-- HEAD_svelte-u9bgzb_START --><meta name="hf:doc:metadata" content="{"title":"Adding a Custom Task","local":"adding-a-custom-task","sections":[{"title":"Step by step creation of a custom task","local":"step-by-step-creation-of-a-custom-task","sections":[],"depth":2}],"depth":1}"><!-- HEAD_svelte-u9bgzb_END --> <p></p> <h1 class="relative group"><a id="adding-a-custom-task" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#adding-a-custom-task"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>Adding a Custom Task</span></h1> <p data-svelte-h="svelte-15bwdkb">To add a new task, first either open an issue, to determine whether it will be | |
| integrated in the core evaluations of lighteval, in the extended tasks, or the | |
| community tasks, and add its dataset on the hub.</p> <ul data-svelte-h="svelte-18vxpvw"><li>Core evaluations are evaluations that only require standard logic in their | |
| metrics and processing, and that we will add to our test suite to ensure non | |
| regression through time. They already see high usage in the community.</li> <li>Extended evaluations are evaluations that require custom logic in their | |
| metrics (complex normalisation, an LLM as a judge, …), that we added to | |
| facilitate the life of users. They already see high usage in the community.</li> <li>Community evaluations are submissions by the community of new tasks.</li></ul> <p data-svelte-h="svelte-c3h151">A popular community evaluation can move to become an extended or core evaluation over time.</p> <div class="course-tip bg-gradient-to-br dark:bg-gradient-to-r before:border-green-500 dark:before:border-green-800 from-green-50 dark:from-gray-900 to-white dark:to-gray-950 border border-green-50 text-green-700 dark:text-gray-400"><p data-svelte-h="svelte-1t4f4pd">You can find examples of custom tasks in the <a href="https://github.com/huggingface/lighteval/tree/main/community_tasks">community_task</a> directory.</p></div> <h2 class="relative group"><a id="step-by-step-creation-of-a-custom-task" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#step-by-step-creation-of-a-custom-task"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>Step by step creation of a custom task</span></h2> <div class="course-tip course-tip-orange bg-gradient-to-br dark:bg-gradient-to-r before:border-orange-500 dark:before:border-orange-800 from-orange-50 dark:from-gray-900 to-white dark:to-gray-950 border border-orange-50 text-orange-700 dark:text-gray-400"><p data-svelte-h="svelte-l65q4m">To contribute your custom metric to the lighteval repo, you would first need | |
| to install the required dev dependencies by running <code>pip install -e .[dev]</code> | |
| and then run <code>pre-commit install</code> to install the pre-commit hooks.</p></div> <p data-svelte-h="svelte-4w3uiu">First, create a python file under the <code>community_tasks</code> directory.</p> <p data-svelte-h="svelte-dnqoqp">You need to define a prompt function that will convert a line from your | |
| dataset to a document to be used for evaluation.</p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-comment"># Define as many as you need for your different tasks</span> | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">prompt_fn</span>(<span class="hljs-params">line, task_name: <span class="hljs-built_in">str</span> = <span class="hljs-literal">None</span></span>): | |
| <span class="hljs-string">"""Defines how to go from a dataset line to a doc object. | |
| Follow examples in src/lighteval/tasks/default_prompts.py, or get more info | |
| about what this function should do in the README. | |
| """</span> | |
| <span class="hljs-keyword">return</span> Doc( | |
| task_name=task_name, | |
| query=line[<span class="hljs-string">"question"</span>], | |
| choices=[<span class="hljs-string">f" <span class="hljs-subst">{c}</span>"</span> <span class="hljs-keyword">for</span> c <span class="hljs-keyword">in</span> line[<span class="hljs-string">"choices"</span>]], | |
| gold_index=line[<span class="hljs-string">"gold"</span>], | |
| instruction=<span class="hljs-string">""</span>, | |
| )<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-16x99kz">Then, you need to choose a metric: you can either use an existing one (defined | |
| in <code>lighteval.metrics.metrics.Metrics</code>) or <a href="adding-a-new-metric">create a custom one</a>).</p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START -->custom_metric = SampleLevelMetric( | |
| metric_name=<span class="hljs-string">"my_custom_metric_name"</span>, | |
| higher_is_better=<span class="hljs-literal">True</span>, | |
| category=MetricCategory.IGNORED, | |
| use_case=MetricUseCase.NONE, | |
| sample_level_fn=<span class="hljs-keyword">lambda</span> x: x, <span class="hljs-comment"># how to compute score for one sample</span> | |
| corpus_level_fn=np.mean, <span class="hljs-comment"># How to aggregate the samples metrics</span> | |
| )<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-i9ana">Then, you need to define your task using <a href="/docs/lighteval/pr_706/en/package_reference/tasks#lighteval.tasks.lighteval_task.LightevalTaskConfig">LightevalTaskConfig</a>. | |
| You can define a task with or without subsets. | |
| To define a task with no subsets:</p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-comment"># This is how you create a simple task (like hellaswag) which has one single subset</span> | |
| <span class="hljs-comment"># attached to it, and one evaluation possible.</span> | |
| task = LightevalTaskConfig( | |
| name=<span class="hljs-string">"myothertask"</span>, | |
| prompt_function=prompt_fn, <span class="hljs-comment"># must be defined in the file or imported from src/lighteval/tasks/tasks_prompt_formatting.py</span> | |
| suite=[<span class="hljs-string">"community"</span>], | |
| hf_repo=<span class="hljs-string">""</span>, | |
| hf_subset=<span class="hljs-string">"default"</span>, | |
| hf_avail_splits=[], | |
| evaluation_splits=[], | |
| few_shots_split=<span class="hljs-literal">None</span>, | |
| few_shots_select=<span class="hljs-literal">None</span>, | |
| metric=[], <span class="hljs-comment"># select your metric in Metrics</span> | |
| )<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-1r4bpbw">If you want to create a task with multiple subset, add them to the | |
| <code>SAMPLE_SUBSETS</code> list and create a task for each subset.</p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START -->SAMPLE_SUBSETS = [] <span class="hljs-comment"># list of all the subsets to use for this eval</span> | |
| <span class="hljs-keyword">class</span> <span class="hljs-title class_">CustomSubsetTask</span>(<span class="hljs-title class_ inherited__">LightevalTaskConfig</span>): | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">__init__</span>(<span class="hljs-params"> | |
| self, | |
| name, | |
| hf_subset, | |
| </span>): | |
| <span class="hljs-built_in">super</span>().__init__( | |
| name=name, | |
| hf_subset=hf_subset, | |
| prompt_function=prompt_fn, <span class="hljs-comment"># must be defined in the file or imported from src/lighteval/tasks/tasks_prompt_formatting.py</span> | |
| hf_repo=<span class="hljs-string">""</span>, | |
| metric=[custom_metric], <span class="hljs-comment"># select your metric in Metrics or use your custom_metric</span> | |
| hf_avail_splits=[], | |
| evaluation_splits=[], | |
| few_shots_split=<span class="hljs-literal">None</span>, | |
| few_shots_select=<span class="hljs-literal">None</span>, | |
| suite=[<span class="hljs-string">"community"</span>], | |
| generation_size=-<span class="hljs-number">1</span>, | |
| stop_sequence=<span class="hljs-literal">None</span>, | |
| ) | |
| SUBSET_TASKS = [CustomSubsetTask(name=<span class="hljs-string">f"mytask:<span class="hljs-subst">{subset}</span>"</span>, hf_subset=subset) <span class="hljs-keyword">for</span> subset <span class="hljs-keyword">in</span> SAMPLE_SUBSETS]<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-th00fz">Here is a list of the parameters and their meaning:</p> <ul data-svelte-h="svelte-1i36g3q"><li><code>name</code> (str), your evaluation name</li> <li><code>suite</code> (list), the suite(s) to which your evaluation should belong. This | |
| field allows us to compare different task implementations and is used as a | |
| task selection to differentiate the versions to launch. At the moment, you’ll | |
| find the keywords [“helm”, “bigbench”, “original”, “lighteval”, “community”, | |
| “custom”]; for core evals, please choose <code>lighteval</code>.</li> <li><code>prompt_function</code> (Callable), the prompt function you defined in the step | |
| above</li> <li><code>hf_repo</code> (str), the path to your evaluation dataset on the hub</li> <li><code>hf_subset</code> (str), the specific subset you want to use for your evaluation | |
| (note: when the dataset has no subset, fill this field with <code>"default"</code>, not | |
| with <code>None</code> or <code>""</code>)</li> <li><code>hf_avail_splits</code> (list), all the splits available for your dataset (train, | |
| valid or validation, test, other…)</li> <li><code>evaluation_splits</code> (list), the splits you want to use for evaluation</li> <li><code>few_shots_split</code> (str, can be <code>null</code>), the specific split from which you | |
| want to select samples for your few-shot examples. It should be different | |
| from the sets included in <code>evaluation_splits</code></li> <li><code>few_shots_select</code> (str, can be <code>null</code>), the method that you will use to | |
| select items for your few-shot examples. Can be <code>null</code>, or one of:<ul><li><code>balanced</code> select examples from the <code>few_shots_split</code> with balanced | |
| labels, to avoid skewing the few shot examples (hence the model | |
| generations) toward one specific label</li> <li><code>random</code> selects examples at random from the <code>few_shots_split</code></li> <li><code>random_sampling</code> selects new examples at random from the | |
| <code>few_shots_split</code> for every new item, but if a sampled item is equal to | |
| the current one, it is removed from the available samples</li> <li><code>random_sampling_from_train</code> selects new examples at random from the | |
| <code>few_shots_split</code> for every new item, but if a sampled item is equal to | |
| the current one, it is kept! Only use this if you know what you are | |
| doing.</li> <li><code>sequential</code> selects the first <code>n</code> examples of the <code>few_shots_split</code></li></ul></li> <li><code>generation_size</code> (int), the maximum number of tokens allowed for a | |
| generative evaluation. If your evaluation is a log likelihood evaluation | |
| (multi-choice), this value should be -1</li> <li><code>stop_sequence</code> (list), a list of strings acting as end of sentence tokens | |
| for your generation</li> <li><code>metric</code> (list), the metrics you want to use for your evaluation (see next | |
| section for a detailed explanation)</li> <li><code>trust_dataset</code> (bool), set to True if you trust the dataset.</li></ul> <p data-svelte-h="svelte-7umcxy">Then you need to add your task to the <code>TASKS_TABLE</code> list.</p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-comment"># STORE YOUR EVALS</span> | |
| <span class="hljs-comment"># tasks with subset:</span> | |
| TASKS_TABLE = SUBSET_TASKS | |
| <span class="hljs-comment"># tasks without subset:</span> | |
| <span class="hljs-comment"># TASKS_TABLE = [task]</span><!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-41yve7">Once your file is created you can then run the evaluation with the following command:</p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START -->lighteval accelerate \ | |
| <span class="hljs-string">"model_name=HuggingFaceH4/zephyr-7b-beta"</span> \ | |
| <span class="hljs-string">"community|{custom_task}|{fewshots}|{truncate_few_shot}"</span> \ | |
| --custom-tasks {path_to_your_custom_task_file}<!-- HTML_TAG_END --></pre></div> <a class="!text-gray-400 !no-underline text-sm flex items-center not-prose mt-4" href="https://github.com/huggingface/lighteval/blob/main/docs/source/adding-a-custom-task.mdx" target="_blank"><span data-svelte-h="svelte-1kd6by1"><</span> <span data-svelte-h="svelte-x0xyl0">></span> <span data-svelte-h="svelte-1dajgef"><span class="underline ml-1.5">Update</span> on GitHub</span></a> <p></p> | |
| <script> | |
| { | |
| __sveltekit_kkv2dy = { | |
| assets: "/docs/lighteval/pr_706/en", | |
| base: "/docs/lighteval/pr_706/en", | |
| env: {} | |
| }; | |
| const element = document.currentScript.parentElement; | |
| const data = [null,null]; | |
| Promise.all([ | |
| import("/docs/lighteval/pr_706/en/_app/immutable/entry/start.282ba36b.js"), | |
| import("/docs/lighteval/pr_706/en/_app/immutable/entry/app.aa546cdc.js") | |
| ]).then(([kit, app]) => { | |
| kit.start(app, element, { | |
| node_ids: [0, 2], | |
| data, | |
| form: null, | |
| error: null | |
| }); | |
| }); | |
| } | |
| </script> | |
Xet Storage Details
- Size:
- 23.7 kB
- Xet hash:
- ebbea5c70afcf303427a485055fa0b668ac7ea3651540fa855079441e2ecb464
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.