lighteval/pr_1326/en/package_reference/pipeline.md

Pipeline

Pipeline[[lighteval.pipeline.Pipeline]]

lighteval.pipeline.Pipeline[[lighteval.pipeline.Pipeline]]

lighteval.pipeline.Pipeline(tasks: str, pipeline_parameters: PipelineParameters, evaluation_tracker: EvaluationTracker, model_config: lighteval.models.abstract_model.ModelConfig | None = None, model = None, metric_options = None)

Source

PipelineParameters[[lighteval.pipeline.PipelineParameters]]

lighteval.pipeline.PipelineParameters[[lighteval.pipeline.PipelineParameters]]

lighteval.pipeline.PipelineParameters(launcher_type: ParallelismManager, job_id: int = 0, dataset_loading_processes: int = 1, nanotron_checkpoint_path: str | None = None, custom_tasks_directory: str | None = None, num_fewshot_seeds: int = 1, max_samples: int | None = None, cot_prompt: str | None = None, remove_reasoning_tags: bool = True, reasoning_tags: str | list[tuple[str, str]] = "[('<think>', '</think>')]", load_responses_from_details_date_id: str | None = None, bootstrap_iters: int = 1000, load_tasks_multilingual: bool = False)

Source

ParallelismManager[[lighteval.pipeline.ParallelismManager]]

lighteval.pipeline.ParallelismManager[[lighteval.pipeline.ParallelismManager]]

lighteval.pipeline.ParallelismManager(value, names = None, module = None, qualname = None, type = None, start = 1)

Source

An enumeration.