---
tags:
- ColBERT
- PyLate
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:1188486
- loss:Contrastive
base_model: jhu-clsp/ettin-encoder-17m
datasets:
- benjamintli/code-retrieval-combined-v2-llm-negatives
pipeline_tag: sentence-similarity
library_name: PyLate
---
# PyLate model based on jhu-clsp/ettin-encoder-17m
This is a [PyLate](https://github.com/lightonai/pylate) model finetuned from [jhu-clsp/ettin-encoder-17m](https://huggingface.co/jhu-clsp/ettin-encoder-17m) on the [code-retrieval-combined-v2-llm-negatives](https://huggingface.co/datasets/benjamintli/code-retrieval-combined-v2-llm-negatives) dataset. It maps sentences & paragraphs to sequences of 128-dimensional dense vectors and can be used for semantic textual similarity using the MaxSim operator.
## Model Details
### Model Description
- **Model Type:** PyLate model
- **Base model:** [jhu-clsp/ettin-encoder-17m](https://huggingface.co/jhu-clsp/ettin-encoder-17m)
- **Document Length:** 180 tokens
- **Query Length:** 32 tokens
- **Output Dimensionality:** 128 tokens
- **Similarity Function:** MaxSim
- **Training Dataset:**
- [code-retrieval-combined-v2-llm-negatives](https://huggingface.co/datasets/benjamintli/code-retrieval-combined-v2-llm-negatives)
### Model Sources
- **Documentation:** [PyLate Documentation](https://lightonai.github.io/pylate/)
- **Repository:** [PyLate on GitHub](https://github.com/lightonai/pylate)
- **Hugging Face:** [PyLate models on Hugging Face](https://huggingface.co/models?library=PyLate)
### Full Model Architecture
```
ColBERT(
(0): Transformer({'max_seq_length': 31, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
(1): Dense({'in_features': 256, 'out_features': 128, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'use_residual': False})
)
```
## Usage
First install the PyLate library:
```bash
pip install -U pylate
```
### Retrieval
Use this model with PyLate to index and retrieve documents. The index uses [FastPLAID](https://github.com/lightonai/fast-plaid) for efficient similarity search.
#### Indexing documents
Load the ColBERT model and initialize the PLAID index, then encode and index your documents:
```python
from pylate import indexes, models, retrieve
# Step 1: Load the ColBERT model
model = models.ColBERT(
model_name_or_path="colbert-code-17m",
)
# Step 2: Initialize the PLAID index
index = indexes.PLAID(
index_folder="pylate-index",
index_name="index",
override=True, # This overwrites the existing index if any
)
# Step 3: Encode the documents
documents_ids = ["1", "2", "3"]
documents = ["document 1 text", "document 2 text", "document 3 text"]
documents_embeddings = model.encode(
documents,
batch_size=32,
is_query=False, # Ensure that it is set to False to indicate that these are documents, not queries
show_progress_bar=True,
)
# Step 4: Add document embeddings to the index by providing embeddings and corresponding ids
index.add_documents(
documents_ids=documents_ids,
documents_embeddings=documents_embeddings,
)
```
Note that you do not have to recreate the index and encode the documents every time. Once you have created an index and added the documents, you can re-use the index later by loading it:
```python
# To load an index, simply instantiate it with the correct folder/name and without overriding it
index = indexes.PLAID(
index_folder="pylate-index",
index_name="index",
)
```
#### Retrieving top-k documents for queries
Once the documents are indexed, you can retrieve the top-k most relevant documents for a given set of queries.
To do so, initialize the ColBERT retriever with the index you want to search in, encode the queries and then retrieve the top-k documents to get the top matches ids and relevance scores:
```python
# Step 1: Initialize the ColBERT retriever
retriever = retrieve.ColBERT(index=index)
# Step 2: Encode the queries
queries_embeddings = model.encode(
["query for document 3", "query for document 1"],
batch_size=32,
is_query=True, # # Ensure that it is set to False to indicate that these are queries
show_progress_bar=True,
)
# Step 3: Retrieve top-k documents
scores = retriever.retrieve(
queries_embeddings=queries_embeddings,
k=10, # Retrieve the top 10 matches for each query
)
```
### Reranking
If you only want to use the ColBERT model to perform reranking on top of your first-stage retrieval pipeline without building an index, you can simply use rank function and pass the queries and documents to rerank:
```python
from pylate import rank, models
queries = [
"query A",
"query B",
]
documents = [
["document A", "document B"],
["document 1", "document C", "document B"],
]
documents_ids = [
[1, 2],
[1, 3, 2],
]
model = models.ColBERT(
model_name_or_path="colbert-code-17m",
)
queries_embeddings = model.encode(
queries,
is_query=True,
)
documents_embeddings = model.encode(
documents,
is_query=False,
)
reranked_documents = rank.rerank(
documents_ids=documents_ids,
queries_embeddings=queries_embeddings,
documents_embeddings=documents_embeddings,
)
```
## Training Details
### Training Dataset
#### code-retrieval-combined-v2-llm-negatives
* Dataset: [code-retrieval-combined-v2-llm-negatives](https://huggingface.co/datasets/benjamintli/code-retrieval-combined-v2-llm-negatives) at [1917069](https://huggingface.co/datasets/benjamintli/code-retrieval-combined-v2-llm-negatives/tree/19170694c88339259e15b3dcc7f99db2136bd00a)
* Size: 1,188,486 training samples
* Columns: query, positive, source, hard_negatives, and negatives
* Approximate statistics based on the first 1000 samples:
| | query | positive | source | hard_negatives | negatives |
|:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|:-----------------------------------|:-----------------------------------------------------------------------------------|
| type | string | string | string | list | string |
| details | - min: 6 tokens
- mean: 23.61 tokens
- max: 32 tokens
| - min: 14 tokens
- mean: 31.43 tokens
- max: 32 tokens
| - min: 4 tokens
- mean: 7.46 tokens
- max: 10 tokens
| | - min: 16 tokens
- mean: 31.15 tokens
- max: 32 tokens
|
* Samples:
| query | positive | source | hard_negatives | negatives |
|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| wait for AWS PCA CSR propagation before issuing certificate | func (c *ACMPCA) WaitUntilCertificateAuthorityCSRCreated(input *GetCertificateAuthorityCsrInput) error {
return c.WaitUntilCertificateAuthorityCSRCreatedWithContext(aws.BackgroundContext(), input)
} | csn_syntethic | ['func (c *ACMPCA) WaitUntilAuditReportCreated(input *DescribeCertificateAuthorityAuditReportInput) error {\n\treturn c.WaitUntilAuditReportCreatedWithContext(aws.BackgroundContext(), input)\n}'] | func (c *ACMPCA) WaitUntilAuditReportCreated(input *DescribeCertificateAuthorityAuditReportInput) error {
return c.WaitUntilAuditReportCreatedWithContext(aws.BackgroundContext(), input)
} |
| func (f *Filter) Gather() ([]*dto.MetricFamily, error) {
mfs, err := f.Gatherer.Gather()
if err != nil {
| return nil, err
}
return f.Matcher.Match(mfs), nil
} | csn_ccr | ['\n\tf.err = err\n\tif err != nil {\n\t\tfor _, node := range f.cells {\n\t\t\tnode.PropagateWatchError(err)\n\t\t}\n\t}\n}'] |
f.err = err
if err != nil {
for _, node := range f.cells {
node.PropagateWatchError(err)
}
}
}def latent_to_dist(name, x, hparams, output_channels=None):
"""Map latent to the mean and log-scale of a Gaussian.
Args:
name: variable scope.
x: 4-D Tensor of shape (NHWC)
hparams: HParams.
latent_architecture - can be "single_conv", "glow_nn" or "glow_resnet",
default = single_conv
latent_encoder_depth - int, depth of architecture, valid if
latent_architecture is "glow_nn" or "glow_resnet".
latent_pre_output_channels - 512, valid only when latent_architecture
is "glow_nn".
latent_encoder_width - 512, maximum width of the network
output_channels: int, number of output channels of the mean (and std).
if not provided, set it to be the output channels of x.
Returns:
dist: instance of tfp.distributions.Normal
Raises:
ValueError: If architecture not in ["single_conv", "glow_nn"]
"""
architecture = hparams.get("latent_a... | mid_channels=mid_channels)
mean_log_scale = conv("glow_nn_zeros", mean_log_scale,
filter_size=[3, 3], stride=[1, 1],
output_channels=2*output_channels,
apply_actnorm=False, conv_init="zeros")
elif architecture == "glow_resnet":
h = x
for layer in range(depth):
h3 = conv_stack("latent_resnet_%d" % layer, h,
mid_channels=width, output_channels=x_shape[-1],
dropout=hparams.coupling_dropout)
h += h3
mean_log_scale = conv("glow_res_final", h, conv_init="zeros",
output_channels=2*output_channels,
apply_actnorm=False)
else:
raise ValueError("expected architecture to be single_conv or glow_nn "
"got %s" % architecture)
mean = mean_log_scale[:, :, :, 0::2]
log_scale = mean_log_scale[:, :, :, 1::2]
return tfp.distribu... | csn_ccr | [' first_relu=False,\n padding="SAME",\n strides=(2, 2),\n force2d=True,\n name="conv0")\n x = common_layers.conv_block(\n x, 64, [((1, 1), (3, 3))], padding="SAME", force2d=True, name="conv1")\n x = xnet_resblock(x, min(128, hidden_dim), True, "block0")\n x = xnet_resblock(x, min(256, hidden_dim), False, "block1")\n return xnet_resblock(x, hidden_dim, False, "block2")'] | first_relu=False,
padding="SAME",
strides=(2, 2),
force2d=True,
name="conv0")
x = common_layers.conv_block(
x, 64, [((1, 1), (3, 3))], padding="SAME", force2d=True, name="conv1")
x = xnet_resblock(x, min(128, hidden_dim), True, "block0")
x = xnet_resblock(x, min(256, hidden_dim), False, "block1")
return xnet_resblock(x, hidden_dim, False, "block2") |
* Loss: pylate.losses.contrastive.Contrastive
### Evaluation Dataset
#### code-retrieval-combined-v2-llm-negatives
* Dataset: [code-retrieval-combined-v2-llm-negatives](https://huggingface.co/datasets/benjamintli/code-retrieval-combined-v2-llm-negatives) at [1917069](https://huggingface.co/datasets/benjamintli/code-retrieval-combined-v2-llm-negatives/tree/19170694c88339259e15b3dcc7f99db2136bd00a)
* Size: 12,005 evaluation samples
* Columns: query, positive, source, hard_negatives, and negatives
* Approximate statistics based on the first 1000 samples:
| | query | positive | source | hard_negatives | negatives |
|:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|:-----------------------------------|:-----------------------------------------------------------------------------------|
| type | string | string | string | list | string |
| details | - min: 6 tokens
- mean: 23.78 tokens
- max: 32 tokens
| - min: 14 tokens
- mean: 31.45 tokens
- max: 32 tokens
| - min: 4 tokens
- mean: 7.55 tokens
- max: 10 tokens
| | - min: 14 tokens
- mean: 31.15 tokens
- max: 32 tokens
|
* Samples:
| query | positive | source | hard_negatives | negatives |
|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| public static function list_templates($competencyid, $onlyvisible) {
global $DB;
$sql = 'SELECT tpl.*
FROM {' . template::TABLE . '} tpl
JOIN {' . self::TABLE . '} tplcomp
ON tplcomp.templateid = tpl.id
WHERE tplcomp.competencyid = ? ';
$params = array($competencyid);
if ($onlyvisible) {
$sql .= ' AND tpl.visible = ?';
$params[] = 1;
}
| $sql .= ' ORDER BY tpl.id ASC';
$results = $DB->get_records_sql($sql, $params);
$instances = array();
foreach ($results as $result) {
array_push($instances, new template(0, $result));
}
return $instances;
} | csn_ccr | [" FROM {assign_grades} g\n JOIN(' . $esql . ') e ON e.id = g.userid\n WHERE g.assignment = :assignid';\n\n return $DB->count_records_sql($sql, $params);\n }"] | FROM {assign_grades} g
JOIN(' . $esql . ') e ON e.id = g.userid
WHERE g.assignment = :assignid';
return $DB->count_records_sql($sql, $params);
} |
| python multiprocessing sandboxed process isolation file system network | def start(self):
'''Create a process in which the isolated code will be run.'''
assert self._client is None
logger.debug('IsolationContext[%d] starting', id(self))
# Create the queues
request_queue = multiprocessing.Queue()
response_queue = multiprocessing.Queue()
# Launch the server process
server = Server(request_queue, response_queue) # Do not keep a reference to this object!
server_process = multiprocessing.Process(target=server.loop)
server_process.start()
# Create a client to talk to the server
self._client = Client(server_process, request_queue, response_queue) | csn_syntethic | ['def start(cls, _init_logging=True):\n """\n Arrange for the subprocess to be started, if it is not already running.\n\n The parent process picks a UNIX socket path the child will use prior to\n fork, creates a socketpair used essentially as a semaphore, then blocks\n waiting for the child to indicate the UNIX socket is ready for use.\n\n :param bool _init_logging:\n For testing, if :data:`False`, don\'t initialize logging.\n """\n if cls.worker_sock is not None:\n return\n\n if faulthandler is not None:\n faulthandler.enable()\n\n mitogen.utils.setup_gil()\n cls.unix_listener_path = mitogen.unix.make_socket_path()\n cls.worker_sock, cls.child_sock = socket.socketpair()\n atexit.register(lambda: clean_shutdown(cls.worker_sock))\n mitogen.core.set_cloexec(cls.worker_sock.fileno())\n mitogen.core.set_cloexec(cls.child_sock.fileno())\n\n cls.profiling = os.environ.get(\'MITOGEN_PROFILING\') is not None\n if cls.profiling:\n mitogen.core.enable_profiling()\n if _init_logging:\n ansible_mitogen.logging.setup()\n\n cls.original_env = dict(os.environ)\n cls.child_pid = os.fork()\n if cls.child_pid:\n save_pid(\'controller\')\n ansible_mitogen.logging.set_process_name(\'top\')\n ansible_mitogen.affinity.policy.assign_controller()\n cls.child_sock.close()\n cls.child_sock = None\n mitogen.core.io_op(cls.worker_sock.recv, 1)\n else:\n save_pid(\'mux\')\n ansible_mitogen.logging.set_process_name(\'mux\')\n ansible_mitogen.affinity.policy.assign_muxprocess()\n cls.worker_sock.close()\n cls.worker_sock = None\n self = cls()\n self.worker_main()'] | def start(cls, _init_logging=True):
"""
Arrange for the subprocess to be started, if it is not already running.
The parent process picks a UNIX socket path the child will use prior to
fork, creates a socketpair used essentially as a semaphore, then blocks
waiting for the child to indicate the UNIX socket is ready for use.
:param bool _init_logging:
For testing, if :data:`False`, don't initialize logging.
"""
if cls.worker_sock is not None:
return
if faulthandler is not None:
faulthandler.enable()
mitogen.utils.setup_gil()
cls.unix_listener_path = mitogen.unix.make_socket_path()
cls.worker_sock, cls.child_sock = socket.socketpair()
atexit.register(lambda: clean_shutdown(cls.worker_sock))
mitogen.core.set_cloexec(cls.worker_sock.fileno())
mitogen.core.set_cloexec(cls.child_sock.fileno())
cls.profiling = os.environ.get('MI... |
| updates the current cookies with a new set
@param array $cookies new cookies with which to update current ones
@return boolean always return true
@access private | function UpdateCookies($cookies)
{
if (sizeof($this->cookies) == 0) {
// no existing cookies: take whatever is new
if (sizeof($cookies) > 0) {
$this->debug('Setting new cookie(s)');
$this->cookies = $cookies;
}
return TRUE;
}
if (sizeof($cookies) == 0) {
// no new cookies: keep what we've got
return TRUE;
}
// merge
foreach ($cookies as $newCookie) {
if (!is_array($newCookie)) {
continue;
}
if ((!isset($newCookie['name'])) \|\| (!isset($newCookie['value']))) {
continue;
}
$newName = $newCookie['name'];
$found = FALSE;
for ($i = 0; $i < count($this->cookies); $i++) {
$cookie = $this->cookies[$i];
if (!is_array($cookie)) {
continue;
... | csn | ["public function setCookies(array $cookies) : Request\n {\n $query = http_build_query($cookies);\n parse_str($query, $this->cookies);\n\n if ($cookies) {\n $cookie = str_replace('&', '; ', $query);\n $this->setHeader('Cookie', $cookie);\n } else {\n $this->removeHeader('Cookie');\n }\n\n return $this;\n }"] | public function setCookies(array $cookies) : Request
{
$query = http_build_query($cookies);
parse_str($query, $this->cookies);
if ($cookies) {
$cookie = str_replace('&', '; ', $query);
$this->setHeader('Cookie', $cookie);
} else {
$this->removeHeader('Cookie');
}
return $this;
} |
* Loss: pylate.losses.contrastive.Contrastive
### Training Hyperparameters
#### Non-Default Hyperparameters
- `eval_strategy`: steps
- `per_device_train_batch_size`: 256
- `per_device_eval_batch_size`: 256
- `learning_rate`: 3e-06
- `num_train_epochs`: 1
- `fp16`: True
- `hub_model_id`: colbert-code-17m
#### All Hyperparameters
Click to expand
- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: steps
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 256
- `per_device_eval_batch_size`: 256
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 3e-06
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 1
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.0
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: True
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `parallelism_config`: None
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch_fused
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: colbert-code-17m
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `hub_revision`: None
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`:
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `liger_kernel_config`: None
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: None
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: proportional
- `router_mapping`: {}
- `learning_rate_mapping`: {}