File size: 1,083 Bytes
feba2ad
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
"""
Pico Config Package

The modules of this package are where you can specify the hyperparameters for the Pico model,
the dataset, the training process, evaluation, etc.

As with anything else in Pico, we've designed for the configuration setup to be as flexible
as possible. By default the configs are implemented as vanilla dataclasses -- this makes it easy to
switch to different config management systems if you want, like hydra.

Some things to NOTE:
- All hyperparameters are initialized with default values, which can be overridden.
- The default vocab size is set to the size of the OLMo tokenizer.
"""

# For convenience, we export the config classes here
from .checkpointing_config import CheckpointingConfig
from .data_config import DataConfig
from .evaluation_config import EvaluationConfig
from .model_config import ModelConfig
from .monitoring_config import MonitoringConfig
from .training_config import TrainingConfig

__all__ = [
    "CheckpointingConfig",
    "DataConfig",
    "EvaluationConfig",
    "ModelConfig",
    "MonitoringConfig",
    "TrainingConfig",
]