Experiment¶
Here is a tutorial on Google Colab that shows how to use the experiment module
Create Experiment¶
-
labml.experiment.
create
(*, uuid: Optional[str] = None, name: Optional[str] = None, python_file: Optional[str] = None, comment: Optional[str] = None, writers: Set[str] = None, ignore_callers: Set[str] = None, tags: Optional[Set[str]] = None, disable_screen: bool = False)[source]¶ Create an experiment
- Keyword Arguments
name (str, optional) – name of the experiment
python_file (str, optional) – path of the Python file that created the experiment
comment (str, optional) – a short description of the experiment
writers (Set[str], optional) – list of writers to write stat to. Defaults to
{'tensorboard', 'sqlite', 'web_api'}
.ignore_callers – (Set[str], optional): list of files to ignore when automatically determining
python_file
tags (Set[str], optional) – Set of tags for experiment
-
labml.experiment.
record
(*, name: Optional[str] = None, comment: Optional[str] = None, writers: Set[str] = None, tags: Optional[Set[str]] = None, exp_conf: Dict[str, any] = None, lab_conf: Dict[str, any] = None, token: str = None, disable_screen: bool = False)[source]¶ This combines
create()
,configs()
andstart()
.- Keyword Arguments
name (str, optional) – name of the experiment
comment (str, optional) – a short description of the experiment
writers (Set[str], optional) – list of writers to write stat to. Defaults to
{'tensorboard', 'sqlite', 'web_api'}
.tags (Set[str], optional) – Set of tags for experiment
exp_conf (Dict[str, any], optional) – a dictionary of experiment configurations
lab_conf (Dict[str, any], optional) – a dictionary of configurations for LabML. Use this if you want to change default configurations such as
web_api
, anddata_path
.token (str, optional) – a shortcut to provide LabML mobile app token (or url -
web_api
) instead of including it inlab_conf
. You can set this withlabml.lab.configure()
, or with a configuration file for the entire project.
-
labml.experiment.
evaluate
()[source]¶ This should be used for evaluation of a saved experiment. This will not record anything.
Checkpoints¶
-
labml.experiment.
load
(run_uuid: str, checkpoint: Optional[int] = None)[source]¶ Loads a the run from a previous checkpoint. You need to separately call
experiment.start
to start the experiment.- Parameters
run_uuid (str) – experiment will start from a saved state in the run with UUID
run_uuid
checkpoint (str, optional) – if provided the experiment will start from given checkpoint. Otherwise it will start from the last checkpoint.
-
labml.experiment.
load_models
(models: List[str], run_uuid: str, checkpoint: Optional[int] = None)[source]¶ Loads and starts the run from a previous checkpoint.
- Parameters
models (List[str]) – List of names of models to be loaded
run_uuid (str) – experiment will start from a saved state in the run with UUID
run_uuid
checkpoint (str, optional) – if provided the experiment will start from given checkpoint. Otherwise it will start from the last checkpoint.
-
labml.experiment.
add_pytorch_models
(*args, **kwargs)[source]¶ Set variables for saving and loading
- Parameters
models (Dict[str, torch.nn.Module]) – a dictionary of torch modules used in the experiment. These will be saved with
labml.experiment.save_checkpoint()
and loaded withlabml.experiment.load()
.
-
labml.experiment.
add_sklearn_models
(models: Dict[str, any])[source]¶ Warning
This is still experimental.
Set variables for saving and loading
- Parameters
models (Dict[str, any]) – a dictionary of SKLearn models These will be saved with
labml.experiment.save_checkpoint()
and loaded withlabml.experiment.load()
.
-
labml.experiment.
add_model_savers
(savers: Dict[str, labml.internal.experiment.ModelSaver])[source]¶ Add custom model savers
- Parameters
savers (Dict[str, ModelSaver]) – a dictionary of
labml.experiment.ModelSaver
. These will be saved withlabml.experiment.save_checkpoint()
and loaded withlabml.experiment.load()
.
-
class
labml.experiment.
ModelSaver
[source]¶ An abstract class defining model saver/loader.
The implementation should keep a reference to the model and load and save the model parameters.
Configurations & Hyper-parameters¶
-
labml.experiment.
configs
(*args)[source]¶ Calculate configurations
This has multiple overloads
-
labml.experiment.
configs
(conf_dict: Dict[str, any])[source]
-
labml.experiment.
configs
(conf_dict: Dict[str, any], conf_override: Dict[str, any])[source]
-
labml.experiment.
configs
(conf: BaseConfigs)[source]
-
labml.experiment.
configs
(conf: BaseConfigs, run_order: List[Union[List[str], str]])[source]
-
labml.experiment.
configs
(conf: BaseConfigs, *run_order: str)[source]
-
labml.experiment.
configs
(conf: BaseConfigs, conf_override: Dict[str, any])[source]
-
labml.experiment.
configs
(conf: BaseConfigs, conf_override: Dict[str, any], run_order: List[Union[List[str], str]])[source]
-
labml.experiment.
configs
(conf: BaseConfigs, conf_override: Dict[str, any], *run_order: str)[source]
- Parameters
conf (BaseConfigs, optional) – configurations object
conf_dict (Dict[str, any], optional) – a dictionary of configs
conf_override (Dict[str, any], optional) – a dictionary of configs to be overridden
-
-
labml.experiment.
load_configs
(run_uuid: str, *, is_only_hyperparam: bool = True)[source]¶ Load configs of a previous run
- Parameters
run_uuid (str) – if provided the experiment will start from a saved state in the run with UUID
run_uuid
- Keyword Arguments
is_only_hyperparam (bool, optional) – if True all only the hyper parameters are returned
Bundle checkpoints¶
-
labml.experiment.
save_bundle
(path: pathlib.Path, run_uuid: str, checkpoint: Optional[int] = None, *, data_files: List[str] = None)[source]¶ Create a
.tar.gz
file with the configs and checkpoints that can be distributed and loaded easily.- Parameters
path (Path) –
.tar.gz
file pathrun_uuid (str) – experiment run to bundle
checkpoint (str, optional) – if provided the given checkpoint will be bundled. Otherwise it will bundle the last checkpoint.
data_files – List of data files (relative to
labml.lab.get_data_path()
) to be bundled.