Activation Checkpointing
Contents
Activation Checkpointing¶
You can specify your own activation checkpointing related configuration under activation_checkpointing
like:
{
"activation_checkpointing": {
"enable": "bool",
"cpu_checkpointing": "bool",
"partitioned_checkpointing": "bool",
"contiguous_checkpointing": "bool"
}
}
3. partitioned_checkpointing: bool
¶
type: bool
default: False
Enable partitioned checkpointing.
Note that this is only available when you are using tensor model parallelism.
4. contiguous_checkpointing: bool
¶
Enable contiguous checkpointing.
Note that this is only available when you are using partitioned checkpointing.