Model Parallelism
Contents
Model Parallelism
You can specify your own model parallelism related configuration under model_parallelism
like:
{
"model_parallelism": {
"enable": "bool",
"tensor_parallel_size": "int"
}
}
2. tensor_parallel_size: int
type: int
default: 1
This means tensor model parallelism degree. If you don’t know what tensor model parallelism is, you can find the detail of it from here.
Note there are some rules for the tensor parallel size.
tensor_parallel_size
must be same or smaller than total number of gpus.tensor_parallel_size
must be power of 2. (e.g. 2, 4, 8, 16, …)tensor_parallel_size
must be positive number.
SUPPORTED MODELS
The following list includes currently supported models.
Albert
Bert
Bart
Blenderbot
BlenderbotSmall
T5
GPT2
GPTNeo
GPTJ
Electra
Roberta
We will support almost all models in the near future ;)