Model Parallelism

You can specify your own model parallelism related configuration under model_parallelism like:

{
  "model_parallelism": {
    "tensor_parallel_size": int    
  }
}

tensor_parallel_size: int

This means tensor model parallelism degree. If you don’t know what tensor model parallelism is, you can find the detail of it from here.

Note there are some rules for the tensor parallel size.

  • tensor_parallel_size must be same or smaller than total number of gpus.

  • tensor_parallel_size must be power of 2. (e.g. 2, 4, 8, 16, …)

  • tensor_parallel_size must be positive number.

SUPPORTED MODELS

The following list includes currently supported models.

  • Albert

  • Bert

  • Bart

  • T5

  • GPT2

  • GPTNeo

  • GPTJ

  • Electra

  • Roberta

We will support almost all models in the near future ;)