Struct CompileSpec::Input ¶
-
Defined in File trtorch.h
Nested Relationships ¶
This struct is a nested type of Struct CompileSpec .
Struct Documentation ¶
-
struct
trtorch:: CompileSpec
::
Input
-
A struct to hold an input range (used by TensorRT Optimization profile)
This struct can either hold a single vector representing an input shape, signifying a static input shape or a set of three input shapes representing the min, optiminal and max input shapes allowed for the engine.
Public Functions
-
Input
( std::vector<int64_t> shape , TensorFormat format = TensorFormat :: kContiguous )
-
Construct a new Input spec object for static input size from vector, optional arguments allow the user to configure expected input shape tensor format. dtype (Expected data type for the input) defaults to PyTorch / traditional TRT convection (FP32 for FP32 only, FP16 for FP32 and FP16, FP32 for Int8)
- Parameters
-
-
shape
: Input tensor shape -
dtype
: Expected data type for the input (Defaults to Float32) -
format
: Expected tensor format for the input (Defaults to contiguous)
-
-
Input
( std::vector<int64_t> shape , DataType dtype , TensorFormat format = TensorFormat :: kContiguous )
-
Construct a new Input spec object for static input size from vector, optional arguments allow the user to configure expected input shape tensor format.
- Parameters
-
-
shape
: Input tensor shape -
dtype
: Expected data type for the input (Defaults to Float32) -
format
: Expected tensor format for the input (Defaults to contiguous)
-
-
Input
( c10::ArrayRef<int64_t> shape , TensorFormat format = TensorFormat :: kContiguous )
-
Construct a new Input spec object for static input size from c10::ArrayRef (the type produced by tensor.sizes()), vector, optional arguments allow the user to configure expected input shape tensor format dtype (Expected data type for the input) defaults to PyTorch / traditional TRT convection (FP32 for FP32 only, FP16 for FP32 and FP16, FP32 for Int8)
- Parameters
-
-
shape
: Input tensor shape -
format
: Expected tensor format for the input (Defaults to contiguous)
-
-
Input
( c10::ArrayRef<int64_t> shape , DataType dtype , TensorFormat format = TensorFormat :: kContiguous )
-
Construct a new Input spec object for static input size from c10::ArrayRef (the type produced by tensor.sizes()), vector, optional arguments allow the user to configure expected input shape tensor format.
- Parameters
-
-
shape
: Input tensor shape -
dtype
: Expected data type for the input (Defaults to Float32) -
format
: Expected tensor format for the input (Defaults to contiguous)
-
-
Input
( std::vector<int64_t> min_shape , std::vector<int64_t> opt_shape , std::vector<int64_t> max_shape , TensorFormat format = TensorFormat :: kContiguous )
-
Construct a new Input Range object dynamic input size from c10::ArrayRef (the type produced by tensor.sizes()) for min, opt, and max supported sizes. dtype (Expected data type for the input) defaults to PyTorch / traditional TRT convection (FP32 for FP32 only, FP16 for FP32 and FP16, FP32 for Int8)
- Parameters
-
-
min_shape
: Minimum shape for input tensor -
opt_shape
: Target optimization shape for input tensor -
max_shape
: Maximum acceptible shape for input tensor -
format
: Expected tensor format for the input (Defaults to contiguous)
-
-
Input
( std::vector<int64_t> min_shape , std::vector<int64_t> opt_shape , std::vector<int64_t> max_shape , DataType dtype , TensorFormat format = TensorFormat :: kContiguous )
-
Construct a new Input spec object for a dynamic input size from vectors for minimum shape, optimal shape, and max shape supported sizes optional arguments allow the user to configure expected input shape tensor format.
- Parameters
-
-
min_shape
: Minimum shape for input tensor -
opt_shape
: Target optimization shape for input tensor -
max_shape
: Maximum acceptible shape for input tensor -
dtype
: Expected data type for the input (Defaults to Float32) -
format
: Expected tensor format for the input (Defaults to contiguous)
-
-
Input
( c10::ArrayRef<int64_t> min_shape , c10::ArrayRef<int64_t> opt_shape , c10::ArrayRef<int64_t> max_shape , TensorFormat format = TensorFormat :: kContiguous )
-
Construct a new Input Range object dynamic input size from c10::ArrayRef (the type produced by tensor.sizes()) for min, opt, and max supported sizes. dtype (Expected data type for the input) defaults to PyTorch / traditional TRT convection (FP32 for FP32 only, FP16 for FP32 and FP16, FP32 for Int8)
- Parameters
-
-
min_shape
: Minimum shape for input tensor -
opt_shape
: Target optimization shape for input tensor -
max_shape
: Maximum acceptible shape for input tensor -
format
: Expected tensor format for the input (Defaults to contiguous)
-
-
Input
( c10::ArrayRef<int64_t> min_shape , c10::ArrayRef<int64_t> opt_shape , c10::ArrayRef<int64_t> max_shape , DataType dtype , TensorFormat format = TensorFormat :: kContiguous )
-
Construct a new Input Range object dynamic input size from c10::ArrayRef (the type produced by tensor.sizes()) for min, opt, and max supported sizes.
- Parameters
-
-
min_shape
: Minimum shape for input tensor -
opt_shape
: Target optimization shape for input tensor -
max_shape
: Maximum acceptible shape for input tensor -
dtype
: Expected data type for the input (Defaults to Float32) -
format
: Expected tensor format for the input (Defaults to contiguous)
-
-
bool
get_explicit_set_dtype
( )
Public Members
-
std::vector<int64_t>
min_shape
-
Minimum acceptable input size into the engine.
-
std::vector<int64_t>
opt_shape
-
Optimal input size into the engine (size optimized for given kernels accept any size in min max range)
-
std::vector<int64_t>
max_shape
-
Maximum acceptable input size into the engine.
-
std::vector<int64_t>
shape
-
Input shape to be fed to TensorRT, in the event of a dynamic shape, -1’s will hold the place of variable dimensions
-
DataType
dtype
-
Expected data type for the input.
-
TensorFormat
format
-
Expected tensor format for the input.
-