Struct Input ¶
-
Defined in File torch_tensorrt.h
Struct Documentation ¶
-
struct
torch_tensorrt
::
Input
¶
-
A struct to hold an input range (used by TensorRT Optimization profile)
This struct can either hold a single vector representing an input shape, signifying a static input shape or a set of three input shapes representing the min, optiminal and max input shapes allowed for the engine.
Public Functions
-
Input
( std::vector<int64_t> shape , TensorFormat format = TensorFormat ::kContiguous ) ¶
-
Construct a new Input spec object for static input size from vector, optional arguments allow the user to configure expected input shape tensor format. dtype (Expected data type for the input) defaults to PyTorch / traditional TRT convection (FP32 for FP32 only, FP16 for FP32 and FP16, FP32 for Int8)
- Parameters
-
-
shape
: Input tensor shape -
format
: Expected tensor format for the input (Defaults to contiguous)
-
-
Input
( std::vector<int64_t> shape , DataType dtype , TensorFormat format = TensorFormat ::kContiguous ) ¶
-
Construct a new Input spec object for static input size from vector, optional arguments allow the user to configure expected input shape tensor format.
- Parameters
-
-
shape
: Input tensor shape -
dtype
: Expected data type for the input (Defaults to the type of the weights in the first tensor calculation if detectable else Float32) -
format
: Expected tensor format for the input (Defaults to contiguous)
-
-
Input
( c10::ArrayRef<int64_t> shape , TensorFormat format = TensorFormat ::kContiguous ) ¶
-
Construct a new Input spec object for static input size from c10::ArrayRef (the type produced by tensor.sizes()), vector, optional arguments allow the user to configure expected input shape tensor format dtype (Expected data type for the input) defaults to PyTorch / traditional TRT convection (FP32 for FP32 only, FP16 for FP32 and FP16, FP32 for Int8)
- Parameters
-
-
shape
: Input tensor shape -
format
: Expected tensor format for the input (Defaults to contiguous)
-
-
Input
( c10::ArrayRef<int64_t> shape , DataType dtype , TensorFormat format = TensorFormat ::kContiguous ) ¶
-
Construct a new Input spec object for static input size from c10::ArrayRef (the type produced by tensor.sizes()), vector, optional arguments allow the user to configure expected input shape tensor format.
- Parameters
-
-
shape
: Input tensor shape -
dtype
: Expected data type for the input (Defaults to the type of the weights in the first tensor calculation if detectable else Float32) -
format
: Expected tensor format for the input (Defaults to contiguous)
-
-
Input
( std::vector<int64_t> min_shape , std::vector<int64_t> opt_shape , std::vector<int64_t> max_shape , TensorFormat format = TensorFormat ::kContiguous ) ¶
-
Construct a new Input spec object dynamic input size from c10::ArrayRef (the type produced by tensor.sizes()) for min, opt, and max supported sizes. dtype (Expected data type for the input) defaults to PyTorch / traditional TRT convection (FP32 for FP32 only, FP16 for FP32 and FP16, FP32 for Int8)
- Parameters
-
-
min_shape
: Minimum shape for input tensor -
opt_shape
: Target optimization shape for input tensor -
max_shape
: Maximum acceptible shape for input tensor -
format
: Expected tensor format for the input (Defaults to contiguous)
-
-
Input
( std::vector<int64_t> min_shape , std::vector<int64_t> opt_shape , std::vector<int64_t> max_shape , DataType dtype , TensorFormat format = TensorFormat ::kContiguous ) ¶
-
Construct a new Input spec object for a dynamic input size from vectors for minimum shape, optimal shape, and max shape supported sizes optional arguments allow the user to configure expected input shape tensor format.
- Parameters
-
-
min_shape
: Minimum shape for input tensor -
opt_shape
: Target optimization shape for input tensor -
max_shape
: Maximum acceptible shape for input tensor -
dtype
: Expected data type for the input (Defaults to the type of the weights in the first tensor calculation if detectable else Float32) -
format
: Expected tensor format for the input (Defaults to contiguous)
-
-
Input
( c10::ArrayRef<int64_t> min_shape , c10::ArrayRef<int64_t> opt_shape , c10::ArrayRef<int64_t> max_shape , TensorFormat format = TensorFormat ::kContiguous ) ¶
-
Construct a new Input spec object dynamic input size from c10::ArrayRef (the type produced by tensor.sizes()) for min, opt, and max supported sizes. dtype (Expected data type for the input) defaults to PyTorch / traditional TRT convection (FP32 for FP32 only, FP16 for FP32 and FP16, FP32 for Int8)
- Parameters
-
-
min_shape
: Minimum shape for input tensor -
opt_shape
: Target optimization shape for input tensor -
max_shape
: Maximum acceptible shape for input tensor -
format
: Expected tensor format for the input (Defaults to contiguous)
-
-
Input
( c10::ArrayRef<int64_t> min_shape , c10::ArrayRef<int64_t> opt_shape , c10::ArrayRef<int64_t> max_shape , DataType dtype , TensorFormat format = TensorFormat ::kContiguous ) ¶
-
Construct a new Input spec object dynamic input size from c10::ArrayRef (the type produced by tensor.sizes()) for min, opt, and max supported sizes.
- Parameters
-
-
min_shape
: Minimum shape for input tensor -
opt_shape
: Target optimization shape for input tensor -
max_shape
: Maximum acceptible shape for input tensor -
dtype
: Expected data type for the input (Defaults to the type of the weights in the first tensor calculation if detectable else Float32) -
format
: Expected tensor format for the input (Defaults to contiguous)
-
-
Input
( at::Tensor tensor ) ¶
-
Construct a new Input spec object using a torch tensor as an example The tensor’s shape, type and layout inform the spec’s values.
Note: You cannot set dynamic shape through this method, you must use an alternative constructor
- Parameters
-
-
tensor
: Reference tensor to set shape, type and layout
-
Public Members
-
std::vector<int64_t>
min_shape
¶
-
Minimum acceptable input size into the engine.
-
std::vector<int64_t>
opt_shape
¶
-
Optimal input size into the engine (size optimized for given kernels accept any size in min max range)
-
std::vector<int64_t>
max_shape
¶
-
Maximum acceptable input size into the engine.
-
std::vector<int64_t>
shape
¶
-
Input shape to be fed to TensorRT, in the event of a dynamic shape, -1’s will hold the place of variable dimensions
-
TensorFormat
format
¶
-
Expected tensor format for the input.
-