Template Class Int8Calibrator ¶
-
Defined in File ptq.h
Inheritance Relationships ¶
Base Type ¶
-
private Algorithm
Class Documentation ¶
-
template
<
typename
Algorithm
,
typename
DataLoaderUniquePtr
>
class torch_tensorrt :: ptq :: Int8Calibrator : private Algorithm ¶
-
Generic Int8Calibrator implementation based on a specified TensorRT calibration algorithm and a LibTorch DataLoader.
- tparam Algorithm
-
class nvinfer1::IInt8Calibrator (Default: nvinfer1::IInt8EntropyCalibrator2) - Algorithm to use
- tparam DataLoaderUniquePtr
-
std::unique_ptr<torch::data::DataLoader> - DataLoader type
Public Functions
-
inline
Int8Calibrator
(
DataLoaderUniquePtr
dataloader
,
const
std
::
string
&
cache_file_path
,
bool
use_cache
)
¶
-
Construct a new Int8Calibrator object.
Using the provided DataLoader, construct a calibrator that can be used for PTQ with Torch-TensorRT
- Parameters
-
-
dataloader – std::unqiue_ptr<torch::data::DataLoader> - A unique pointer to the DataLoader, should be what is returned from the make_data_loader factory
-
cache_file_path – const std::string& - A path to store / find the calibration cache
-
use_cache – : bool - Whether to use the cache (if it exists)
-
-
inline
int
getBatchSize
(
)
const
noexcept
override
¶
-
Get the Batch Size for the next batch (always 1 due to issues with TRT and explicit batch)
- Returns
-
int
-
inline
bool
getBatch
(
void
*
bindings
[
]
,
const
char
*
names
[
]
,
int
nbBindings
)
noexcept
override
¶
-
Get the next Batch.
- Parameters
-
-
bindings – void*[] - An array of binding pointers (fed in from TensorRT calibrator), these buffers should be filed with batch data for each input
-
names – const char*[] - Names of bindings
-
nbBindings – int - Number of bindings
-
- Returns
-
true - There is a new batch for the calibrator to consume
- Returns
-
false - There is not a new batch for the calibrator to consume
-
inline
const
void
*
readCalibrationCache
(
size_t
&
length
)
noexcept
override
¶
-
Read calibration cache.
How to read from the calibration cache, only enabled if use_cache is set
- Parameters
-
length –
- Returns
-
const void* - Pointer to cache data
-
inline
void
writeCalibrationCache
(
const
void
*
cache
,
size_t
length
)
noexcept
override
¶
-
Write calibration cache.
Write a the calibration cache provided by TensorRT to a specified file
- Parameters
-
-
cache – const void* - cache data
-
length – size_t - length of cache
-
-
inline
operator
nvinfer1
::
IInt8Calibrator
*
(
)
¶
-
operator to cast to nvinfer1::IInt8Calibrator*
Convience function to convert to a IInt8Calibrator* to easily be assigned to the ptq_calibrator field in CompileSpec
- Returns
-
nvinfer1::IInt8Calibrator*