Template Class Int8Calibrator ¶
-
Defined in File ptq.h
Inheritance Relationships ¶
Base Type ¶
-
private Algorithm
Class Documentation ¶
-
template<typename
Algorithm
, typenameDataLoaderUniquePtr
>
classtorch_tensorrt::ptq
::
Int8Calibrator
: private Algorithm ¶
-
Generic Int8Calibrator implementation based on a specified TensorRT calibration algorithm and a LibTorch DataLoader.
- Template Parameters
-
-
Algorithm
: class nvinfer1::IInt8Calibrator (Default: nvinfer1::IInt8EntropyCalibrator2) - Algorithm to use -
DataLoaderUniquePtr
: std::unique_ptr<torch::data::DataLoader> - DataLoader type
-
Public Functions
-
Int8Calibrator
( DataLoaderUniquePtr dataloader , const std::string & cache_file_path , bool use_cache ) ¶
-
Construct a new Int8Calibrator object.
Using the provided DataLoader, construct a calibrator that can be used for PTQ with Torch-TensorRT
- Parameters
-
-
dataloader
: std::unqiue_ptr<torch::data::DataLoader> - A unique pointer to the DataLoader, should be what is returned from the make_data_loader factory -
cache_file_path
: const std::string& - A path to store / find the calibration cache -
use_cache
: : bool - Whether to use the cache (if it exists)
-
-
int
getBatchSize
( ) const noexcept override ¶
-
Get the Batch Size for the next batch (always 1 due to issues with TRT and explicit batch)
- Return
-
int
-
bool
getBatch
( void * bindings [], const char * names [], int nbBindings ) noexcept override ¶
-
Get the next Batch.
- Return
-
true - There is a new batch for the calibrator to consume
- Return
-
false - There is not a new batch for the calibrator to consume
- Parameters
-
-
bindings
: void*[] - An array of binding pointers (fed in from TensorRT calibrator), these buffers should be filed with batch data for each input -
names
: const char*[] - Names of bindings -
nbBindings
: int - Number of bindings
-
-
const
void *
readCalibrationCache
( size_t & length ) noexcept override ¶
-
Read calibration cache.
How to read from the calibration cache, only enabled if use_cache is set
- Return
-
const void* - Pointer to cache data
- Parameters
-
-
length
:
-
-
void
writeCalibrationCache
( const void * cache , size_t length ) noexcept override ¶
-
Write calibration cache.
Write a the calibration cache provided by TensorRT to a specified file
- Parameters
-
-
cache
: const void* - cache data -
length
: size_t - length of cache
-
-
operator nvinfer1::IInt8Calibrator*
( ) ¶
-
operator to cast to nvinfer1::IInt8Calibrator*
Convience function to convert to a IInt8Calibrator* to easily be assigned to the ptq_calibrator field in CompileSpec
- Return
-
nvinfer1::IInt8Calibrator*