CUB
|
A simple caching allocator for device memory allocations.
active_stream
. Once freed, the allocation becomes available immediately for reuse within the active_stream
with which it was associated with during allocation, and it becomes available for reuse within other streams when all prior work submitted to active_stream
has completed.bin_growth
provided during construction. Unused device allocations within a larger bin cache are not reused for allocation requests that categorize to smaller bin sizes.bin_growth
^ min_bin
) are rounded up to (bin_growth
^ min_bin
).bin_growth
^ max_bin
) are not rounded up to the nearest bin and are simply freed when they are deallocated instead of being returned to a bin-cache.max_cached_bytes
, allocations for that device are simply freed when they are deallocated instead of being returned to their bin-cache.bin_growth
= 8min_bin
= 3max_bin
= 7max_cached_bytes
= 6MB - 1BDefinition at line 104 of file util_allocator.cuh.
Public Methods | |
CachingDeviceAllocator (unsigned int bin_growth, unsigned int min_bin, unsigned int max_bin, size_t max_cached_bytes, bool skip_cleanup=false) | |
Constructor. More... | |
CachingDeviceAllocator (bool skip_cleanup=false) | |
Default constructor. More... | |
cudaError_t | SetMaxCachedBytes (size_t max_cached_bytes) |
Sets the limit on the number bytes this allocator is allowed to cache per device. | |
cudaError_t | DeviceAllocate (int device, void **d_ptr, size_t bytes, cudaStream_t active_stream=0) |
Provides a suitable allocation of device memory for the given size on the specified device. More... | |
cudaError_t | DeviceAllocate (void **d_ptr, size_t bytes, cudaStream_t active_stream=0) |
Provides a suitable allocation of device memory for the given size on the current device. More... | |
cudaError_t | DeviceFree (int device, void *d_ptr) |
Frees a live allocation of device memory on the specified device, returning it to the allocator. More... | |
cudaError_t | DeviceFree (void *d_ptr) |
Frees a live allocation of device memory on the current device, returning it to the allocator. More... | |
cudaError_t | FreeAllCached () |
Frees all cached device allocations on all devices. | |
virtual | ~CachingDeviceAllocator () |
Destructor. | |
|
inline |
Constructor.
bin_growth | Geometric growth factor for bin-sizes |
min_bin | Minimum bin |
max_bin | Maximum bin |
max_cached_bytes | Maximum aggregate cached bytes per device |
skip_cleanup | Whether or not to skip a call to FreeAllCached() when the destructor is called. (Useful for preventing warnings when the allocator is declared at file/static/global scope: by the time the destructor is called on program exit, the CUDA runtime may have already shut down and freed all allocations.) |
Definition at line 259 of file util_allocator.cuh.
|
inline |
Default constructor.
Configured with:
bin_growth
= 8min_bin
= 3max_bin
= 7max_cached_bytes
= (bin_growth
^ max_bin
) * 3) - 1 = 6,291,455 byteswhich delineates five bin-sizes: 512B, 4KB, 32KB, 256KB, and 2MB and sets a maximum of 6,291,455 cached bytes per device
skip_cleanup | Whether or not to skip a call to FreeAllCached() when the destructor is called. (Useful for preventing warnings when the allocator is declared at file/static/global scope: by the time the destructor is called on program exit, the CUDA runtime may have already shut down and freed all allocations.) |
Definition at line 294 of file util_allocator.cuh.
|
inline |
Provides a suitable allocation of device memory for the given size on the specified device.
Once freed, the allocation becomes available immediately for reuse within the active_stream
with which it was associated with during allocation, and it becomes available for reuse within other streams when all prior work submitted to active_stream
has completed.
[in] | device | Device on which to place the allocation |
[out] | d_ptr | Reference to pointer to the allocation |
[in] | bytes | Minimum number of bytes for the allocation |
[in] | active_stream | The stream to be associated with this allocation |
Definition at line 347 of file util_allocator.cuh.
|
inline |
Provides a suitable allocation of device memory for the given size on the current device.
Once freed, the allocation becomes available immediately for reuse within the active_stream
with which it was associated with during allocation, and it becomes available for reuse within other streams when all prior work submitted to active_stream
has completed.
[out] | d_ptr | Reference to pointer to the allocation |
[in] | bytes | Minimum number of bytes for the allocation |
[in] | active_stream | The stream to be associated with this allocation |
Definition at line 485 of file util_allocator.cuh.
|
inline |
Frees a live allocation of device memory on the specified device, returning it to the allocator.
Once freed, the allocation becomes available immediately for reuse within the active_stream
with which it was associated with during allocation, and it becomes available for reuse within other streams when all prior work submitted to active_stream
has completed.
Definition at line 506 of file util_allocator.cuh.
|
inline |
Frees a live allocation of device memory on the current device, returning it to the allocator.
Once freed, the allocation becomes available immediately for reuse within the active_stream
with which it was associated with during allocation, and it becomes available for reuse within other streams when all prior work submitted to active_stream
has completed.
Definition at line 604 of file util_allocator.cuh.