bartorch.use_cuda_memcache

bartorch.use_cuda_memcache()

Set whether BART keeps freed device memory for its own reuse.

Off, memory goes back to the driver as soon as BART is done with it, where torch’s allocator can take it, at the cost of slower allocation. A BART command clears the cache when it ends; the setting matters for operators.

Parameters:

enable (bool) – Keep freed memory in BART’s cache.