bartorch.use_cuda_memcache

bartorch.use_cuda_memcache(enable)

Whether BART keeps freed device memory for its own reuse.

Off, memory goes back to the driver as soon as BART is done with it, where torch’s allocator can take it, at the cost of slower allocation.