I’m training multiple models sequentially, which will be memory-consuming if I keep all models without any cleanup. However, I am not aware of any way to the graph and free the GPU memory in Tensorflow 2.x. Is there a way to do so?
tf.keras.backend.clear_session does not work in my case as I’ve defined some custom layers
tf.compat.v1.reset_default_graph does not work either.
We had another thread about this at:
https://tensorflow-prod.ospodiscourse.com/t/releasing-memory-after-gpu-usage/3991/6
Hi, sorry for answering late. I notice that it invoke cuda.close(). I suppose that I could not reallocate GPU memory If I did that. Am I right? In fact, I only want to free some of the memory related to the model I want to delete, but not all. Moreover, my model does not involve tf.keras.Model; I use a custom training loop