How to limit the cpu memory

General Discussion
#1

Hi every one
I have two gpu cards(3090) to execute two different tensorflow(2.9.1) programs. Each of the programs may consume over half of cpu memory (512GB) on my computer.
When I executed the programs written in the functional API style(model.fit was used), the two programs carried out the training with cpu memory consumption around 90 %. However after I rewrote the programs in the subclassing style, the programs or the computer were stopped on the way after the cpu memory consumption of around 100%.
I tested to limit the buffersize in prefetch() and size in batch() in the pipline of the input data, however there is no effect.

How can I limit the cpu memory consumption in the subclassing style?
Best regards
Yu

#3

Hi @Yuichiro_Morishita, As per my knowledge setting the memory limit on the CPU is not supported, currently it is only available on the GPU.As this is a feature request type of issue could you please create a issue on the github. Thank You

#5

Dear Kiran

Thank you for your reply.
I will deal with the problem by increasing the swap memory.

#7

Dear

this is my self-reply to my question.
Increasing the swap memory helps to avoid the hung up the computer, but the speed is too slow to put up with. I changed the pipeline from reading from CPU memory to from HDD( in the former all the data are loaded in memory, while in the latter the filenames are loaded). In this case, two different programs are run in parallel.