the second model always entered the tests of over aftering 10 epoches, which made out the error called CUDA out of memory

[Eoiphany]

RuntimeError: CUDA out of memory. Tried to allocate 1.78 GiB (GPU 0; 4.00 GiB total capacity; 7.42 GiB already allocated;

0 bytes free; 9.23 GiB reserved in total by PyTorch)

If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.

See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

the breakpoints are here:
allPreds = t.mm(user, t.transpose(iEmbeds, 1, 0)) * (1 - trnMask) - trnMask * 1e8