Problem: Run into a CUDA error during training
Solutions:
This error occurs due to the following two reasons:

Inconsistency between the number of labels/classes and the number of output unitsThe input of the loss function may be incorrect

In my case the error occurs as the loss function is not correctly chosen : change to nn.BCELoss() from nn.CrossEntropyLoss()
Reference: https://towardsdatascience.com/cuda-error-device-side-assert-triggered-c6ae1c8fa4c3