1. Error report
Internal: Blas SGEMM launch failed
2. Ideas & Solutions
Thinking:
I checked all kinds of blogs for GPU and cudnn problems
most of the solutions are to restart the kernel, re install cudnn, install cudnn patches and so on
but for me, they are useless
The final solution:
add this paragraph to the running file
#Prevent cuDNN from reporting errors: Option 2
gpus = tf.config.experimental.list_physical_devices('GPU')
if gpus:
try:
# Currently, memory growth needs to be the same across GPUs
for gpu in gpus:
tf.config.experimental.set_memory_growth(gpu, True)
logical_gpus = tf.config.experimental.list_logical_devices('GPU')
print(len(gpus), "Physical GPUs,", len(logical_gpus), "Logical GPUs")
except RuntimeError as e:
# Memory growth must be set before GPUs have been initialized
print(e)