RuntimeError: cuDNN error: CUDNN_STATUS_NOT_INITIALIZED

[buptgxt]

Traceback (most recent call last):
File "main.py", line 116, in
main(args)
File "main.py", line 38, in main
trainer.train(c_ids, q_ids, a_ids, start_positions, end_positions)
File "/home/2018/Info-HCVAE-master/vae/trainer.py", line 35, in train
loss.backward()
File "/home/2018/anaconda3/envs/transformers/lib/python3.6/site-packages/torch/tensor.py", line 150, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/home/2018/anaconda3/envs/transformers/lib/python3.6/site-packages/torch/autograd/init.py", line 99, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: cuDNN error: CUDNN_STATUS_NOT_INITIALIZED

cuda:9.0 cudnn:7
python3.6 pytorch1.3

thank you for your work! I am trying to train the model but I’ve got an error that is “RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED”. I got this problem when this part of code runs: loss.backward()
Can you help me to solve it?

[seanie12]

I think it might be due to the GPU memory issue.

Try using smaller batch size.