RuntimeError: Function 'MulBackward0' returned nan values in its 0th output.

I pull the codes from gitlab and did not change them. When I run the command below:
python train_patch.py exp1
the nan value error occurs,
starting test read
img read!
One epoch is 4
Running epoch 0: 0%| | 0/4 [00:00<?, ?it/s]display: unable to open X server `' @ error/display.c/DisplayImageCommand/426.
sys:1: RuntimeWarning: Traceback of forward call that caused the error:
File "train_patch.py", line 205, in
main()
File "train_patch.py", line 202, in main
trainer.train()
File "train_patch.py", line 100, in train
max_prob = self.prob_extractor(output)
File "/home/ljack6996/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/ljack6996/adversarial/load_data.py", line 60, in forward
confs_if_object = self.config.loss_target(output_objectness, confs_for_class)
File "/home/ljack6996/adversarial/patch_config.py", line 29, in
self.loss_target = lambda obj, cls: obj * cls

Traceback (most recent call last):
File "train_patch.py", line 205, in
main()
File "train_patch.py", line 202, in main
trainer.train()
File "train_patch.py", line 115, in train
loss.backward()
File "/home/ljack6996/anaconda3/lib/python3.7/site-packages/torch/tensor.py", line 102, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/home/ljack6996/anaconda3/lib/python3.7/site-packages/torch/autograd/init.py", line 90, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: Function 'MulBackward0' returned nan values in its 0th output.

Would somebody could help for this issue? Thank you in advance!