Logits after MFC is all nan at the start of finetune stage, which makes the model parameters to be all nan.