Follow-up from "Adds FE for ui setting"

The following discussion from !200503 (merged) should be addressed:

• @anna_vovchenko started a discussion:

  thought: I think we might not need to guard this component after #561240 (closed) is done. We might want to remove this as part of the follow-up.