The objective is to test a reasonably large language model
(GPT2-XL, 1.5B parameters) on the Kilt(https://ai.facebook.com/tools/kilt/)
version of the ELI5(https://www.aclweb.org/anthology/P19-1346/) task when
combined with a retriever (REALM (https://arxiv.org/abs/2002.08909) in this case).

The objective is to observe whether larger causal language models can make use of the
retrieved reference data, as their reasoning capacities are stronger than that
previously tested models.

Inspecting what kinds of retrievals are useful and why would be a next step, as
well as investigating the effects of the retrieval on factual consistency in
generation, which is a problem of major interest right now.

PiperOrigin-RevId: 347074648