Skip to content

Global query vector

Created by: fisheggg

Hi,

According to the paper (page 4, Chapter Output space, last paragraph), There's a fourth type of query vector called global:

And finally, a global output o_G, which attends to all the outputs in the model. Although o_G and o_F do contain similar information, we found it useful to still keep two different heads.

Currently we don't have this type of token in the TokenTypes enum. I'll try to submit a PR for this :)