Global query vector
Created by: fisheggg
Hi,
According to the paper (page 4, Chapter Output space, last paragraph), There's a fourth type of query vector called global:
And finally, a global output
o_G, which attends to all the outputs in the model. Althougho_Gando_Fdo contain similar information, we found it useful to still keep two different heads.
Currently we don't have this type of token in the TokenTypes enum. I'll try to submit a PR for this :)