Online Appendix to Knowledge Base Extraction for Learning Agents