Fix issue with not parsing Robots

Fixes #398 (closed)

Approver's checklist:

• The author has submitted the FSF Copyright Assignment and is listed in AUTHORS
• There is a test suite reasonably covering new functionality or modifications
• Function naming, parameters, return values, types, etc., are consistent with existing code
• This feature/change has adequate documentation added (if appropriate)
• No obvious mistakes / misspelling in the code