Add configuration option to fail on warnings
Background
It would be nice to have to a configuration parameter to make warnings errors (For example, for #483 (closed), we would like to make that an error)
Acceptance Criteria
It's possible to configure warnings to be fatal