Add configuration option to fail on warnings

Background

It would be nice to have to a configuration parameter to make warnings errors (For example, for #483 (closed), we would like to make that an error)

Acceptance Criteria

It's possible to configure warnings to be fatal