textproc/hs-pandoc: Disable optimizations for aarch64.

The same time the port is failing on aarch64 on the cluster for unknown
reason. Disable optimizations to see if that helps.