Fix some cmake issues.

We shouldn't be building benchmarks by default on peoples' systems. Also fixed some test dependency issues if sparse libraries are detected.

Fixes #2529 (closed).