Make database backend swappable

There has been a lot of criticism from the start of Conduit using sled as the storage backend. The main arguments against it were:

  1. It's new and was not tested much yet, making it potentially unreliable.
  2. There are not many tools to inspect or manipulate the db in case of problems.

We still chose sled because:

  1. The sled v1.0 release was originally planned i n January. This gave a sense of reliability.
  2. Sled fits the keep-it-simple architecture Conduit is going for.
  3. Conduit is very fast with sled, mostly because it's an in-memory database. See romeo juliet benchmark.

However, over time we noticed more problems with sled:

  1. Memory usage sometimes goes very high with large databases.
  2. Sled currently deadlocks in some situations

These issues with sled will probably improve at some point (the deadlock is already fixed on sled's master branch), but they left a bad impression.

How should we move forward? I think it is a good idea to invest time into a swappable database backend, so it's easy to change when issues pop up.

What alternatives to sled would you recommend that fit to Conduit? Post them as comments with a short explanation and pros and cons (or link to a page that does so).