Ditch "esm" as a dependency

Based on issue #653 (closed)

That issue makes me wonder if "esm" is needed at all, as a dependency. Since it doesn't have full support for latest Node.js versions (v12+) where native ESM support is good anyway. And probably will never have, because project is abandoned (based on repo activity).

And I'm not sure if people with older Node.js versions (v10 is the oldest supported) even use "imports" in server code, especially in tests. 🤔 Since they probably need high level of stability and/or work with older codebase.

Maybe ditch it in v15 or v16?