Use cached metadata for npm metadata endpoint at the project level

🌱 Context

At this point we should be ready to start returning a cached metadata at the project level endpoint.

last_downloaded_at field of Packages::Npm::MetadataCache will help with an observation of usage of cached metadata and will be a criteria to cleanup old caches.

⚒️ Proposal

  1. Separate the instance and project level metadata endpoints.
  2. Use Packages::Npm::MetadataCache with a fallback to "on the fly" generation for the project level endpoint.
  3. Touch last_downloaded_at field of Packages::Npm::MetadataCache when cached metadata is returned.

Important

Hide behind a feature flag

⚠️ The case when a metadata cache entry exists in the database but a file is missing should be handled properly. This is needed to avoid errors on secondary Geo sites since NPM metadata cache files are not replicated by Geo.

Edited by Dzmitry (Dima) Meshcharakou