feat(datastore): importer: use progressbar to show import progress

What does this MR do?

This MR changes the default import tool to display a progress bar and write the detailed logs to a timestamped file in binaries directory by default. The Importer keeps the current behavior by default, so that we can see detailed information if CI tests fail.

The import tool gets a new option: --log-to-stdout, so that the import tool can be configured to behave as is does today, printing the logs to STDOUT.

The GetLogger function gets a new WithWriter option to set the log's writer. This is needed to write the detail logs to a file, rather than standard out.

What does this MR look like

(click on it to get clearer font)

Error Example:

10:16 $ ./bin/registry database import config-walk-database-fs.yml
|  [0s] registry import starting, detailed log written to: /home/hayley/go/src/github.com/docker/distribution/2023-12-05T10:16:22-08:00-registry-import.log
|  [0s] step one: import manifests
|  [0s] step one: import manifests failed to import metadata: pre importing all repositories: : Path not found: /docker/registry/v2/repositories

Comments

Overall, I'm pleased with these aesthetics. Step two happens so quickly that its progress bar is hard to catch, but for very large repositories or for when there are many new writes it's still useful. It's essentially the same as step one's progress bar, but we could potentially track step two progress by repository, rather than tag, although in this example it would be similarly hard to catch.

Another option would be to print the end state of each repo to the screen, leaving behind one line of a completed progress bar per repository. This quickly fills the screen with stale data, and I found it to be less elegant than the present version.

Author checklist

• Feature flags
  • Added feature flag:
  • This feature does not require a feature flag
• I added unit tests or they are not required
• I added documentation (or it's not required)
• I followed code review guidelines
• I followed Go Style guidelines
• For database changes including schema migrations:
  • Manually run up and down migrations in a postgres.ai production database clone and post a screenshot of the result here.
  • If adding new queries, extract a query plan from postgres.ai and post the link here. If changing existing queries, also extract a query plan for the current version for comparison.
  • Do not include code that depends on the schema migrations in the same commit. Split the MR into two or more.
• Ensured this change is safe to deploy to individual stages in the same environment (cny -> prod). State-related changes can be troublesome due to having parts of the fleet processing (possibly related) requests in different ways.

Reviewer checklist

• Ensure the commit and MR tittle are still accurate.
• If the change contains a breaking change, apply the breaking change label.
• If the change is considered high risk, apply the label high-risk-change
• Identify if the change can be rolled back safely. (note: all other reasons for not being able to rollback will be sufficiently captured by major version changes).

If the MR introduces database schema migrations:

• Ensure the commit and MR tittle start with fix:, feat:, or perf: so that the change appears on the Changelog

If the changes cannot be rolled back follow these steps:

• If not, apply the label cannot-rollback.
• Add a section to the MR description that includes the following details:
  • The reasoning behind why a release containing the presented MR can not be rolled back (e.g. schema migrations or changes to the FS structure)
  • Detailed steps to revert/disable a feature introduced by the same change where a migration cannot be rolled back. (note: ideally MRs containing schema migrations should not contain feature changes.)
  • Ensure this MR does not add code that depends on these changes that cannot be rolled back.