comet: Implement explicit FileSet and component ordering in bulk ingest

Description

Currently, FileSets and component objects are displayed in random order after bulk import. The entries start in CSV order, but that's not guaranteed once Sidekiq processes them concurrently. Even if jobs are enqueued in order, they'll likely finish in different orders based on thread availability.

Note: Order of objects within collections is out of scope for this ticket (which is just as well since Valkyrie manages collection member relationships differently from object membership relationships)

Summary

Implement ordered FileSet / component object functionality to allow CSV bulk imports to specify the display order of FileSets and components.

➡️ After a few discussions with @arwenhutt @chrissyrissmeyer @dmolesUC we think a simple ordering column would best meet user needs. (See column A, order, in the sample spreadsheet, order_paged_object_3_2025-07-25_proposed). We did consider other non-simple order solutions too (column F, order_within_parent ), but decided simple order was more desirable.

This enhancement will enable users to control the presentation order by arranging files in their desired sequence within their CSV using a simple order column, with the first FileSet listed becoming the primary FileSet/representative thumbnail in the display order.

Display Requirements

  • Frontend Display: User interface displays FileSets and component objects in the specified order across all relevant views
  • Search Results: Ordered FileSet and component objects maintain their sequence in search result previews
  • API Consistency: API endpoints return FileSets and component objects in the correct order

Acceptance Criteria:

  • CSV Row Order Mapping: Filesets and component objects are displayed in the order defined by the CSV order column (and filesets and component objects can be mixed).
    • If the CSV order column is not present, they should be displayed in the order they appear in the CSV (use the row index to define the order)
  • First Fileset Priority: For each object, the first FileSet in the defined order becomes the default primary FileSet/representative thumbnail for that object
  • Document the technical limitations of our ordering solution and think about how it may impact updates to order which will be addressed in #1972

Implementation notes

  • Simplest thing is probably to stick the order in the parsed_metadata of the Bulkrax entry (we have an order column in PendingRelationship, but it doesn't solve this problem because those records are deleted as the relationships are created)
  • The values of the order column aren't persistent, only the order (we can continue to just use the order of the member_ids array, and at export we can generate order column values as needed)
  • The order is global across the whole CSV (no need to order objects and filesets separately, or restart ordering for each object; single order applies to both filesets and component objects, so you can define an order that for instance alternates filesets and component objects)

Examples from Ohio State and Emory:

2025-09-12:

Ohio State: We're using an older version of Bulkrax at Ohio State and not on Valkyrie. The files ingest in random order unless we add an order column. We borrowed code from another repository for the order column. However, the ordering happens the record ingest is complete. That means the representative media and thumbnail are just whatever file randomly ingested first. So, no order by default.

See code example from OSU: billCox_OSU_Order.txt

2025-05-13: Discussion thread from the #Bulkrax channel (.txt) and sample CSV from Emory added here.

Emory: We did a local workaround even prior to Valkyrie to support the fileset rows' ordering in a CSV, because part sequence is important, and we didn't want to have to use the File Manager to re-sort parts each time we added things. In Valkyrie-Bulkrax 8.x, individual FileSet rows aren't yet supported, so we're controlling order in a different way. We currently use Bulkrax in a Hyrax 3.x application as well as our new Hyrax 5.x application.

Emory Repo: https://github.com/emory-libraries/dlp-selfdeposit

Edited by Arwen Hutt