Research: Sub-threading of URL updates
As we begin planning for members to update their own resource resolution URLs (#1444 (closed)), support, product, and the technical team met earlier today to consider ramifications of allowing members to process these updates themselves.
What types of considerations do we need to make for very large requests, like the Wiley request of 9 million URL updates from earlier this year?
Right now, we are proposing that we limit each file size of the URL update to 400 KB. A file that meets that size limit would currently process in ~ 30 to 35 minutes, based on support's experience processing these files. Thus, if a member did attempt an update themselves of a million plus DOIs, which support receives once or twice per year, the total processing time of those million plus DOIs would be days. Those updates, by themself, may not be overly concerning. Support and product can set expectations based on current processing times, but is there a way to expedite these URL update submissions?
@myalter believes we may consider sub-threading the URL updates, but suggests that we research this idea. Any downstream considerations? This idea of speeding the processing of these updates is a particular concern during times of high volumes of URL update requests across the organization (e.g., January/February when support currently sees these requests peak).
Note: this sub-threading of URL updates would benefit members and support regardless of how/when we make the changes in #1444 (closed).
How urgent
Would like to see this research completed by early-December 2021, so any changes to sub-threading can be implemented ahead of January surge in URL update requests and to coincide with implementation of #1444 (closed).
Definition of ready
-
Product owner: @SaraBowman -
Tech lead: -
Service:: or C:: label applied -
Definition of done updated -
Acceptance testing plan: -
Weight applied
Definition of done
-
Unit tests identified, implemented, and passing -
Code reviewed -
Available for acceptance testing via a staging URL, or otherwise -
Consider any impacts to current or future architecture/infrastructure, and update specifications and documentation as needed -
Knowledge base reviewed and updated -
Public documentation reviewed and updated -
Acceptance criteria met -
AC 1 -
AC 2
-
-
Acceptance testing passed -
Deployed to production