Jon receives a regular report of errors and anomalies in our systems. This report includes issues in various different systems, and a lot of noise. We'd like to find a way to reduce the noise, and also siphon off the errors that the membership team can fix in Sugar to a separate distribution list so we're reducing the toil for Jon in having to identify relevant errors and pass them manually to the membership team.
Single report with a lot of errors/anomalies relating to Sugar, CS, Intacct, ORCID etc.
Remove the noise from the report Where a prefix is identified as deleted in the admin tool, don't alert that the prefix is no longer in Sugar. (eg | 10.30444 | |----------| | 10.30446 | | 10.30449 | | 10.30453 | | 10.30450 | | 10.30451 | | 10.30443 | | 10.30448 | | 10.30447 | | 10.30452 | | 10.11585 | There may be more!
Put some of the messages in a separate report with a distribution list of member@crossref.org Errors that the membership team can fix in Sugar should be included in this list. This includes:
This is not causing an immediate problem, but will help to reduce toil for a developer and mean that genuine problems in the data are fixed faster, reducing the risk of the wrong members being invoiced, reports not being sent to the right place, and metadata being incorrect.
Migrated this to two tickets in Jira; first reduces noise (https://crossref.atlassian.net/browse/CR-1269) second siphons off emails to go to membership team directly (https://crossref.atlassian.net/browse/CR-1270)
Closing this issue here
As an X I want to do Y because of Z
Prior to and during Backlog Refinement, consider the potential impacts this user story may have on the following areas:
Additional details about the above items can be found here.
As an X I want to do Y because of Z
Prior to and during Backlog Refinement, consider the potential impacts this user story may have on the following areas:
Additional details about the above items can be found here.
We requested Robert Tupelo-Schneck do some investigation to find oddly configured DOI Prefixes, so that we could be sure that #1461 will behave as expected. He found 80 prefixes with nonstandard configurations which could mean that content negotiation doesn't work for DOIs on those prefixes, and will continue to not work unless we fix them.
I've not checked them all, but in an indicitive number of cases the HS_NAMESPACE
is missing a <locs>
element that points to 10.SERV/CROSSREF
, which is where we specify the Crossref-wide content negotiation behaviour.
Checking the handles: http://hdl.handle.net/ (check "Authoritative Query" and "Don't Redirect to URLs")
A probleatic one, 0.NA/10.5578
, gives:
<HS_NAMESPACE>
<DOI.RA>10.SERV/CROSSREF</DOI.RA>
</HS_NAMESPACE>
Where a normal Crossref DOI prefix 0.NA/10.1371
gives:
<HS_NAMESPACE>
<DOI.RA>10.SERV/CROSSREF</DOI.RA>
<locs>10.SERV/CROSSREF</locs>
</HS_NAMESPACE>
The list is at the end of the ticket.
Any DOI on these prefixes won't work with content negotiation. This is likely the cause of #563 .
I would expect that Content Negotiation works for all Crossref content DOIs. Possibly not Funders though?
List of prefixes:
0.NA/10.5578
0.NA/10.5579
0.NA/10.5577
0.NA/10.5575
0.NA/10.5572
0.NA/10.5589
0.NA/10.5587
0.NA/10.5588
0.NA/10.5581
0.NA/10.5582
0.NA/10.5580
0.NA/10.5585
0.NA/10.5586
0.NA/10.5583
0.NA/10.5584
0.NA/10.5598
0.NA/10.5599
0.NA/10.5592
0.NA/10.5593
0.NA/10.5596
0.NA/10.5597
0.NA/10.5594
0.NA/10.5595
0.NA/10.5619
0.NA/10.5617
0.NA/10.5618
0.NA/10.5611
0.NA/10.5610
0.NA/10.5615
0.NA/10.5616
0.NA/10.5613
0.NA/10.5614
0.NA/10.5628
0.NA/10.5629
0.NA/10.5622
0.NA/10.5623
0.NA/10.5620
0.NA/10.5621
0.NA/10.5626
0.NA/10.5627
0.NA/10.5624
0.NA/10.5625
0.NA/10.5633
0.NA/10.5634
0.NA/10.5631
0.NA/10.5632
0.NA/10.5630
0.NA/10.5608
0.NA/10.5609
0.NA/10.5606
0.NA/10.5600
0.NA/10.5601
0.NA/10.5604
0.NA/10.5605
0.NA/10.5602
0.NA/10.5603
0.NA/10.13101
0.NA/10.5516
0.NA/10.3853
0.NA/10.1601
0.NA/10.1956
0.NA/10.1989
0.NA/10.18571
0.NA/10.18570
0.NA/10.18569
0.NA/10.18566
0.NA/10.18565
0.NA/10.18568
0.NA/10.18567
0.NA/10.7257
0.NA/10.7255
0.NA/10.7256
0.NA/10.6339
0.NA/10.6219
0.NA/10.CRADMIN
0.NA/10.488
0.NA/10.5890
0.NA/10.3906
0.NA/10.3912
0.NA/10.11
0.NA/10.2378
0.NA/10.2399
0.NA/10.1009
Closing here as this has been moved to jira, link above
Sally transferred the following titles from Kare Publishing (10.14744) to AVES Publishing Co. (10.5152) on 5 January.
However, username kare has turned up on the AVES Publishing Co. (10.5152) Q4 invoice suggesting that deposits made by kare were being charged to AVES Publishing Co. (10.5152).
Upon further investigation, it doesn't look like any kare submissions were being charged on the invoice, as is supported by this report (see depositor ID - all submissions are from depositor ID 25527 (username ajca):
Current_Year_Journal_DOIs_deposited_during_period_for_given_prefix__ORACLE___17___1_.xls
Moved to Jira: https://crossref.atlassian.net/browse/CR-498
Informatics Publishing, a sponsoring org, reached out to us to clarify the final line on their Q3 2020 invoice where the Description reads: CY Book Chapters (250 and under) 09/2020 : 10.37247 : CY Book Chapters
And where the quantity value given was: 125
Their sponsored member Vide Leaf (10.37247) did not register any book chapter DOIs in September 2020. They did register 125 book chapter DOIs across July and August 2020.
So, it appears that the book chapter lines on invoices refer to the number of DOIs registered across the whole quarter, unlike the rest of the content types which are listed month by month.
I assume this has something to do with the volume discounts for book chapters. However, it's confusing to have the description state a particular month, if the count is for the whole quarter.
Can we change the invoice lines for book chapters to say, for example, Q3 2020, rather than 9/2020 to avoid type of confusion?
moved to gitlab: https://crossref.atlassian.net/browse/CR-497
As we begin planning for members to update their own resource resolution URLs (#1444), support, product, and the technical team met earlier today to consider ramifications of allowing members to process these updates themselves.
What types of considerations do we need to make for very large requests, like the Wiley request of 9 million URL updates from earlier this year?
Right now, we are proposing that we limit each file size of the URL update to 400 KB. A file that meets that size limit would currently process in ~ 30 to 35 minutes, based on support's experience processing these files. Thus, if a member did attempt an update themselves of a million plus DOIs, which support receives once or twice per year, the total processing time of those million plus DOIs would be days. Those updates, by themself, may not be overly concerning. Support and product can set expectations based on current processing times, but is there a way to expedite these URL update submissions?
@myalter believes we may consider sub-threading the URL updates, but suggests that we research this idea. Any downstream considerations? This idea of speeding the processing of these updates is a particular concern during times of high volumes of URL update requests across the organization (e.g., January/February when support currently sees these requests peak).
Note: this sub-threading of URL updates would benefit members and support regardless of how/when we make the changes in #1444.
Would like to see this research completed by early-December 2021, so any changes to sub-threading can be implemented ahead of January surge in URL update requests and to coincide with implementation of #1444.
We've released the feature allowing users to bulk update their own URLs and so far there has not been a performance issue, so I'm closing this issue.
Right now we have some very basic alerting in place through Slack in the #queue-view channel that pings the channel when the queue exceeds 35,000 submissions to doi.crossref.org. This alarm is helpful, but support still relies heavily on members and observation to determine when problems arise in submission processing (as was exemplified with this incident from October 2021: https://status.crossref.org/incidents/z4jx9js265pp).
After talking with Mike, Jon, and Sara, we'd all like to expand those checks to give us more visibility into the health of submission processing in production.
We'd like to monitor (with checks every 5 to 10 minutes) for the following:
Would like to have this in place by 2022 January 01