/cleanup/dsd method must repair wrong [ARTEFACT] table entries
Today, maybe due to incomplete transaction management, it can happen that the [ARTEFACT] table in the dataDb contains incorrect content:
Especially, if a DSD in the [ARTEFACT] table has a linked MSD ([MSD_ID] has a value) but that MSD is not present in this table.
As this issue results in an error "Dotstat.Transfer.Exception.TransferFailedException" for any data upload, this incorrect [ARTEFACT] table content needs to be repaired with the /cleanup/dsd
method, e.g. using the option cleanupMsdOnly=true
.
Original request
Receiving error "Dotstat.Transfer.Exception.TransferFailedException" after re-uploading structures and data
I deleted all the structures about my dsd ds_nadet2008 and reuploaded them. But when I load the data I'm receiving a mail as mentioned in this ticket (Dotstat.Transfer.Exception.TransferFailedException) Therefore I tried the cleanup/mappingsets and cleanup/DSD, but this didn't work.error_after_patch_8.1.2.docx NADET2008_Multiple_added_annotations.xmlNADET2008_3.csvnadet2008_metadata.csv
Transfer logs details:
{
"requestId": 623,
"action": "Import",
"userEmail": "hilde.oeyen@nbb.be",
"artefact": "BE2:DS_NADET2008@DF_NADET2008(1.0)",
"sourceDataspace": null,
"destinationDataspace": "predisseminate",
"sourceData": "NADET2008_3.csv",
"submissionTime": "2022-05-20T11:22:30.94",
"executionStart": "2022-05-20T11:22:31.16",
"executionEnd": "2022-05-20T11:22:32.61",
"executionStatus": "Completed",
"outcome": "Error",
"logs": [
{
"id": 623,
"userEmail": null,
"date": "2022-05-20T09:22:31.167",
"level": "NOTICE",
"server": "c49408d717f9",
"logger": "DotStatServices.Transfer.Controllers.ImportController.Import",
"message": "The request with ID 623 was successfully registered.",
"exception": ""
},
{
"id": 623,
"userEmail": null,
"date": "2022-05-20T09:22:31.503",
"level": "ERROR",
"server": "c49408d717f9",
"logger": "DotStat.Db.Repository.SqlServer.SqlTransactionRepository.TryNewTransaction",
"message": "Exception of type 'DotStat.Db.Exception.ArtefactNotFoundException' was thrown.",
"exception": "DotStat.Db.Exception.ArtefactNotFoundException: Exception of type 'DotStat.Db.Exception.ArtefactNotFoundException' was thrown.\n at DotStat.Db.Repository.SqlServer.SqlManagementRepository.GetArtefactByDbId(Int32 dbId)\n at DotStat.Db.Repository.SqlServer.SqlManagementRepository.VerifyAndUpdateDsdMsd(Dsd dsd)\n at DotStat.Db.Repository.SqlServer.SqlManagementRepository.FillDbIdsAndCreateMissingDbObjects(Dataflow dataflow, Int32 transactionId, Boolean skipDsd)\n at DotStat.Db.Repository.SqlServer.SqlTransactionRepository.TryNewTransaction(Int32 transactionId, Dataflow dataflow, IMappingStoreDataAccess mappingStoreDataAccess, TargetVersion transferTargetVersion, Boolean isRollbackOrRestore, DotStatPrincipal principal, String sourceDataspace, String dataSource, TransactionType transactionType)"
},
{
"id": 623,
"userEmail": null,
"date": "2022-05-20T09:22:32.843",
"level": "WARN",
"server": "c49408d717f9",
"logger": "DotStatServices.Transfer.Controllers.ImportController+<>c__DisplayClass20_1`1+<<Import>b__0>d.MoveNext",
"message": "The processing of observations was stopped and dropped due to one or more errors. Please read the details of the error(s) to see how to resolve it/them.",
"exception": ""
},
{
"id": 623,
"userEmail": null,
"date": "2022-05-20T09:22:32.843",
"level": "ERROR",
"server": "c49408d717f9",
"logger": "DotStatServices.Transfer.Controllers.ImportController+<>c__DisplayClass20_1`1+<<Import>b__0>d.MoveNext",
"message": "Exception of type 'DotStat.Transfer.Exception.TransferFailedException' was thrown.",
"exception": "DotStat.Transfer.Exception.TransferFailedException: Exception of type 'DotStat.Transfer.Exception.TransferFailedException' was thrown.\n at DotStat.Transfer.Manager.FileTransferManager`1.Transfer(T transferParam, CancellationToken cancellationToken) in /app/DotStat.Transfer/Manager/FileTransferManager.cs:line 45\n at DotStatServices.Transfer.Controllers.ImportController.<>c__DisplayClass20_1`1.<<Import>b__0>d.MoveNext() in /app/DotStatServices.Transfer/Controllers/ImportController.cs:line 516"
}
],
"lastUpdated": "2022-05-20T11:22:32.61"
}