Batch metadata processing or parsing will fail the index tasks: IndexInstanceNode matching query does not exist.
I ran through a File Metadata Processing per type for all documents (~150) and at the end it gave a lot of errors with the same exception. Looks like it's failing on the index task after metadata generation.
I purged all the Mayan caches, reset all indexes and when manually rebuilding them they all work okay without issue with the right no of documents.
If I clear them again, run the metadata processing per type for all documents again no exception occurs and the right no of documents is in each index.
However, if then run metadata processing per type again (with indexes already built and looking normal), it will get through all the metadata but fall over on the indexing part at the end. It's almost as if an earlier re-indexing job for some of the documents changes the index and breaks the subsequent reindexing jobs.
Always at the end the right no of documents are in the index, but I wanted to flag it in case there's some kind of race condition that might hit elsewhere.
I should note that I only have 1 index queue running at a time.
I've just tried another test but with file parsing and the exact same situation occurs. A few documents will index but the rest will fail but the overall index numbers look okay at the end (but no exception if I reset indexes first). When doing it with Parsing it is slower though (the parsing is quick but the indexing failure is slow). My concern is potentially the index might get out of sync when doing a bulk re-parse or metadata reprocess with this error occurring unless the user resets all indexes before running a re-parse or metadata re-process. I didn't notice this before 3.3.6 but then again I've not been looking at the logs much since 3.3.4
Traceback (most recent call last):
File "/opt/mayan-edms/local/lib/python3.7/site-packages/django/db/models/fields/related_descriptors.py", line 178, in __get__
rel_obj = getattr(instance, self.cache_name)
AttributeError: 'IndexInstanceNode' object has no attribute '_parent_cache'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/mayan-edms/local/lib/python3.7/site-packages/celery/app/trace.py", line 385, in trace_task
R = retval = fun(*args, **kwargs)
File "/opt/mayan-edms/local/lib/python3.7/site-packages/celery/app/trace.py", line 648, in __protected_call__
return self.run(*args, **kwargs)
File "/opt/mayan-edms/local/lib/python3.7/site-packages/mayan/apps/document_indexing/tasks.py", line 45, in task_index_document
Index.objects.index_document(document=document)
File "/opt/mayan-edms/local/lib/python3.7/site-packages/mayan/apps/document_indexing/managers.py", line 19, in index_document
index.index_document(document=document)
File "/opt/mayan-edms/local/lib/python3.7/site-packages/mayan/apps/document_indexing/models.py", line 124, in index_document
index_instance_node.delete_empty()
File "/opt/mayan-edms/local/lib/python3.7/site-packages/mayan/apps/document_indexing/models.py", line 390, in delete_empty
if self.parent.is_root_node():
File "/opt/mayan-edms/local/lib/python3.7/site-packages/django/db/models/fields/related_descriptors.py", line 184, in __get__
rel_obj = self.get_object(instance)
File "/opt/mayan-edms/local/lib/python3.7/site-packages/django/db/models/fields/related_descriptors.py", line 159, in get_object
return qs.get(self.field.get_reverse_related_filter(instance))
File "/opt/mayan-edms/local/lib/python3.7/site-packages/django/db/models/query.py", line 380, in get
self.model._meta.object_name
mayan.apps.document_indexing.models.DoesNotExist: IndexInstanceNode matching query does not exist.```