Skip to content

Add operation status lock.

Arber X requested to merge arber/lock_on_dictionary_modification into master

Description

  grpc._server][ERROR]: Exception iterating responses: dictionary changed size during iteration
Traceback (most recent call last):
  File "/python3.6-buildgrid/0.0.2~20190918.1+b20190918T21505995/libexec/python3.6-buildgrid/lib/python3.6/site-packages/grpc/_server.py", line 452, in _take_response_from_response_iterator
    return next(response_iterator), True
  File "/python3.6-buildgrid/0.0.2~20190918.1+b20190918T21505995/libexec/python3.6-buildgrid/lib/python3.6/site-packages/buildgrid/server/execution/service.py", line 108, in Execute
    request.skip_cache_lookup)
  File "/python3.6-buildgrid/0.0.2~20190918.1+b20190918T21505995/libexec/python3.6-buildgrid/lib/python3.6/site-packages/buildgrid/server/execution/instance.py", line 87, in execute
    skip_cache_lookup=skip_cache_lookup)
  File "/python3.6-buildgrid/0.0.2~20190918.1+b20190918T21505995/libexec/python3.6-buildgrid/lib/python3.6/site-packages/buildgrid/server/scheduler.py", line 231, in queue_job_action
    self._update_job_operation_stage(job.name, operation_stage)
  File "/python3.6-buildgrid/0.0.2~20190918.1+b20190918T21505995/libexec/python3.6-buildgrid/lib/python3.6/site-packages/buildgrid/server/scheduler.py", line 523, in _update_job_operation_stage
    self.__peer_message_queues)
  File "/python3.6-buildgrid/0.0.2~20190918.1+b20190918T21505995/libexec/python3.6-buildgrid/lib/python3.6/site-packages/buildgrid/server/job.py", line 375, in update_operation_stage
    peer_message_queues=peer_message_queues)
  File "/python3.6-buildgrid/0.0.2~20190918.1+b20190918T21505995/libexec/python3.6-buildgrid/lib/python3.6/site-packages/buildgrid/server/job.py", line 643, in _send_operations_updates
    relevant_queues = {peer: mqs for peer, mqs in peer_message_queues.items()
  File "/python3.6-buildgrid/0.0.2~20190918.1+b20190918 ... TRUNCATED

Lock to prevent concurrent access when updating operation status.

Edited by Arber X

Merge request reports