You need to sign in or sign up before continuing.
Job.result() may hang forever if exception occurs while checking job's status
I use common way of working with Python API:
app = everest.App(..., session)
job = app.run(...)
try:
result = job.result()
except everest.JobException:
...
This scheme may hang forever if an exception occurs while session checks job statuses in a timer thread. In case of exception the timer doesn't get reset. And the job's state doesn't change. So job.result() never returns.
I had the following exception around Jan 1 06:04:
Exception in thread Thread-44019:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/usr/local/lib/python2.7/threading.py", line 1082, in run
self.function(*self.args, **self.kwargs)
File "/home/vladimirv/mc2/agentLight/tasks/5a4644e914000096663a6cef-0-0/everest.py", line 197, in __checkJobs
job_status = self.getJobStatus(job.id)
File "/home/vladimirv/mc2/agentLight/tasks/5a4644e914000096663a6cef-0-0/everest.py", line 96, in getJobStatus
r = self.session.get(self.endpoint + '/api/jobs/' + job_id)
File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 487, in get
return self.request('GET', url, **kwargs)
File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 475, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 585, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 453, in send
raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', error("(104, 'ECONNRESET')",))