Crash handling SIDN uploads
This causes the worker to crash and retry. Probably due to incorrect labels. Probably not covered in testcases.
2020-09-27 13:16 ERROR - Task websecmap.api.logic.sidn_handle_domain_upload[92972bf2-5dea-4777-8e2f-4c038727bcf6] raised unexpected: UnicodeError("encoding with 'idna' codec failed (UnicodeError: label empty or too long)")
Traceback (most recent call last):
File "/usr/lib/python3.8/encodings/idna.py", line 165, in encode
raise UnicodeError("label empty or too long")
UnicodeError: label empty or too long
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/pyenv/lib/python3.8/site-packages/celery/app/trace.py", line 412, in trace_task
R = retval = fun(*args, **kwargs)
File "/pyenv/lib/python3.8/site-packages/celery/app/trace.py", line 704, in __protected_call__
return self.run(*args, **kwargs)
File "/pyenv/lib/python3.8/site-packages/sentry_sdk/integrations/celery.py", line 171, in _inner
reraise(*exc_info)
File "/pyenv/lib/python3.8/site-packages/sentry_sdk/_compat.py", line 57, in reraise
raise value
File "/pyenv/lib/python3.8/site-packages/sentry_sdk/integrations/celery.py", line 166, in _inner
return f(*args, **kwargs)
File "/source/websecmap/api/logic.py", line 199, in sidn_handle_domain_upload
has_been_added = existing_second_level_url.add_subdomain(new_subdomain, "added via SIDN")
File "/usr/lib/python3.8/contextlib.py", line 75, in inner
return func(*args, **kwds)
File "/source/websecmap/organizations/models.py", line 393, in add_subdomain
if not Url.is_valid_url(new_url):
File "/source/websecmap/organizations/models.py", line 449, in is_valid_url
valid_domain = domain(url.encode("idna").decode())
UnicodeError: encoding with 'idna' codec failed (UnicodeError: label empty or too long)
[2020-09-27 13:16:43,458: ERROR/ForkPoolWorker-4] Task websecmap.api.logic.sidn_handle_domain_upload[92972bf2-5dea-4777-8e2f-4c038727bcf6] raised unexpected: UnicodeError("encoding with 'idna' codec failed (UnicodeError: label empty or too long)")
Traceback (most recent call last):
File "/usr/lib/python3.8/encodings/idna.py", line 165, in encode
raise UnicodeError("label empty or too long")
UnicodeError: label empty or too long
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/pyenv/lib/python3.8/site-packages/celery/app/trace.py", line 412, in trace_task
R = retval = fun(*args, **kwargs)
File "/pyenv/lib/python3.8/site-packages/celery/app/trace.py", line 704, in __protected_call__
return self.run(*args, **kwargs)
File "/pyenv/lib/python3.8/site-packages/sentry_sdk/integrations/celery.py", line 171, in _inner
reraise(*exc_info)
File "/pyenv/lib/python3.8/site-packages/sentry_sdk/_compat.py", line 57, in reraise
raise value
File "/pyenv/lib/python3.8/site-packages/sentry_sdk/integrations/celery.py", line 166, in _inner
return f(*args, **kwargs)
File "/source/websecmap/api/logic.py", line 199, in sidn_handle_domain_upload
has_been_added = existing_second_level_url.add_subdomain(new_subdomain, "added via SIDN")
File "/usr/lib/python3.8/contextlib.py", line 75, in inner
return func(*args, **kwds)
File "/source/websecmap/organizations/models.py", line 393, in add_subdomain
if not Url.is_valid_url(new_url):
File "/source/websecmap/organizations/models.py", line 449, in is_valid_url
valid_domain = domain(url.encode("idna").decode())
UnicodeError: encoding with 'idna' codec failed (UnicodeError: label empty or too long)