Airtable import is failing sometimes due to Brotli decoding issue
Describe the problem
Airtable import is failing sometimes.
When I debugged into it, I found that the difference between successful/failing responses is that:
- For successful responses, the Content-Encoding is
gzip
. - For failing responses, the Content-Encoding is
br
.
We should add Brotli
to the dependencies in order to fix it.
Steps to reproduce
- Try many times importing from Airtable any base.
- Sometimes it works, and sometimes it fails.
Actual result
[2023-02-07 10:10:17,457: ERROR/ForkPoolWorker-2] Task baserow.core.jobs.tasks.run_async_job[ef86776a-4ab8-4b74-a949-8bc625a65706] raised unexpected: AttributeError("'NoneType' object has no attribute 'group'")
Traceback (most recent call last):
File "/baserow/venv/lib/python3.9/site-packages/celery/app/trace.py", line 451, in trace_task
R = retval = fun(*args, **kwargs)
File "/baserow/venv/lib/python3.9/site-packages/celery/app/trace.py", line 734, in __protected_call__
return self.run(*args, **kwargs)
File "/baserow/backend/src/baserow/core/jobs/tasks.py", line 73, in run_async_job
raise e
File "/baserow/backend/src/baserow/core/jobs/tasks.py", line 36, in run_async_job
JobHandler().run(job)
File "/baserow/backend/src/baserow/core/jobs/handler.py", line 62, in run
return job_type.run(job, progress)
File "/baserow/backend/src/baserow/contrib/database/airtable/job_type.py", line 118, in run
databases, id_mapping = AirtableHandler.import_from_airtable_to_group(
File "/baserow/backend/src/baserow/contrib/database/airtable/handler.py", line 567, in import_from_airtable_to_group
request_id, init_data, cookies = cls.fetch_publicly_shared_base(share_id)
File "/baserow/backend/src/baserow/contrib/database/airtable/handler.py", line 78, in fetch_publicly_shared_base
request_id = re.search('requestId: "(.*)",', decoded_content).group(1)
AttributeError: 'NoneType' object has no attribute 'group'
Expected result
- Should import correctly.
Environment
N/A