`meltano ui` warns about "line buffering (buffering=1) isn't supported in binary mode" on Python 3.8
Overview
I'm currently hosting Meltano on production grade. Thanks to state deployment guidelines and best practises in the
Meltano
documentation.
- https://meltano.com/docs/production.html#deployment-in-production
- https://meltano.com/docs/orchestration.html#orchestration
- https://gitlab.com/meltano/files-docker-compose/-/tree/master/bundle
We had a quick chat with @DouweM related to issue. Double checked each of what mentioned on the entire documentation page. However, I'm not able to overcome error by myself. Airflow is not consistent. So, I'm slog on to move forward. I need your helps to fix the error.
I had a try with multiple meltano images that you provide on pipeline meltano/meltano:latest
, latest-python3.6
, latest-python3.7
and latest-python3.8
. I would say the most stable one was 3.6 and the logs belong to it.
- Airflow logs can be found in here: logs.txt
Traceback (most recent call last):
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/flask_admin/base.py", line 69, in inner
return self._run_view(f, *args, **kwargs)
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/flask_admin/base.py", line 368, in _run_view
return fn(self, *args, **kwargs)
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/flask_login/utils.py", line 258, in decorated_view
return func(*args, **kwargs)
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/airflow/www/utils.py", line 384, in view_func
return f(*args, **kwargs)
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/airflow/www/utils.py", line 290, in wrapper
return f(*args, **kwargs)
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/airflow/utils/db.py", line 74, in wrapper
return func(*args, **kwargs)
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/airflow/www/views.py", line 1611, in tree
start_date=min_date, end_date=base_date, session=session)
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/airflow/utils/db.py", line 70, in wrapper
return func(*args, **kwargs)
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/airflow/models/dag.py", line 867, in get_task_instances
tis = tis.order_by(TaskInstance.execution_date).all()
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3373, in all
return list(self)
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 100, in instances
cursor.close()
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
with_traceback=exc_tb,
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 80, in instances
rows = [proc(row) for row in fetch]
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 80, in <listcomp>
rows = [proc(row) for row in fetch]
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 588, in _instance
populators,
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 725, in _populate_full
dict_[key] = getter(row)
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/sqlalchemy/sql/sqltypes.py", line 1723, in process
return loads(value)
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/dill/_dill.py", line 283, in loads
return load(file, ignore, **kwds)
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/dill/_dill.py", line 278, in load
return Unpickler(file, ignore=ignore, **kwds).load()
File "/project/.meltano/orchestrators/airflow/venv/lib/python3.6/site-packages/dill/_dill.py", line 481, in load
obj = StockUnpickler.load(self)
ValueError: unsupported pickle protocol: 5
Similar Issues
Possible fixes
I assume that airflow bundle should be maintained https://gitlab.com/meltano/files-airflow
Further regression test
To reproduce the error follow below steps:
-
Create virtual environment and init -
Populate Meltano project with tap-adwords
,tap-facebook
andtarget-snowflake
- meltano add extractor tap-facebook
- meltano add extractor tap-adwords
- meltano add loader target-snowflake
-
Add Airflow orchestration to the project. -
Build Meltano with docker-compose bundle. - docker-compose -f docker-compose.prod.yml up -d --build
-
Testing & Debugging logs - docker-compose logs airflow-scheduler