-
Tim Swast authored
* Add quickstart example DAG. Add first airflow example DAG. This DAG demonstrates how to call Dataproc from airflow. Change-Id: I2f0464da37f11085dfc927f4f4d96d6a99634aad * Add region tags for cloud.google.com. Also, adjusts indentation to 4 spaces so that `flake8` passes. Change-Id: Id3e1fbfa5bd41a4e11e0cd7991b058b063241e6b * Add composer_quickstart_steps region tag. Change-Id: Ib0bbacd91529fac272e2da0c72abca5db9b4fad8 * Test Composer workflow sample. The GCP Airflow operators only work on Python 2.7 (a constraint they inherited from Apache Beam), so I've made some changes to nox.py to exclude them from the Python 3 builds. Change-Id: Ia10971dd5a7b14279b236041836e317e79693258 * Add DAG that runs BigQuery and notifies user. Add a sample Airflow DAG that will create BigQuery dataset, run BigQuery, export results to Cloud Storage, notify user by email when results are ready, and clean up the dataset. Change-Id: Ia8242df29223d910b2d269a9bb93720b35470b7a * Composer: add sample to get GCS bucket for DAGs Since there are not Cloud client libraries for Composer yet, this sample uses the REST API directly via the google-auth and requests libraries. Sample to be used at https://cloud.google.com/composer/docs/how-to/using/managing-dags Also, enables Kokoro testing for Composer samples. (Uses Python 2.7 since Cloud Composer is currently restricted to Python 2.7) Change-Id: Icb37e079992c88eedc06cdcc3d72db5106d10ef5 * Add tests for BQ notify DAG. Requires master copy of Airflow for `bigquery_get_data` operartor. Change-Id: I73cd2cfb2458b67bed1a77e65966d5018e8bb45d * Composer: Fix flake8 errors. Change-Id: I2856bc6cb866bd6f7abbac8de3323797a83c9857 * Composer: add region tags to notification DAG sample. Change-Id: I657e052fa851daa7c72045762090a2e27dd406d3 * Set machine type in quickstart to n1-standard-1. The default machine type was n1-standard-4, which exceeds trial project quota. This CL changes the machine type to n1-standard-1 since a more powerful machine is not necessary for quickstart sample data. Change-Id: I46af68c29145f7a7ce303afdad4708bda7d9e6dd * Add composer config to travis test env. Change-Id: I9c27c75cbea8c5ed4edf859d26980e24ea270eea
This project manages its dependencies using pip.
Learn more