This repository contains Docker and Docker Compose files for building and running Apache Airflow 2.0 as described in our Medium article.
For simple usage and running with no additional dependencies use running with the official image. If additional configuration is needed use custom image running instructions below.
make run
- Configure Airflow running parameters (.env)
- Configure Dockerfile-dev and docker-compose.dev.yml
- Add dependencies in requirements.txt
make run-dev
- Airflow UI: localhost:8080
- Username: admin
- Password: admin
Note: username and password can be configured in /scripts/entrypoint.sh
It's possible to set any configuration value for Airflow from environment variables in .env file where we can set airflow running variables to override the default configuration that is generated as airflow.cfg file after the initial run.
The general rule is the environment variable should be named
AIRFLOW__<section>__<key>, for example AIRFLOW__CORE__SQL_ALCHEMY_CONN sets
the sql_alchemy_conn config option in the [core] section.
Check out the Airflow documentation for more details
You can also define connections via environment variables by prefixing them
with AIRFLOW_CONN_ - for example AIRFLOW_CONN_POSTGRES_MASTER=postgres://user:password@localhost:5432/master
for a connection called "postgres_master". The value is parsed as a URI. This
will work for hooks etc, but won't show up in the "Ad-hoc Query" section unless
an (empty) connection is also created in the DB