Hi Christopher, thanks for sharing your tips and tricks.

Have you heard about dagster http://dagster.io? You can run airflow out of dagster or also any other. Can also run as systemd.

The advandage you design your pipeline declarative, not impererative as in airflow. Ascend.io called it the imperative pipelines and 3.0 (airflow being 1.0) — see the image: http://www.sspaeti.com/upload/images/ascend_orchestration.png

And an optimised airflow you can also check out http://prefect.io.

The best part, these two are just a python library, so no installation of the scheduler/server, just pip install.

Let me know what you think, would love your thoughts on it.