Also check out alternatives to airflow, especially dagster is a good one where it’s similar to dbt, but you can combine sql, pyspark, scala, jupyter notebooks, and so on: https://www.quora.com/What-are-common-alternatives-to-Apache-Airflow/answer/Simon-Spaeti