Simon Späti
1 min readFeb 28, 2020

--

Hi Sean Knight, thanks for sharing these tools which of some I haven’t even heard from. These are all good tools, but I believe there is a kind of shift in ETL tools. Ascend.io called it the imperative pipelines and 3.0 (airflow being 1.0) — see the image: http://www.sspaeti.com/upload/images/ascend_orchestration.png.

But there is already open-source tools to do the imperative pipelining called http://dagster.io. And optimised airflow called http://prefect.io. Check it out, you won’t be disappointed, and of course, all python based. Even better, these two are just a python library, so no installation of scheduler/server, just pip install.

--

--

Simon Späti
Simon Späti

Written by Simon Späti

Data Engineer & Technical Author with 15+ years of experience. I enjoy maintaining awareness of new innovative and emerging open-source technologies.

Responses (1)