Simon Späti
Feb 15, 2022

--

Thanks for the write-up! I love this, especially as both #prefect and #dagster have now integrated into Airbyte, making it even easier. That said, I would say you cannot mention Prefect without mentioning Dagster. And my opinion between these is if you have a small toy project, it doesn't matter. But if you use it at work with several different teams or different people, go for dagster. It has much better higher-level abstractions with resources and assets that allow you to debug locally and deploy on Kubernetes with the same code. Unit tests and testing, in general, is included and and and (check their docs, 1A!), that makes you very happy in the long run. It's also hugely easy to set up (one-liner), and the slack is helping everyone asking (same as prefect). I hope that helps.

The link to the dagster integration for Airbyte: https://www.youtube.com/watch?v=oOOuQQsnPTM&t=8s&ab_channel=Airbyte

--

--

Simon Späti
Simon Späti

Written by Simon Späti

Data Engineer & Technical Author with 15+ years of experience. I enjoy maintaining awareness of new innovative and emerging open-source technologies.

Responses (1)