I find myself, again and again, asking if Rust will be for DE. The thing that delta lake and datafusion exist or invest in Rust is a good sign. But after all, Rust is a systems programming language and, therefore, compared to C++. I assume things that have to be fast, like delta lake, that interact with the file system and process lots of data, will be written in Rust. But as a data engineer, a level higher up speed is important, not that essential in most cases.

And on the other hand, it seems that Scala has more ideal tools that suit real fast data wrangling. But as you said, at the end of the day, you need to choose the tools that fit best, I agree—looking forward to how this will turn out.