Looks like a pretty cool community building great tools with care. Using a functional language for data transforms sounds like a sane idea, haven't played around with it yet but it's definitely on my list now.
However they claim using Haskell for data science is "fast", which doesn't really mean anything until you have numbers to show. A little benchmark with pandas and polars wouldn't hurt I guess.
Depends what you're doing, but it really can have C-comparable speed. [0]
Unoptimised/naive Haskell might be that slow. But that's true of a lot of languages and isn't particularly interesting to me. Java is slow if you do everything the naive way, too.
Funny how you mention it is a functional programming torture course, but the Parquet writer introduced in this article is written in a highly imperative style. I have not seen this much IORefs and MutableByteArrays in recent memory.
However they claim using Haskell for data science is "fast", which doesn't really mean anything until you have numbers to show. A little benchmark with pandas and polars wouldn't hurt I guess.
Unoptimised/naive Haskell might be that slow. But that's true of a lot of languages and isn't particularly interesting to me. Java is slow if you do everything the naive way, too.
[0] https://entropicthoughts.com/on-competing-with-c-using-haske...
Should be mandatory