I recently had to create a reproducible version of incredibly complicated and messy R concoctions our data scientists came up with.
I did it with pandas without much experience with it and a lot of AI help (essentially to fill in the blanks the data scientists had left, because they only had to do the calculation once).
I then created a polars version which uses lazyframes. It ended up being about 20x faster than the first version. I did try to do some optimizations by hand to make the execution planner work even better which I believe paid off.
If you have to do a large non interactive analytical calculation (i.e. not in a notebook) polars seems to be way ahead imo!
I do wish that it was just as easy to use as a rust library though.. the focus however seems to be on being competitive in python land mainly.
I did it with pandas without much experience with it and a lot of AI help (essentially to fill in the blanks the data scientists had left, because they only had to do the calculation once).
I then created a polars version which uses lazyframes. It ended up being about 20x faster than the first version. I did try to do some optimizations by hand to make the execution planner work even better which I believe paid off.
If you have to do a large non interactive analytical calculation (i.e. not in a notebook) polars seems to be way ahead imo!
I do wish that it was just as easy to use as a rust library though.. the focus however seems to be on being competitive in python land mainly.