Comment

Comment by timfi

It may be "close[r] to the data", but be ware Python UDFs usually still have context switches between the python interpreter and the query executor. If these pile up — as is the case if you fire off queries in, say, a loop — you will still see lousy performance. There is currently research under way, to optimize UDFs and other forms of computations over database resident data. For Python, for example, you can checkout the python to SQL compiler [0] and associated demo-paper [1] we produced for this years Sigmod. (Disclaimer: as you may have noticed thanks to that " we", I'm an author on that paper.) Though it may only support a limited subset of Python, one can already do alot with just that.

[0]: https://apfel-db.informatik.uni-tuebingen.de [1]: https://db.inf.uni-tuebingen.de/publications/2022/Hirn-grust...

parent timfi 4 years ago

No comments yet

Be the first to share your thoughts.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton