felipe_aramburu

This is actually going to be pretty sick for us at BlazingSQL. We are freaking frothing at the chance to be able to scale beyond system and gpu memory without having to shove data through pipes that are no longer needed.

RDMA will be a necessary building block for most of us that are working on HPC solutions and the fact that now we can directly access fast persistent storage from a GPU is going to allow us to scale several orders of magnitude bigger than we currently can.

Have you compared performance between your suggested solutions and what can be achieved using hardware vendor platforms? If not then whats kind of pathetic is how quickly you dismiss the people above who say the HAVE done this before.

If you have seen something we have not when it comes to performance then please by all means share it so we can learn!

When you say SQL format do you mean being able to read the output of a jdbc or odbc driver? If this is the case then mostly just time. You are not the first person to ask about this and now that there are java bindings in cudf this might become easier to make a reality in the next few months.

Or do you mean being able to read a database's file format natively? If this is the case there are many reasons. 1. There are many poorly/non documented formats 2. Even if you decide to read some other DB's format natively, those formats change over time 3. Little control of how and where the data is laid out

Well it pretty much always was a part of the eco system it just was not open source. We have been contributors to rapids for a while. And yes, we are betting on Nvidia for sure.

For most people building GP GPU solutions they are going to have to make a decision when it comes to which hardware they want to support. After that decision is made it really isn't something you can revisit without copious amounts of money.

Thanks Leo! We love having you all as early adopters of our tech!

GPU memory is expensive but a big as #@$% computer is even more expensive. When we show comparisons to things like spark we are doing so use cost basis. So if we say something like we are x times faster than this technology on this workload what we did was launch clusters that have similar costs. Total cost of ownership is also reduced by the fact that the engine itself is totally ephemeral. You can turn it off and on within seconds.

The collab link below let's you use a gpu for free on Google cloud

Thats a great question. The answer is two-fold.

Early on when we first started playing around with General Processing on GPU's we had Nvidia cards to begin with and I started looking at the apis that were available to me.

The CUDA ones were easier for me to get started, had tons of learning content that Nvidia provided, and were more performant on the cards that I had at the time compared to other options. So we built up lots of expertise in this specific way of coding for GPUS. We also found time and time again that it was faster than opencl for what we were trying to do and the hardware available to us on cloud providers was Nvidia GPUs.

The second answer to this question is that blazingsql is part of a greater ecosystem. rapids.ai and the largest contributor by far is Nvidia. We are really happy to be working with their developers to grow this eco system and that means that the technology will probably be CUDA only unless we somehow program "backends" like they did with thrust but that would be eons away from now.

Blazingsql is built on top of CUDF. We are contributors to rapidsai