votick

first time i’ve found a fellow manitoban on HN greetings !

I can’t use it on mobile looks like which is annoying as we’ve had touch screen phones for almost 20yrs now, the good thing is at least I care enough im annoyed I can’t use it. I’d imagine you’ll get a lot more sharing if mobile works

I thought this was a "your mom is compatible with everyone" joke

This seems completely reasonable pre revenue and I’ve no doubt they make an ai app that 1000x’s everyone’s investment using the best ai technology and algorithms and maybe even some better data. hard to know tho for sure sometimes they lose money sometimes they won’t lose money but ai big and I think ai stay big so more ai

This is clean, nice work!

Hey HN, this is a lil side project I've been working on.

I've ran a ML hosting platform (banana.dev) last few yrs.

Not being able to get GPUs has stressed me out a lot. So took a crack at solving it.

This is a super alpha product, please don't trust it with sensitive workloads or production applications yet. Down to chat if you'd like to use this at scale!

Would love honest feedback, and feel free to AMA! -kyle

author here:

hey HN, we use used to run an ML consultancy for a year that helped companies build & host models in prod. We learned how tedious & expensive it was to host ML. Customer models had to run on a fleet of always-on GPUs that would often get <10% utilization, which felt like a big money sink.

Over time we built infrastructure to improve GPU utilization. Six months ago we made a pivot to focus solely on productizing this infra into a hosting platform for ML teams to use that would remove the pain of deployment and reduce the cost of hosting models.

We deploy on A100 GPUs, and you pay per second of inference. If you aren’t running inferences you pay nothing. Couple points to clarify: Yes, the models are actually cold-booted, we aren’t just running them in the background. We boot models faster due to how we manage OS memory. Yes, there is still cold-boot time, it’s not instant but it’s significantly faster (e.g., 15 seconds instead of 10 minutes for some transformers like GPTJ).

Lastly, model quality is not lost on Banana because we aren’t doing traditional weight quantization or network pruning which makes networks smaller/faster but sacrifices quality. You can think of Banana more as a compiler + hosting platform. We break down your code to run faster on GPUs.

Try it out and let us know what you think!