>Perhaps I am missing something, what are you imagining of “a locally ran inference engine” that doesn’t already exist in a DGX workstation, Framework desktop, MacBook, etc?
Someone much smarter than me would figure this out.
One option would be an alternative bare metal OS which could allocate ram more carefully.
I'm sure with billions upon billions in VC funding a solution could be found.
Someone much smarter than me would figure this out.
One option would be an alternative bare metal OS which could allocate ram more carefully.
I'm sure with billions upon billions in VC funding a solution could be found.