Yeah; plus we know they're designing with potential future local-only Vs. hybrid Vs. cloud-only in mind depending on hardware/market changes. So having it be this flexible buys them a ton of future potential.
People have speculated that with FPGAs or even an ASIC you could take some smaller models we have today, turn the chip into essentially that model-as-hardware, and get some really compelling performance/power-usage characteristics.
At that point running a GOOD quantized Q8 model on a phone is not beyond the realm.
People have speculated that with FPGAs or even an ASIC you could take some smaller models we have today, turn the chip into essentially that model-as-hardware, and get some really compelling performance/power-usage characteristics.
At that point running a GOOD quantized Q8 model on a phone is not beyond the realm.