Two_hands
Joined 5 years ago
Karma
159
Submissions
63
Member since
Aug 31
https://ym2132.github.io
View on Hacker NewsRecent submissions
Comment · 7 months ago
I tried to build something: https://github.com/YM2132/PR_guard which aims to help in these cases. It's not perfect but with stronger AI detection tools (Pangram) it could be improved although the issue of cost then arises and who pays for it.
Comment · 10 months ago
Agreed, I have seen some speedups with ONNX if I'm being honest but the process especially on MacOS is a bit messy. I'll try out Executorch and see how it compares, cheers for the recommendation
Comment · 10 months ago
To be honest it's a shame the whole thing is closed up, I guess it's to be expected from Apple, but I reckon CoreML would be benefit a lot from at least exposing the internals/allowing users to define new ops.
Also, the ANE only allows some operators to be ran on it right? There's very little transparency/control on what can be offloaded to it and cannot which makes using it difficult.
Comment · 10 months ago
Wow I didn't know that.
The worst part of it is as you say we all accept it and no one talks about it.
Is there any recommended reading you'd suggest to look into this more and the impacts of it?
Comment · 10 months ago
Yeah I can see why they let it be that way, but the fact it is pretty undefined is what bugged me. I suppose it depends on what your goals are - efficiency vs reproducibility.
Also I did run a test of FP16 vs FP32 for a large matmul on the Apple GPU and the FP16 calculation was 1.28x faster so it makes sense that they'd go for FP16 as a default.
Comment · 10 months ago
cheers for the tip, I'll give it a go
Comment · 10 months ago
Thank you for reading.
Also generally I think CoreML isn't the best. The best solution for ORT would probably be to introduce a pure MPS provider (https://github.com/microsoft/onnxruntime/issues/21271), but given they've already bought into CoreML the effort may not be worth the reward for the core team. Which fair enough as it's a pretty mammoth task
Comment · 10 months ago
Working on a blog post about default behaviour in ONNX Runtime when using the CoreML execution provider. Basically the default args lead to your model being ran in FP16 not FP32.
You can find more details at my site soon: https://ym2132.github.io
Comment · 11 months ago
interesting, $10 per hour is pretty reasonable.
ha, thats probably why I noticed the EyesOff accuracy drops so much at longer ranges, I suppose two models would do better but atm battery drain is a big issue.
I'm not sure if it's important or not, but the app comes from my own problems working in public so I'm happy to continue working on it. I do want to train and deploy an optimised model, something much smaller.
Sounds great, once a POC get's built I'll let you know and can see about the clinical side.
Thanks for the tips! I'll be sure to post something and reach out if I get round to implementing such a model.