As it happens, I just got my employer's permission to release as open source a Triton back-end for Metal and D3D12 GPUs here: https://github.com/dropbox/neso .
Finally, I am also going to be switching the inferencing part of Witchcraft from current Candle on MacOS and OpenVINO on Windows to just Candle with Neso; https://github.com/dropbox/witchcraft
As an example of you how can use it to deploy real models there is this project doing ASR and TTS: https://github.com/dropbox/nspeech .
Finally, I am also going to be switching the inferencing part of Witchcraft from current Candle on MacOS and OpenVINO on Windows to just Candle with Neso; https://github.com/dropbox/witchcraft