KANs can be modeled as just another activation architecture in normal MLPs, which is of course not surprising, since they are very flexible. I made a chart of different types of architectures here: https://x.com/thomasahle/status/1796902311765434694
Curiously KANs are not very efficient when implemented with normal matrix multiplications in Pytorch, say. But with a custom cuda kernel, or using torch.compile they can be very fast: https://x.com/thomasahle/status/1798408687981297844
Curiously KANs are not very efficient when implemented with normal matrix multiplications in Pytorch, say. But with a custom cuda kernel, or using torch.compile they can be very fast: https://x.com/thomasahle/status/1798408687981297844