Comment by 0xecro1

Interesting approach. I work in embedded Linux/edge AI where we constantly struggle to move knowledge from large training models down to quantized INT8 models on constrained hardware (ARM Cortex-A class). Have you tested transfer to quantized or pruned targets? If the behavioural encoding survives that compression, this could be a much cleaner path than classical distillation for on-device deployment.