Random prototype knowledge distillation for cross-architecture knowledge transfer

A widely held assumption in knowledge distillation is that teacher and student representations must occupy a compatible or explicitly aligned feature space. This thesis proposes RPKD (Random Prototype Knowledge Distillation), which discards this assumption by projecting both networks into a shared s