Tiny Model, Big Shield: Adaptive Distillation for Intrusion Detection

Online attacks are growing. They need fast security tools. But many deep models are too big. They need lots of memory and power. Small edge devices cannot run them. These devices have limited battery. They also have weak processors.

A teacher network learns to spot attacks. Then a student network copies it. The teacher is trained first. The student tries to match its output. The teacher's output is changed with a temperature. The temperature changes with each example. The outputs are also lined up with a Z‑score. The Z‑score keeps the numbers comparable. The student also learns from the true labels.