How Effective Are SETA Programs Anyway: Learning and Forgetting in Security Awareness Training

Prevalent security threats caused by human errors necessitate security education, training, and awareness (SETA) programs in organizations. Despite strong theoretical foundations in behavioral cybersecurity, field evidence on the effectiveness of SETA programs in mitigating actual threats is scarce. Since memory decay will inevitably occur after absorbing a broad range of cybersecurity knowledge in a single session, the effectiveness of SETA programs in longer terms is unclear. This study investigates whether and how knowledge gained through SETA programs can mitigate human errors in a longitudinal setting. In a baseline experiment, we established that SETA programs reduce phishing susceptibility by 50%, whereas the training intensity does not affect the susceptibility rate. In a follow-up experiment, we found that SETA programs can increase users' cybersecurity knowledge by 12-17%, but the increment wears off within a month. Furthermore, technical-level knowledge decays faster than application-level knowledge. The longer "shelf-life" of application-level knowledge explains why training intensity makes no difference in the baseline experiment. This study reveals a (relatively) more effective component of SETA programs and casts doubts on the overall effectiveness of SETA programs in the long run.