A new article by Woods and Seymour (2024) in the Journal of Cyber Policy provides a refreshing approach to the identification and establishment of appropriate cybersecurity policies. The article, titled “Evidence-based cybersecurity policy: A meta-review of security control effectiveness,” evaluates the effectiveness of cybersecurity policies and controls over a ten-year period. The literature included in their meta-analysis only included studies based on empirical evidence that investigated the statistical relationship between security controls and firm-level cyber risk outcomes in the real world. The following table lists the studies included in the meta-analysis.
The key findings can be summarized as follows:
Most Effective Interventions
- Attack Surface Management: Consistently emerged as the strongest predictor of cyber incidents across studies. This encompasses various hardening measures and system configuration practices.
- Patch Cadence: Ranked as the second most effective intervention. The speed at which security patches are applied significantly impacts an organization’s cyber risk profile.
Other Notable Findings
Multi-Factor Authentication (MFA): Showed high efficacy for individual account protection (99% risk reduction for Microsoft Azure AD accounts), but its effectiveness varied when implemented organization-wide.
Cloud vs. On-Premises Email: Organizations using cloud-hosted email services experienced lower claim frequencies compared to those with on-premises Exchange servers.
VPN Providers: Specific VPN providers were associated with significantly higher rates of compromise, highlighting the importance of vendor selection.
Monitoring Activities: While not as impactful as hardening and patch management, various types of monitoring were associated with reduced claims costs.
Implications for Cybersecurity Practices
- Focus on Processes Over Products: The evidence suggests that how controls are implemented matters more than which specific products are used. This emphasizes the importance of proper configuration and maintenance.
- No Silver Bullets: The review found little evidence supporting the efficacy of off-the-shelf security solutions. Instead, it highlights the importance of system configuration and maintenance.
- Secure-by-Design Approach: The findings support the idea that secure-by-design principles are more effective than bolt-on security solutions.
- Risk-Based Decision Making: The evidence suggests that policy measures mandating specific controls are unlikely to result in significant risk reduction. Instead, organizations should be supported in making risk-based decisions.
Limitations and Future Directions
- Causality Issues: Most studies established correlations rather than causal relationships between controls and outcomes.
- Measurement Challenges: Many studies relied on externally observable data, potentially missing the impact of internal controls.
- Historical Validity: The evolving threat landscape may limit the applicability of past findings to future risks.
- Data Availability: Improving public access to cybersecurity incident data could enhance future research and policy formulation.
For cybersecurity specialists, this meta-review underscores the importance of focusing on fundamental security practices like attack surface reduction and efficient patch management. It also highlights the need for continuous evaluation of security measures in light of emerging threats and changing IT landscapes.
Reference:
Woods, D. W., & Seymour, S. (2024). Evidence-based cybersecurity policy? A meta-review of security control effectiveness. Journal of Cyber Policy. Advance online publication. https://doi.org/10.1080/23738871.2024.2335461
