📊 Security Metrics: Measuring What Actually Matters
"If you can't measure it, you can't prove it works. If you measure the wrong thing, you prove nothing."
🍎 The Golden Apple: Vanity Metrics vs. Real Security (or: How to Lie with Statistics While Feeling Secure)
Security teams love metrics. Dashboards full of numbers. Colorful charts. Executive briefings with upward-trending lines. It's security theater with better production values.
Most security metrics measure the wrong things. FNORD. Are you measuring security or measuring the appearance of measuring security? (Asking for a friend. The friend is paranoia.)
Number of policies written? Irrelevant if unenforced (PDF doesn't stop hackers). Security training completion rate? Useless if employees still click phishing links (compliance ≠ comprehension). Vulnerability scan count? Meaningless without patch deployment speed (scanning vulnerabilities you never fix is like diagnosing cancer and celebrating the diagnosis). The security-industrial complex sells tools that generate metrics that prove you bought tools. Circular reasoning is circular.
Measure outcomes, not activities. Measure risk reduction, not effort expended. Or keep measuring inputs and wondering why outputs still suck. Your choice. Nothing is true.
ILLUMINATION FOR THE INITIATED: Vanity metrics make executives feel good (dopamine hit from green dashboards). Real metrics reveal uncomfortable truths (like that your security posture is held together by hope and duct tape). Choose truth over comfort—security depends on it. But truth is painful. Which is why most organizations choose comfort. And get breached. The cycle continues. FNORD.
🛡️ The Five Categories of Security Metrics That Matter
1. Detection & Response
How fast do you detect and stop attacks?
MTTD: Mean Time To Detect (hours/days). MTTR: Mean Time To Respond (hours). MTTR: Mean Time To Recover (hours).
2. Vulnerability Management
How fast do you patch critical risks?
Time to patch critical CVEs: Days from disclosure to deployment. Open high/critical vulnerabilities: Absolute count, trending down.
3. Incident Trends
Are you getting better or worse?
Incidents per month: Trending down? Severity distribution: More critical or more informational? Repeat incidents: Learning from failures?
4. Access Control
Who has access to what?
Accounts with excessive privileges: Count, review frequency. Unused accounts: Dormant credentials are risk. MFA coverage: Percentage of critical systems.
5. Security Awareness
Do users fall for attacks?
Phishing simulation click rate: Percentage clicking malicious links. Reported suspicious emails: User vigilance indicator. Policy violations: Incidents from user mistakes.
CHAOS ILLUMINATION: Metrics drive behavior. Measure the wrong thing, get the wrong outcome. Measure vulnerabilities found? Teams stop looking. Measure vulnerabilities fixed? Teams hunt obsessively.
📋 Hack23's Security Metrics Dashboard
Our metrics program focuses on risk reduction: ISMS-PUBLIC Repository | Security Metrics
- OpenSSF Scorecard Target: ≥7.0/10 — Supply chain security assessment across all repos (CIA: 7.2, Black Trigram, CIA CM)
- SLSA Level 3 Build Provenance — Cryptographically signed attestations for all releases (non-falsifiable provenance)
- CII Best Practices: Passing+ — Open source maturity badges (CIA, Black Trigram, CIA CM all passing)
- SonarCloud Quality Gates: Passed — Zero high/critical vulnerabilities, <3% duplication, ≥80% coverage target
- Vulnerability SLAs — Critical CVEs: 7 days, High: 30 days, Medium: 90 days (live tracking in GitHub Security)
- FOSSA License Compliance — Automated SBOM generation, continuous dependency license monitoring
- GitHub Advanced Security — Secret scanning, Dependabot alerts, CodeQL SAST on every commit
- AWS Security Services — GuardDuty threat detection, Security Hub findings, Config compliance, Inspector vulnerabilities
META-ILLUMINATION: Security metrics aren't about proving you're perfect—they're about proving you're improving. Trend matters more than absolute numbers.
🎯 Vanity Metrics vs. Real Metrics
🎭 Vanity Metrics
- ❌ Number of security policies
- ❌ Training completion percentage
- ❌ Vulnerability scans performed
- ❌ Security tools purchased
- ❌ Compliance certifications held
- ❌ Security team size
Why they're useless: Measure activity, not outcome.
✅ Real Metrics
- ✅ Mean Time To Detect/Respond
- ✅ Phishing simulation click rate
- ✅ Critical vulnerabilities patched <48h
- ✅ False positive rate in alerts
- ✅ Repeat incidents (learning failure)
- ✅ Accounts with excessive privileges
Why they matter: Measure security posture improvement.
🔍 The Five Principles of Effective Security Metrics
- Measure Outcomes, Not Activities - "Vulnerabilities fixed" > "vulnerability scans run"
- Focus on Trends, Not Snapshots - Direction matters more than absolute numbers
- Make Metrics Actionable - Every metric should drive specific decisions
- Avoid Perverse Incentives - Don't measure what can be gamed without improving security
- Report Honestly - Metrics revealing problems are valuable—hiding problems is deadly
ULTIMATE ILLUMINATION: Security metrics should make you uncomfortable. If your dashboard shows only green, you're measuring the wrong things—or lying to yourself.
🎯 Metrics for Different Audiences
Different stakeholders need different metrics:
Executive Metrics
- Risk reduction over time
- Compliance status
- Incident trend (severity & frequency)
- Budget efficiency (risk/$)
Technical Team Metrics
- MTTD/MTTR by incident type
- Vulnerability age distribution
- Alert false positive rate
- Coverage gaps by asset type
Audit Metrics
- Control effectiveness evidence
- Policy compliance rates
- Access review completion
- Training completion & testing
🎯 Conclusion: Measure What Matters
Security metrics prove you're improving—or reveal you're not. Measure outcomes, not activities. Measure risk reduction, not effort.
MTTD and MTTR show detection capability. Patching speed shows vulnerability management. Incident trends show learning effectiveness. Phishing rates show awareness impact.
Vanity metrics stroke egos. Real metrics drive improvement. Choose discomfort over delusion.
All hail Eris! All hail Discordia!
"Think for yourself, schmuck! Question your metrics—especially when they make you look good."
🍎 23 FNORD 5
— Hagbard Celine, Captain of the Leif Erikson