Before and After: Measuring Security Posture Improvement With Real Metrics

java dev.to

188 vulnerabilities. That's where we started.

After the modernisation, the jjwt migration, the targeted remediations, and the documented suppressions, here's where we ended up:

6 open findings. All suppressed with documented reasons. 0 unaddressed.

But "188 to 6" is a headline, not a measurement. This article is about what meaningful security posture measurement actually looks like — the metrics that tell a real story versus the ones that just make a dashboard look good.


Why Raw Finding Count Is a Weak Metric

The most common way teams measure SCA progress is finding count. Before: 188. After: 6. Improvement: 182 findings resolved. 97% reduction.

That number is real but it's also misleading in isolation. Here's why.

If I had suppressed all 188 findings without fixing anything, my finding count would also be 6. The dashboard would look identical. The actual security posture would be unchanged.

Finding count measures activity, not outcomes. What you need to measure is:

  • What risk was actually reduced — not just what got closed
  • What risk remains and why — the documented residual risk
  • How the remediation was achieved — fix vs. suppress breakdown
  • What the exploit exposure looks like — before and after exploit maturity These four dimensions together tell a story that a single number can't.

Metric 1: Risk Reduction by Severity

The most important before/after comparison is severity distribution, not total count.

Severity Before After Fixed After Suppressed Remaining
Critical 10 10 0 0
High 99 93 6 0
Medium 59 29 23 0 (7 in 4.x backlog)
Low 20 0 20 0
Total 188 132 49 0 unaddressed

Every Critical finding was fixed — not suppressed, fixed. That's the number that matters most. No Critical vulnerability was accepted as residual risk.

The 6 suppressed High findings are all in the "no known exploit" category with documented unreachability justifications. The 23 suppressed Medium findings are split between test-scope dependencies and unreachable code paths.

What to say when presenting this: "We eliminated all Critical exposure and resolved 94% of High findings through active remediation. The remaining 6 High findings are suppressed with documented justifications — unreachable code paths verified by codebase analysis — and have review dates set for Q2."


Metric 2: Fix vs. Suppress Breakdown

This metric distinguishes genuine security improvement from metric manipulation.

Resolution Type Count % of Total
Fixed — BOM upgrade 89 47%
Fixed — targeted remediation 43 23%
Suppressed — unreachable code path 23 12%
Suppressed — no known exploit 18 10%
Suppressed — test scope 9 5%
Tracked — accepted residual risk 6 3%

70% of findings were actively fixed. 30% were suppressed with documented reasons. Zero were closed without a reason.

The 47% resolved by BOM upgrade is worth highlighting specifically — it demonstrates the leverage of keeping framework versions current. Nearly half the entire vulnerability backlog was resolved by a single architectural change.

What this tells a hiring manager or security team: You didn't just run a scanner and close tickets. You made architectural decisions, did targeted remediations, and applied security judgment to suppression decisions. The breakdown shows the difference.


Metric 3: Exploit Exposure Before and After

Raw CVSS scores don't tell you how much real-world attack risk exists. Exploit maturity does.

Before remediation:

Exploit Maturity Count What It Means
Mature (working exploit exists) 8 Anyone with the tool can exploit this
Proof of concept 43 Requires effort but exploit path is known
No known exploit 137 Theoretical vulnerability, no public exploit

After remediation:

Exploit Maturity Count Status
Mature 0 All fixed
Proof of concept 0 All fixed
No known exploit 6 Suppressed with documentation

Every finding with working exploit code was fixed. Every finding with a published proof of concept was fixed. The 6 remaining findings are all in the "no known exploit" category — the lowest practical risk tier.

This is the metric that matters most for communicating real risk reduction. Going from 8 mature exploits to 0 means an attacker with a public tool can no longer trivially compromise this application. That's a concrete, meaningful security improvement.


Metric 4: Priority Score Reduction

Snyk's priority score (0-1000) combines CVSS, exploit maturity, reachability, and other factors into a single number per finding. It's more nuanced than CVSS alone.

Before: Highest priority score was 919 (spring-web RCE, spring-context RCE, spring-boot-starter-security auth bypass — multiple findings at maximum score)

After: Highest priority score among remaining findings is 329 — the Improper Handling of Case Sensitivity finding in spring-core that requires a major version upgrade to resolve and is suppressed with justification.

The maximum priority score dropped from 919 to 329. The most critical attack vectors have been eliminated.


Metric 5: Dependency Health Score

Beyond individual vulnerabilities, the modernisation improved the overall health of the dependency tree in ways that reduce future vulnerability accumulation.

Before:

  • Spring Boot: 2.0.3/2.0.4 — released May 2018, EOL
  • Spring Framework: 5.0.7 — released June 2018, EOL
  • jjwt: 0.9.1 — released November 2018, unmaintained
  • Java target: 1.8 — EOL for free Oracle support
  • Dependencies: 8 manually pinned, no BOM After:
  • Spring Boot: 3.2.5 — current LTS, actively maintained
  • Spring Framework: 6.1.x — current, actively maintained
  • jjwt: 0.12.0 — current, actively maintained
  • Java target: 17 — current LTS
  • Dependencies: BOM managed, version alignment enforced This isn't just about current CVEs. A project on EOL dependencies accumulates new CVEs constantly as researchers continue to find vulnerabilities in old versions that never receive patches. Moving to actively maintained versions means future CVE disclosures will have patches available.

The dependency health score is a leading indicator — it predicts future vulnerability accumulation, not just current state.


Metric 6: Mean Time to Remediate (Simulated)

In a production environment, MTTR (Mean Time to Remediate) by severity is a standard AppSec programme metric. For this project I can simulate what it would have looked like:

Severity Discovery Date Remediation Date MTTR
Critical (RCE) Day 1 — scan Day 3 — BOM upgrade 2 days
Critical (Auth bypass) Day 1 — scan Day 3 — BOM upgrade 2 days
High (jjwt) Day 1 — scan Day 5 — jjwt migration 4 days
High (Tomcat) Day 1 — scan Day 3 — BOM upgrade 2 days
Medium (MongoDB MitM) Day 1 — scan Day 7 — driver patch 6 days

Average MTTR for Critical: 2 days
Average MTTR for High: 3 days
Average MTTR for Medium: 6 days

Industry benchmarks for production systems typically target Critical remediation within 24-72 hours and High within 7-14 days. This project would have met those benchmarks.


The Dashboard View

If you were presenting this to an engineering team or security leadership, here's how the story looks in dashboard format:

SECURITY POSTURE SUMMARY — pgmpofu/mflix
Scan date: [date]
Previous scan: [date — before remediation]

FINDING SUMMARY
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
                Before    After    Change
Critical          10        0      -100%
High              99        0*     -100%
Medium            59        0*      -100%
Low               20        0*     -100%
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Total            188        6*      -97%

*6 findings suppressed with documented justification
 All suppressions have expiry dates and review schedule

EXPLOIT EXPOSURE
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Mature exploits:   8 → 0    (-100%)
Proof of concept: 43 → 0    (-100%)
No known exploit: 137 → 6   (-96%)

RESOLUTION BREAKDOWN
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Fixed (BOM upgrade):          89  (47%)
Fixed (targeted):             43  (23%)
Suppressed (documented):      56  (30%)
Unaddressed:                   0   (0%)
Enter fullscreen mode Exit fullscreen mode

What These Metrics Don't Measure

Honest measurement includes acknowledging the gaps.

Code-level vulnerabilities. Snyk SCA scans dependencies, not your code. The metrics above say nothing about whether the application code itself has injection vulnerabilities, authentication bypasses, or insecure cryptography. That requires SAST — which is exactly what the SAST tool series was about.

Runtime behaviour. A dependency can be vulnerable without being exploitable in a specific application's runtime configuration. The metrics above are conservative — they count vulnerabilities as present even when compensating controls exist.

New vulnerabilities. The scan represents a point in time. New CVEs are disclosed daily. Without a recurring scan schedule and a process to act on new findings, the posture degrades over time. The metrics are only meaningful if they're refreshed regularly.

The MongoDB 4.x backlog. The 6 tracked residual risk items from the MongoDB driver partial upgrade aren't reflected in these metrics. They're tracked separately and represent the honest incomplete work.


The Number That Actually Matters

Of all the metrics in this article, one number is most important:

0 Critical findings. 0 mature exploits.

Everything else — the 97% reduction, the fix/suppress breakdown, the priority score drop — supports that headline. But that's the number a CISO, a hiring manager, or a security audit cares about.

You started with 10 Critical vulnerabilities including a Remote Code Execution at CVSS 9.8. You ended with zero. Every vulnerability with working public exploit code was remediated. The application went from a state where a knowledgeable attacker with public tools could trivially compromise it, to a state where exploitation requires novel research against theoretical vulnerabilities in unreachable code paths.

That's what security posture improvement looks like in measurable terms.


The remediated repository with full .snyk suppression documentation is at github.com/pgmpofu/mflix.

Final article: I used AI to help remediate vulnerabilities — here's how useful it actually was, where it added value, and where it confidently gave me wrong answers.

Source: dev.to

arrow_back Back to Tutorials