You can prove anything with statistics. What if I told you that in this, the most exciting Premier League football season in years, one of the most valuable players in the entire division was the backup goalkeeper of struggling Watford, Giedrius Arlauskis. With his stunning, flawless 100% win ratio, he must be the best!Obviously this is not true, and Messrs Vardy and Kane, of champions Leicester City and Tottenham Hotspur respectively, need not worry. Mr Arlauskis played just a single Premier League game (out of 38) before being sent back to warm the bench, and will not be troubling the awards ceremonies any time soon.As an industry, cyber security tends to be obsessed with statistics. In an uncertain world, grasping at concrete numbers, either to hang your sales message on (suppliers), or to report upstairs to the board (buyers), provides a welcome opportunity to differentiate yourself from the competition or demonstrate improvement over time. But do these ‘facts’ obscure more important truths?Here are some example claims………………
- “Our SIEM ingests 10,000 events per second – up from 8,000 last year”
- “Our AV signature database is updated 200% faster than our nearest competitor”
- “Our MSSP updates 20,000 IP addresses daily – double the industry average”
These are all well and good – but really, so what? An advanced attacker isn’t bothered if your SIEM is taking more log sources than last year, or if your MSSP has a handle on a few more intel feeds, as multiple attack options remain. Our reliance on improved statistics is just the equivalent of Mr. Arlauskis playing his one game and calling himself the best player in the league.
Fewer breaches, earlier detection, less damage – the only stats that matter
The point is brought home by this year’s Verizon Data Breach and Investigations Report (DBIR). If our suppliers and buyers are consistently improving on their statistics, how is it that attackers were able to breach corporate infrastructure faster than ever before? And how, despite all the numerical improvements trumpeted by the industry, does it now take longer for breaches to be detected?
To add insult to injury, the most common detection mechanism was the ‘knock on the door’ from external law enforcement, rather than any internal detection control.
The industry needs to take a step back from the numbers, and start focusing instead on the attack paths. As an annual contributor to the DBIR, MWR consistently responds where threat actors have evaded traditional MSSP and SIEM controls, and even some more advanced EDR tools, by exploiting the attack paths that remained uncovered.
We feel that a different approach to detection is needed, one that focuses on how advanced threats actually operate, rather than emphasising what we have always done, celebrating the fact that we are doing more of it faster, and simply hoping that this will produce an improvement in the only statistics that really matter – fewer breaches, earlier detection, less damage.
In other words, we need to start looking at our final scores and league position, rather than our passes completed, tackles made and average distance covered per player. That way we may achieve something like the equivalent of Leicester’s final impregnability rather than Villa’s season-long vulnerability
Categories