10-16, 16:00–16:20 (Europe/Luxembourg), Salle Europe
Even reputable vendors sometimes have a hard time consistently communicating uncertainties in a single report. This talk will highlight the challenge at the individual analyst level of working with uncertainties and communicating them. Words of Estimative Probability (WEP) and confidence levels, which address intelligence gaps, assumptions, and conclusions, may appear abstract and difficult to grasp for individuals with technical backgrounds who have transitioned to CTI from working with concrete facts. The presentation will explore various approaches to communicating uncertainties, showcasing their respective advantages and disadvantages for different types of threat report consumers.
This talk will present a comparative study of how security vendors utilize Words of Estimative Probability (WEP) and confidence levels, which are tools used in intelligence analysis to convey uncertainties. It aims to shed light on the varying approaches used in the industry.
While the talk will not exhaustively explain why some vendors struggle in this area at an industry level, it will emphasize that working with uncertainties and effectively communicating them can also be challenging for individual analysts.
WEP and confidence levels might appear difficult to grasp. To bridge this gap, the talk will translate these abstract concepts into language that resonates with the technical audience. It will provide practical guidelines for utilizing WEP and offer specific steps to differentiate terms such as "likely" and "highly likely." Additionally, the presentation will explore various approaches to communicating uncertainties, highlighting their respective advantages and disadvantages for different types of threat report consumers.
Some logical approaches that effectively combine WEP and confidence levels may be complex for untrained readers to comprehend. However, alternative methods that deviate from standard intelligence analysis tradecraft could be viable in certain cases. Regardless of the chosen approach, transparency and consistency are essential considerations for any CTI team, including security vendors.
Ondra Rojcik is a Senior Cyber Threat Intelligence Analyst at Red Hat CTI team. He is providing intelligence analysis and strategic perspective to the Red Hat’s CTI program and its analytical production. Previously he worked for the Czech National Cyber and Information Security Agency (NUKIB) as a Deputy-Director of Department and Head of Strategic Analysis Unit which he co-founded.