Lost in Translation? Making Pentest Reports Speak the Client’s Language
Maria P. Murumaa
Introduction
Penetration testing (PT) reports play a significant role in helping organizations identify and mitigate security vulnerabilities as they are the only tangible product of the conducted tests. The report effectiveness relies on the extent to which customers can translate the findings into actionable decisions.
Our study investigates usability gaps in penetration testing reports from a customer-centric perspective, focusing on the challenges organizations face in understanding, prioritizing, and acting on the provided insights.
Study
We conducted the study with IT professionals from various companies that consume PT reports. These studies took place during workshop events held in Czechia and Estonia. More than 50 participants attended the workshop in Czechia and 32 participants in Estonia.
The study included the following steps:
• Demonstration of a PT scenario – The goal was to show participants how a specific PT scenario is conducted, enabling them to assess the content of the vulnerability finding in the report and identify what they would like to see included.
• Survey – Participants reviewed a report corresponding to the demo. The survey captured their general perceptions and feedback on its content and usability.
• Focus group discussion – Structured, in-depth discussions designed to uncover and explore penetration testing consumers’ expectations, pain points, and preferences regarding reports.
Key findings
Our analysis indicated some differences between the views of technical and managerial participants. For example, for managerial roles it is important that the PT report includes the executive summary, definition of scope and detailed description of testing methodology. On the other hand, more technical roles highlighted the crucial parts as detailed step-by-step explanations of findings and actionable recommendations.
The list below highlights selected actionable findings for improving penetration testing reports to better meet client needs and expectations:
• Machine readability – Machine readability in PT reports refers to the format and structure of these documents being optimized for automated processing by software tools, rather than being exclusively human-readable, as they are typically provided (e.g. PDF). Reports in standardized formats, such as JSON, XML, or CSV, with clearly defined schemas, could improve efficiency.
• Additional resources – Including additional resources was shown to be essential to replicate the testing process, allowing the organization to verify the vulnerability and better understand its root cause.
• Multiple mitigations – Participants have expressed the need for multiple mitigation options in PT reports, rather than a single solution. Providing a variety of mitigation strategies would allow organizations to choose an approach that best fits their resources, risk tolerance, and operational constraints. This flexibility would ensure that remediation efforts are both effective and practical, accommodating different technical environments and business priorities.
• Mitigation impact – Participants emphasized the importance of including the mitigation impact in PT reports. In addition to multiple mitigation options and a preferred solution, they want a clear explanation of how each mitigation would affect the system, security posture, and business operations.
• Preferred mitigation – In addition to multiple mitigation options, participants have indicated a preference to have a preferred mitigation clearly highlighted in PT reports. This approach would allow decision-makers to balance between ideal and practical mitigation strategies.
• Target groups – Labeling the proposed mitigation, with labels suggesting the role typically responsible for the mitigation. For example, for an issue in the development part, use the label “dev”; for a configuration problem, use the label “config”. Using straightforward language, well organized sections, and consistent terminology could make reports more accessible to both technical and non-technical stakeholders.
• Positive findings – Writing positive findings alongside identified vulnerabilities. By highlighting areas where security measures are functioning correctly, reports can provide a balanced view that not only identifies vulnerabilities, but also acknowledges strengths.
Side quest (results soon)
We observed that the majority of feedback was related to the recommendations. In response, we revised and improved the recommendations in an example report and collected feedback from over 200 security professionals. While the data is still being analyzed, we anticipate that by the time of the event, the analysis will be complete. This will allow us to share whether the suggested improvements were effective from the clients' perspective—and whether any unexpected insights emerged.
Conclusion
Our study highlighted a set of actionable steps to improve PT reports with the client experience in mind. Ultimately, when clients can clearly understand and effectively implement the recommended actions, security vulnerabilities are addressed more efficiently. And when that happens, we all move one step closer to a safer and more secure tomorrow.
Want to know more?
„From Reports to Actions: Bridging the Customer Usability Gap in Penetration Testing” K. Galanska, A. Kruzikova, M. P. Murumaa, V. Matyas, M. Just; IEEE Access, vol. 13, pp. 73975-73986, 15.04.2025, 10.1109/ACCESS.2025.3561220
