Joshua Anthony
Cybersecurity is one of the few purchases where the buyer usually cannot evaluate what they received. If a construction firm builds you a weak bridge, it falls down. If a security firm sells you a weak assessment, everything looks fine right up until the day it very much does not. That information gap is exactly what the worst vendors in this industry exploit, and it is why choosing a security partner deserves more rigour than most procurement processes give it.
We have spent over a decade on the delivery side of security engagements across banking, finance, education, telecommunications, aviation and the public sector, and we have also been called in to clean up after other vendors. Here are the questions we would ask if we were the ones buying.
Is it actually a penetration test, or a scanner with a logo on it?
A meaningful share of what gets sold as penetration testing is an automated vulnerability scan exported to PDF with the vendor’s branding on the cover. The difference matters enormously. A scanner finds known signatures. A skilled tester chains findings together, abuses business logic, and demonstrates what an actual attacker could reach from the outside, which is what you are paying to learn.
Ask to see a redacted sample report before signing anything. A real test report walks through attack paths, shows evidence of exploitation, and explains impact in terms of your business. A scanner dump is a ranked list of CVEs with generic remediation text. Ask who will perform the testing and what their individual certifications and backgrounds are, not just the firm’s. And ask about methodology: a partner who can explain how they scope, what rules of engagement they propose, and how they handle findings that turn out to be critical mid-test has done this before.
What happens after the report lands?
This is where most security engagements quietly lose their value. The assessment ships, the invoice is paid, and the findings sit in a shared drive while everyone returns to their day jobs. Six months later, the same vulnerabilities are still open, except now there is a document proving the organisation knew about them.
Ask whether the partner supports remediation, and in what form: hands-on fixes, guidance for your internal team, or prioritised roadmaps that account for your actual budget and staffing. Then ask about retesting. Fixes fail more often than people expect, and a partner who verifies remediation and issues an updated report is closing the loop that makes the whole exercise worthwhile. If the engagement model ends at report handover, you are buying a list of problems, and lists of problems do not stop attackers.
Can they respond when something is actually on fire?
An assessment relationship and an incident are very different tests of a partner. When you are dealing with active ransomware or a compromised payment system, the questions become brutally practical: how fast can capable people be engaged, do they already understand your environment, and can they be physically present if the situation demands it?
This is where local presence stops being a nice-to-have. A partner with responders in your region can have someone on site the same day, already familiar with the infrastructure realities and regulatory environment you operate in, including data protection obligations and any sector rules from central banks or communications regulators about breach notification. A partner nine time zones away is triaging your crisis over a video call while the clock runs. Ask about incident response retainers, guaranteed response times by severity, and whether the responders are the same calibre of people who did your assessment or a separate desk reading a runbook.
Do their recommendations survive scrutiny?
A large portion of the security services market is, functionally, a reselling channel. Advice arrives pre-shaped by whichever vendor’s licences carry the best margin, which is how organisations end up with expensive tooling that overlaps what they already own and ignores their actual gaps.
The test is straightforward: ask the partner to argue against their own recommendation. A team that genuinely evaluated the options can tell you where their preferred product is weak and what would change their answer. A team reading from a partner portal cannot. The strongest security partners start from your environment, threat model and internal skills, then fit controls and tooling to that, and are comfortable recommending things they do not sell, including the answer that sometimes matters most: that you do not need to buy anything, you need to configure what you have.
Have they met the bar they are asking you to meet?
A security firm advising you on governance and controls should submit to the same discipline itself. Certifications such as ISO 27001 mean the partner’s own operations face independent audit, its processes are documented, and it handles your data, credentials and findings under a managed security programme rather than good intentions. This is not decoration. During an engagement, your security partner holds some of the most sensitive information about your organisation that exists anywhere: a written map of exactly how to break in. Ask how that material is stored, who can access it, and how long it is retained.
A partner asking clients to meet a security bar it has not met itself is telling you, quite openly, how seriously to take its advice.
The pattern behind the questions
None of these questions can be answered convincingly by a brochure, and that is the point. They probe the gap between what a vendor presents and how it operates under pressure, because that gap is where security engagements fail. The partners worth keeping are the ones whose answers get more specific the harder you push.
If you are evaluating security partners and want to see how we answer these questions ourselves, get in touch. We are happy to be pushed on all of them.
