Start with Why
Annual penetration testing is a common component in most cybersecurity programs. In fact, some organizations do many per year or undergo specialized penetration tests called red team exercises. The most important question to ask before doing a penetration test is why the organization needs it and what you're trying to accomplish.
For most organizations, annual penetration testing is akin to the annual health checkup or wellness visit with a physician. When performed well, a penetration test gives the organization a high-confidence baseline of how their security program is faring.
There are many reasons for performing a network penetration test:
- Meet business requirements and build trust with customers, partners and insurance providers.
- Test the defenders by validating that detection and response tools are catching live attacks.
- Comply with regulatory and industry requirements that mandate penetration testing, such as PCI-DSS for credit card information, NY DFS 500 for financial institutions, or HIPAA for healthcare organizations.
- Protect sensitive data and validate the controls protecting it.
- Identify vulnerabilities and improve your security program with prescriptive guidance.
- Follow best practices from security control frameworks such as NIST CSF 2.0, CIS Controls and the ISO 27000 series.
- Peace of mind. Am I safe? Am I ready for an attack?
Can't I identify vulnerabilities with a vulnerability scanner or agent-based tool?
Yes! And performing regular vulnerability scans is considered a best practice. An organization should not wait for their annual penetration test to find vulnerabilities. A good penetration test (a theme of this document) should identify vulnerabilities that are missed by tools alone.
Getting Ready for a Scoping Call
Stand up! Do some stretches... I'm kidding (but really, wouldn't that be fun?)... What data should you prepare, and what questions should be answered, to get a good quote?
This will make up the bulk of the penetration test scope and influence cost in terms of testing time.
- How many public IPs make up all my sites, datacenters, and cloud infrastructure?
- How many users are there in my organization, and how many of them have email addresses?
- What are the wireless entry points into my network, and is this something that concerns me?
- How many internal IPs are associated with active devices? (DHCP pools, MAC address tables, and good inventory can help here.)
This question should get to the heart of why the penetration test is being performed. A good penetration testing team should be able to perform a holistic network penetration test (good for ransomware and hygiene checks) or more specific attacks against critical systems or items on your risk register which may critically impact the business (red team exercises). Examples:
- Ransomware incident
- Compromise of key systems
- Data breach with data exfiltration
- Insider threat
There may be systems which should be completely excluded from the scope. If we don't own it, it's unlikely we have the right to try to attack it.
- Do third parties have equipment on your network?
- Do you lease or share a building with other tenants?
All penetration testing involves a non-zero chance of system downtime. Penetration testing involves exploits and tools developed by hackers that don't typically come with much quality control or a support hotline to call. Some exploits, which are used by real attackers, can cause system instability by their very nature (a buffer overflow vulnerability, for example) and may cause systems to crash.
A good penetration tester will be able to minimize this risk. Inexperienced penetration testers are more likely to cause incidents if they haven't developed caution.
- Do I need after-hours testing?
- For applications, do I have separate dev and production copies?
- Should I exclude additional systems or networks which are just too risky?
You'll never be 100% certain you've found all the holes in an environment. But increasing the duration of a pentest can lead to higher confidence.
Questions for the Vendor
They'll need to ask questions to create a quote. But this is also your time to vet the vendor prior to a proposal. Ask questions that align to your business goal, but also look for red flags.
- Can I see a sample report?
- Can you walk me through your methodology?
- Are your penetration testers in-house?
- What country are the penetration testers in, and where will my data be stored?
- Do your penetration testers have certifications? Can I see a CV or a resume for someone who may be on my project?
- Has your team identified or published any CVEs?
- How do you protect my data during and after the engagement?
Red Flags
- Few questions from the vendor.
- Outsourcing. This isn't always a bad thing, but subcontractors may indicate that the vendor doesn't have the skillset in-house and can't provide strategic guidance.
- No sample report.
- Insufficient data protection.
- Regulatory issues with where your data is being kept.
- Unreasonably low prices. If the pentest is < $5,000, you're probably getting an automated test. You could buy those tools and run them year-round for shorter vulnerability dwell time and better outcomes.
How long did that take to get a quote? Was it slow?
The Statement of Work
What should you look for in the statement of work? And don't be afraid to ask the vendor for clarification.
Days of Testing
This may be the most important quality factor, but it should also be a strong basis for comparing costs (dollars per testing day — make sure to exclude reporting). More time testing amounts to higher confidence and fewer generic findings.
Deliverables
What do you get when it's all over? The report will be the basis of your remediation efforts. Will it have prescriptive remediation recommendations? Will the vendor provide an attestation letter?
Project Management
Is it expensive, and are you paying for it? Do you need it?
Pentester Bios
Take a look at the qualifications of the testing team. Does the vendor commit that these are the people who will do the work?
Automated Testing
Look for words like "scan" and "agent", and other evidence of a reliance on tooling rather than skill.
Rules of Engagement
A quality vendor should identify a more specific testing scope either pre-quote or as part of the project.
Retesting
Will the vendor retest and create another report after remediation?
Structured Data Outputs
JSON and CSV outputs of findings following an engagement can be helpful in tracking remediation work. Word documents and PDFs may be pretty, but if your team works in systems, structured data is more useful.
