Knowing that a lock might fail is useful, but it's not the same as kicking the door in and seeing how long it would take to steal your grandmother’s wedding ring.
Comparing a pen test to a vulnerability scan is a similar kind of “theory versus practice” question.
Vulnerability scanning shows you (some of) the software vulnerabilities that weaken your environment. Penetration testing (pen testing) pits an ethical hacker against your systems, demonstrating how a compromise would go down in the real world.
For security teams, this difference matters because:
- A vulnerability scan is like an audit of your “low-hanging fruit” vulnerabilities. It tells you some of the vulnerabilities you need to watch for but not which vulnerabilities put you at the greatest risk. Although a vulnerability scan comes with risk ratings, those ratings don't necessarily match with business risk.
- Penetration testing shows a business's real cyber risk. A penetration test report maps findings to business risk, showing the actual damage threat actors could do from a particular starting point.
You might read a vulnerability scan report and assume you have a certain level of risk based on the provided risk ratings. However, unless you do penetration testing on the same system, you won’t really know what your attack surface looks like.
This blog post goes deeper into why understanding the use cases for a pen test vs a vulnerability scan matters.
What Is a Vulnerability Scan?
A vulnerability scan is a comparison of the software in your environment to known Common Vulnerabilities and Exposures (CVEs).
At the end of a vulnerability scan, you will receive a list of known vulnerabilities the scanner has found in your environment.
Over a period ranging from a few minutes to a few hours, a scanner will look for known vulnerabilities in the environment it is designed for. The report you get when the scan finishes will tell you how many vulnerabilities of different severity levels the scanner found in the scanning target.
Vulnerability scanners are an essential security tool, but depending on your goals, their results might need further investigation from human testers.
Over 28,000 vulnerabilities were published in 2023 alone. Some, due to their exploitability, create extreme risk, but not all are dangerous.
Without manual verification, vulnerability scans:
- Can create false positives and false negatives.
- Give a surface-level view of your actual risk level.
Vulnerability scanners are still an essential security tool. They help security teams cover patch management by checking every detected software version for known vulnerabilities affecting that specific version.
Vulnerability scanners are also able to check for default/weak credentials in a variety of services and can identify some level of information leakage.
However, vulnerability scanners are only a tool, not a replacement for penetration testing.
Even the smartest vulnerability scanners cannot do “human” things like joining the dots between unrelated, often seemingly harmless, vulnerabilities to open up system-level compromise.
What Is Pen Testing (Compared to Vulnerability Testing)?
A pen test is a security exercise that simulates a real-world cyber attack on your environment to find weaknesses. The goal of a pentest is usually to identify all the vulnerabilities affecting the target systems and to exploit them in a risk-controlled manner to assess business impact.
Vulnerability scanning is a process within penetration testing. Pen tests often build on vulnerability scanning to see what vulnerabilities are present within an environment and assess the potential impact of exploiting them.
However, pen testers also scan environments with their own custom tools to identify potential vulnerabilities that vulnerability scanners won’t catch.
Additionally, penetration testers might have creative ideas that a vulnerability scanner cannot have. For example, “What if I try to send this type of request instead of the one the app expects?”
There are lots of vulnerabilities that a vulnerability scanner cannot identify (but a penetration test can), such as:
- Password reuse: Although vulnerability scanners can usually be configured to detect the use of weak credentials, they can’t check if a user has reused their credentials on other systems.
- Authentication issues: A vulnerability scanner will usually not detect a lack of strong authentication (for example, two-factor authentication).
- Business logic flaws: Even if provided with credentials, it is very difficult for vulnerability scanners to find issues related to business logic. For example, if you have a web app with a sequence of steps, a vulnerability scanner will not try to jump from the first step to the last step to attempt to bypass other steps. Even if it did, it probably wouldn't understand that it is a potential security issue.
- Post-exploitation issues: Since a vulnerability scanner does not exploit vulnerabilities, it wouldn't be able to find certain issues that require access to the target system. For example, after compromising a server, a pentest might reveal that passwords are stored in cleartext in a database.
Pen Testing vs. Vulnerability Scanning According to Offensive Security Experts
To help show you where a vulnerability scan ends and a pen test begins, we asked our experienced penetration testing team.
Thanos Polychronis, SECFORCE’s Head of Adversary Simulation, explains that vulnerability scanners can miss exploitable vulnerabilities that allow attackers to move laterally.
“Imagine you test a web application, and after content enumeration, you find a page with a low-risk vulnerability that was not found by a vulnerability scanner. After testing it, you realise you can run an exploit that allows you to pivot inside the network, and eventually, you have control of the entire enterprise domain.”
He highlights how this kind of attack requires a multi-step approach and is based on findings, discovery, and trial and error.
“This is an attack vector that a vulnerability scanner alone would not be able to discover.”
António Quina, SECFORCE’s Technical Director, uses the example of an attacker taking user credentials they find in one place and trying them in other applications.
He says that an attacker could:
“Get a user's credentials, which potentially a vulnerability scanner could do, but then use those credentials to try to authenticate somewhere else. If the user has used the same password in multiple places [which over 65% of people have], you can gain access and then from there try to jump into other systems.”
Vulnerability scanners can be invaluable in finding vulnerabilities, but they don't connect the dots like humans do.
Real attackers are unpredictable. They often get into their target’s networks through apparently less critical vulnerabilities and then pivot their attacks into critical parts of the environment.
For this reason, including medium/low-risk vulnerabilities in a pentest report is important for assessing real-world cyber risk.
On their own, low-risk vulnerabilities may not pose much risk to an organisation, but combining individual lower-risk issues can create high risk and result in an attacker successfully compromising a system.
Here’s an example of a high-risk outcome coming from two lower-risk issues:
- An attacker might be able to enumerate users' emails on a system, which is, on its own, a low/medium risk.
- The attacker can then send the leaked email addresses a link with a payload to exploit a reflected XSS vulnerability affecting one of the organisation’s web apps (medium risk).
- The XSS payload redirects the recipients to a malicious website from where they download malware.
One exploit leads to another and another, and so on. Vulnerability scanners might show you the first part of a possible chain but not the whole thing.
Pen Testing vs.Vulnerability Scanning Comparison Table
To help you see the different use cases of vulnerability scanning versus penetration testing, we created the table below.
Both vulnerability scanning and pen testing identify vulnerabilities.
The core difference is that pen testing also exploits those vulnerabilities to see which ones are dangerous to your organisation and need immediate attention.
- A vulnerability scanner will tell you, “The system is affected by a critical vulnerability.”
- A penetration test will show how pen testers can actually exploit the vulnerability to gain access to your organisation’s personally identifiable information.
However, there really isn't a choice between vulnerability scanning and penetration testing.
You should do both.
Vulnerability scanning is a fundamental cybersecurity task and is the only way to do patch management properly.
When to Do a Vulnerability Scan Versus a Pen Test
Do a vulnerability scan on a monthly basis, if not more. Use your existing team and a vulnerability scanning tool designed for whatever environment(s) you operate. You might also want to hire an external team to validate the scan’s findings, i.e., to check for false positives. The findings will help you optimise your patch management processes.
Do a pen test whenever you release a new solution or change your infrastructure (for example, a new virtual private network), release a new API, deploy a new firewall, and move infrastructure to another site. As a rule, changes to your environment merit pen testing
Are You Being Sold a Vulnerability Scan Instead of a Penetration Test?
“If it's much cheaper than average, it may not be a pentest at all.”
This is the golden rule that our offensive security experts think buyers need to keep in mind.
Some offensive security firms will say/imply they provide pen testing but actually just do vulnerability scanning.
This mix-up can put you at immense risk. As we’ve already explained above, a vulnerability scan is not a substitute for a penetration test. Thinking that a penetration test has taken place when you have just undergone a vulnerability scan can lead to insecure systems, compliance failure, and potential breaches.
If you are contracting a penetration testing firm, here’s what to look out for to make sure you don't get a vulnerability scan instead.
Do the following before a penetration test takes place when reviewing proposals:
- Ask for the vendor’s penetration testing methodology. A test might include a vulnerability scan but should go beyond this and involve certified offensive security professionals. Check the list of all the things a pen testing vendor says they’ll carry out to ensure it’s more than just scanning and reporting.
- Review a sample report. It should be easy for a vendor to give you a redacted report from a past pen testing engagement. Review the quality of the report’s findings, their detail, and the kinds of recommendations they provide.
- Clarify SOWs. The statement of work (SOW) you sign up for should specify the testing approach and reporting format.
Get a Best-In-Class Penetration Test with SECFORCE
SECFORCE offers comprehensive penetration testing, helping to find flaws where others might not.
We can also review your testing programme and provide advice and approach to maximise your results and enhance your security posture.
Contact us to have a chat.
