Have you ever seen a carpenter working a piece of wood? They understand the strength, friction of the wood, the required tools for the job, the best approach to get to the desired outcome. They’ve done it all their lives.
We feel similar about penetration testing. For us it’s second nature, and we would like to share some of our thoughts about our craft.
What is a Penetration test?
First and foremost. What is penetration testing?
Penetration testing is the most widely used method of providing assurance about the security controls in place, protecting a system or application.
The assurance is gained by simulating a scenario whereby a malicious – but not destructive – attacker targets such a system or application in a systematic and repeatable way.
The aim of these tests is to identify the existing vulnerabilities in the security controls protecting the target, so that they can be addressed and decrease the associated risk to acceptable levels.
In summary: it is a controlled simulation of a threat actor conducting a cyber attack on a target.
Threat Actor, you say...
A threat actor is an attacker, and they can be very different indeed. They can be an opportunistic attacker with no prior knowledge connected on the Internet, a disgruntled developer of an application with complete knowledge of the target, or anything in between.
This is particularly relevant for the topic that we are discussing on this post, as the level of prior knowledge of the target determines whether the approach of the penetration test is considered black box (no prior knowledge), white box (full knowledge) or gray box (partial knowledge).
“Which one is best?”, you may ask. Well, as many things in security, “It depends”. Each approach has some pros and cons. Additionally, weirdly, there are even cultural and geographical implications. For example, in the US there is a higher tendency to choose a white box approach than in the rest of the world.
Let’s discuss each of them in detail.
White Box Penetration Testing
White box testing is conducted with full knowledge and access to the inner workings of the target systems. Testers are provided with detailed information about the network architecture, application source code, system configurations, and other relevant information.
The idea is that the more information is provided to the testers, the higher the coverage of the assessment will be, and therefore it will result in a more valuable outcome.
Advantages
- Deeper assessment: White box testing provides penetration testers with full access to the internal workings of the target systems. This allows them to perform a thorough and in-depth analysis of the target. As a result, they can identify deep-rooted vulnerabilities that other testing approaches may miss.
- More accurate results: With complete knowledge of the target systems, white box testing delivers highly accurate results, where there’s no need for making assumptions. In most cases the testers are able to exactly pinpoint the exact line of code (if it is an application test) where the vulnerability exists.
- Higher efficiency: White box testing can be more time efficient compared to Black box testing because it eliminates the need for extensive reconnaissance and enumeration activities. Additionally, the time required for successful exploitation of vulnerabilities (such as SQL injection, command injection, etc.) is significantly reduced when the source code of the application is available.
Disadvantages
- Limited real-world simulation: White box testing lacks the realism of real-world attack scenarios since attackers typically do not have as much extensive knowledge of the target systems. Therefore, the findings of the engagement could potentially be considered unrealistic.
- Higher levels of trust: Even with the existence of NDAs (Non-Disclosure Agreements) in place, providing critical application source code or highly sensitive information requires a high degree of trust on a third party. This is a risk that some organisations cannot take.
White box assessments are typically used in situations where critical systems have been previously subjected to black box penetration tests, and a higher level of assurance is required. By providing further information to the testing team, a more in-depth analysis is expected, with findings that would otherwise remain uncovered.
Additionally, White box testing could potentially be used when there’s some budget constrain and the further information allows the testing team to decrease the required effort by – for example – combining traditional testing with source code review tools.
Black Box Penetration Testing
Black box testing is conducted with no prior knowledge of the target systems. Testers approach the assessment as remote users having no information about the network architecture, internal code, or configurations. For example, in the case of a web application the attacker would simply have the target URL.
Advantages
- Real-World Simulation: Black box testing closely simulates real-world attack scenarios since the testers have no prior knowledge of the target systems. This approach replicates the perspective of an external attacker, providing a more accurate assessment of the organization's security against potential threats.
Disadvantages
- Time-Consuming: Black box testing starts with no prior information, so testers need to spend time exploring the infrastructure or application in a systematic way, without being able to reference the tool output, or behaviour of the application with anything else. Therefore, there is a process of inference where there is a need for exhaustive and resource intensive testing.
- Potentially lower testing coverage: The lack of internal knowledge can result in testers missing certain vulnerabilities that might be discovered in white box testing. During a black-box assessment the testers have limited visibility of the inner working of the systems or application and therefore in some cases is simply impossible to identify issues which may be present.
Black box testing is the most widely approach for penetration testing, due to being a simulation of a realistic scenario. In most cases it is suitable for simulating external threats, such as assessing an organisation's public-facing systems like websites or external applications.
Grey Box Penetration Testing
Grey box testing falls in between white box and black box testing. Testers have limited knowledge of the target systems, usually with some basic information but not full access to source code or exhaustive network details. In practical terms, the information provided depends on the type of engagement, but it is normally linked with the information which could potentially be gained by the attacker at some point, therefore providing a realistic shortcut.
Advantages
- Balanced Realism and In-Depth Analysis: Grey box testing strikes a balance between the realism of black box testing and the in-depth analysis of white box testing. Testers have limited knowledge of the target systems, simulating how a semi-informed attacker might approach the organisation's security.
- Efficient Use of Resources: Grey box testing can be more time and cost-efficient compared to white box testing. Testers don't have to spend as much time on extensive reconnaissance as in black box testing, allowing them to focus on evaluating vulnerabilities more quickly.
Disadvantages
- Limited Realism: While grey box testing provides a realistic perspective to some extent, it still falls short of a completely uninformed attacker's approach. Some attackers may have even less information, making certain scenarios less realistic.
Grey box testing is commonly used when the organisation wants to evaluate the effectiveness of their security measures and they want to maximise their testing resources whilst keeping the assessment as realistic as possible.
Conclusion
Penetration testing assessments can simulate different threat actors with various levels of knowledge about the target system or application.
The level of knowledge greatly influences the outcome of the engagement when it comes to the depth of the assessment and the realism of the simulation.
There’s no right or wrong when it comes to choosing the require approach, as it should be determined but the organisation's drivers and requirements for testing.
