There are two distinct types of testing that can be performed: announced and unannounced. The distinction comes when you define what is being tested: network security devices or network security staff.

Definitions

The following definitions help clarify the differences between the two types of testing.

• Announced testing is an attempt to access and retrieve preidentified flag file(s) or to compromise systems on the client network with the full cooperation and knowledge of the IT staff. Such testing examines the existing security infrastructure and individual systems for possible vulnerabilities. Creating a team-oriented environment in which members of the organization's security staff are part of the penetration team allows for a targeted attack against the most worthwhile hosts.

• Unannounced testing is an attempt to access and retrieve preidentified flag file(s) or to compromise systems on the client network with the awareness of only the upper levels of management. Such testing examines both the existing security infrastructure and the responsiveness of the staff. If intrusion detection and incident response plans have been created, this type of test will identify any weaknesses in their execution. Unannounced testing offers a test of the organization's security procedures in addition to the security of the infrastructure.

In both cases, the IT representative in the organization who would normally report security breaches to legal authorities should be aware of the test to prevent escalation to law enforcement organizations.

Also, management may place certain restrictions on the penetration test itself, such as the need to perform a portion of the test (for example, war dialing) after hours, to avoid certain critical servers on the network, to use only a certain subset of tools or exploits (for example, to omit denial-of-service tools), and so on. Such guidelines that come from upper management apply regardless of the type of engagement. At the conclusion of the engagement, system administrators should be able to review logs to identify the penetration test and to help them identify attacks in the future.

Pros and Cons of Both Types of Penetration Testing

Everything has its advantages and disadvantages. In this section, we discuss the pros and cons of each type of penetration testing.

Pros  Announced testing is an efficient way to check on and tweak the security controls the organization has in place. It creates a team-oriented approach to security and allows the organization's staff to experience firsthand what their network looks like to a possible intruder. Additionally, working with the IT staff allows the tester to concentrate efforts on the most critical systems.

Unannounced testing requires a more subtle approach. The tester tries to identify targets and compromise the security while staying under the radar screen of the target organization. This test may prove more valuable to the organization due to the range of items tested beyond the technology.

Cons  With announced testing, as large holes are identified on the client network, system administrators will close them quickly to avoid compromise. This can make further penetration difficult by not allowing further compromise of the vulnerability. Additionally, an announced test allows security staff time to make temporary changes to the network that add additional security. This gives management a false sense of security. The network may be secure during testing, but as soon as testing is complete and the original settings are restored, any original vulnerabilities will return as well, unbeknownst to the organization.

The risk with unannounced testing is that since the security administrators do not know that a test is being performed, they will respond as they would to a hacker and block the penetration testing efforts (drop connections, reboot machines, and so on). This would indicate a good response/detection process is in place, but it can cut a test short. The danger with this test is that occasionally security administrators have been known to contact the relevant authorities to report the penetration activities. To control this risk, the organization should have an escalation process in place with a specific individual being responsible for contacting authorities. This person should be aware the test is taking place.

Another risk during unannounced testing is that administrators may be making modifications to the environment during the testing period, which could skew the results. If the network administrator is upgrading a system, implementing a new service, or taking certain systems offline during the test, the results may not be as useful as they otherwise would. Additionally, the tester should be aware of quarterly or semi-quarterly events (such as large transfers of information from accounting) and backup schedules to avoid interfering with these operations.

Documented Compromise

At times during penetration testing, the client may be uncomfortable with allowing the tester to perform the actions that actually lead to a compromise. For example, it may be possible to access the router for network A and alter its routing table to appear as if the (attacking) network is a trusted, internal network and then route traffic from that network through the router to another trusted, internal network, network B. Then this compromised router would be able to connect the tester and the target network (B), bypassing security measures through its trust relationship with a less secure network (A).

However, the client may not want this activity to be performed. Altering the routing table may lead to additional complications for the client's network. The client may be satisfied that you can demonstrate that it can be done and describe how to fix the situation. Screen shots of documented system access may work well for this purpose. In such cases, document the possible hack along with its risk level and available countermeasures.