You should note two points above all others when reading this article. First, there is no "one size fits all" IDS solution on the market today, and I highly doubt there will be one anytime soon. The IDS product landscape is a diverse one. Products like ISS RealSecure are easy to install, have a wide range of features, but often fall over in high-bandwidth environments. Enterasys Dragon performs well and is liked by most UNIX-savvy individuals, but its user interface and the learning curve associated with the product will turn away most NT-focused administrators. BlackICE's raw power and simplicity might tempt some small organizations, but when you need to manage hundreds of thousands of events, Cisco's IDS with the Cisco Secure Policy Manager (CSPM) is a much more manageable solution. In short, organizations need to understand what their parameters are, and adopt a product that best serves those requirements.

Second, the product balances change almost yearly. For example, between 1999 and 2000, Cisco went from having one of the worst user interfaces (an HP OpenView hack) in the market, to one of the best (CSPM). In three years time the product known as ID-track from the company called Internet Tools was acquired by Axent, expanded upon, relabeled as NetProwler, and later acquired by Symantec when Axent and Symantec merged. NFR Security, Inc. (NFR) was way ahead of the IDS technology curve in 1998, and is considered by most to be somewhat behind in 2001. The bottom-line is this: Consider the comments in this text, the reviews that are published in magazines, and anything else that you might find on the Internet, but be conscious of the age of the information. The issues will stay somewhat constant, but who and how they are addressed could change in as short a time as six months. Although products like firewalls are fairly mature, and are now mostly differentiated by features, speed, and price, the IDS market is anything but mature. The only thing that you can be sure of on the IDS front is that nothing will remain the same.

Common Evaluation Criteria

When choosing an intrusion detection system, understand that you are choosing two things: a) a product and b) a partner (vendor) who will be updating that product. Although the vendor (or team, in the case of open-source solutions) behind the product is always a consideration, it becomes even more critical in the intrusion detection market. Because IDSs are so time- sensitive, so dependent on product updates, a good system will become increasingly less useful if it is not attended to properly and regularly. Evaluating the vendor's track record in regard to product updates is a worthy effort.

On the product side, there are a number of issues and features that can be found in one IDS, but not in another. However, many of the "bells and whistles" of these products are just that—cute features. Make sure that you evaluate the core components first, and then examine the bonus features. The following is a list of core components that you will want to evaluate when making IDS selection decisions:

· Depth of coverage. One of the more important components of an intrusion detection system is its ability to detect a wide array of attacks. Although a great back-end engine, diverse customization options, and a slick management interface are all strong selling points, if the product is incapable of detecting more than a handful of attacks, it will do little good. Make sure that any NIDS solution you examine is bundled with a healthy set of attack signatures. On the HIDS front, be sure that the product does more than inspect a few log files for a handful of events, and make sure that the product supports all the platforms that you need to monitor. If, for example, the HIDS agents only support Windows NT but you have both Solaris and Linux machines, you are going to come up short in regard to overall coverage.

· Accuracy of coverage. This is a hard factor to determine without thorough testing, but it should be noted that not all signatures have been created equal. False positives are a big problem with most NIDS solutions, and in large environments these misfires can jeopardize the overall effectiveness of the intrusion detection effort. Products designed with the reduction of false positives in mind will become more desirable in the coming years.

· Robust architecture. There are multiple components to an intrusion detection solution, and it is important that both the engines and the IDS framework itself have been designed with strength in mind. On the engine/agent side, products should be able to withstand both attacks and basic evasion techniques. Although evasion has traditionally been a problem that has plagued NIDS devices, and will most likely continue to trouble them for some time, insightful vendors have continued their attempts at addressing these issues. Less insightful vendors have chosen to ignore them, which not only reduces product effectiveness, but also reduces confidence amongst security professionals.

· Scalability. There are multiple components that affect IDSs on the "scaling" front, but the two biggest are in the areas of high-bandwidth monitoring and data management. The bandwidth issues apply to NIDS devices in that many products have problems monitoring high-bandwidth, high-session environments. On the management front, some products struggle with monitoring, storing, and presenting large volumes of alert data. For example, if you deploy a few dozen sensors (host- or network-based) on a high-traffic/high-alert network, they will be pumping a lot of data back to the centralized databases and/or consoles. Some back-end systems will crumble under such loads, or, worse, the volume of data will make it incredibly hard for the security officers to sort through the alerts. However, it should be noted that these issues are not relevant in all environments. For example, if you are looking to place a few ID devices to watch over a few T1 connections, you aren't likely to run into bandwidth and data storage issues.

· Management framework. Being able to detect attacks is crucial for an IDS, but equally important is the ability to clearly and efficiently present the data related to those attacks. If security officers are unable to easily access attack and alert data, the overall usefulness of IDS will be limited. When evaluating intrusion detection systems, be sure to use the management console in a live environment. Make sure you are comfortable with a system's management framework, and make sure it allows you to access the information you want easily. In short, the management framework that is used to control and monitor the devices is almost as important as the HIDS and NIDS devices themselves.

· Timely updates. Much like in the vulnerability assessment (VA) product field, as new attacks continue to surface the need for timely IDS product updates becomes critical. Operating an outdated IDS is analogous to operating an airport without radar. Although updates are a bigger issue in regard to NIDS products, the issue is still relevant to all IDS models.

· Customizability. Some intrusion detection products allow for a diverse range of customization, whereas others are fairly static and inflexible. For some organizations, cus tomization features will not be a big issue because they will be operating IDS solutions with out-of-the-box configurations. For others, customization is a must. However, when choosing an IDS vendor, it's wise to evaluate your needs now, as well as in the future. Although you might not require the ability to write a custom signature today, you might need that functionality in the future.

· Skill set requirements. Intrusion detection devices should be treated like any other component of enterprise IT—properly trained staff should be operating the solution. Unfortunately, the one thing both administrators and managers alike seem to cast aside are the issues surrounding IDS upkeep.