System and software exploits allow hackers to take advantage of weaknesses of particular OSs and applications (often called bugs). Like protocol exploits, they are used by intruders to gain unauthorized access to computers or networks, or to crash or clog up the systems to deny service to others. Common bugs can be categorized as follows:

- Buffer Overflows Many common security holes are based on buffer overflow problems. Buffer overflows occur when the number of bytes or characters input exceeds the maximum number allowed by the programmer writing the program.

- Unexpected Input Programmers may not take steps to define what happens if invalid input (input that doesn’t match program specifications) is entered. Such input could cause the program to crash or open up a way into the system.

- System Configuration Bugs These are not really bugs per se, but rather they are ways of configuring the OS or software that leaves it vulnerable to penetration. Popular software such as Microsoft’s Internet Information Server (IIS), Microsoft’s Internet Explorer (MSIE), Linux Apache Web Server, UNIX Sendmail, and Mac Quicktime, are popular targets of hackers looking for software security holes that can be exploited. Major OS and software vendors regularly release security patches to fix exploitable bugs. It is very important for network administrators to stay up-to-date in applying these fixes and/or service packs to ensure that their systems are as secure as possible.

Trojans,Viruses, and Worms, Oh My!

Intruders who access your systems without authorization or inside attackers with malicious motives, could plant various types of programs to cause damage to your network.There are three broad categories of malicious code:Trojans, viruses, and worms.

- Trojans The name, short for Trojan horse, refers to a software program that appears to perform a useful function, but in fact performs actions that the program user is not aware of or did not intend. Hackers often write Trojans to circumvent the security of a system. Once the Trojan is installed, the hacker can exploit the security holes it creates to gain unauthorized access, or the Trojan program can perform some action such as:

- Deleting or modifying files

- Transmitting files across the network to the intruder

- Installing other programs or viruses Basically, the Trojan can perform any action that the user has privileges and permissions to perform on the system.This means that a Trojan is especially dangerous if the unsuspecting user who installs it is an administrator and has access to the system files. Trojans can be cleverly disguised as innocuous programs, utilities, screensavers, or the like. A Trojan can also be installed by an executable script ( JavaScript, a Java applet, Active-X control, and so forth) on a Web site. Accessing the site can initiate the installation of the program, if the Web browser is configured to allow scripts to run automatically.

- Viruses Includes any programs that are usually installed without the user’s awareness and performs undesired actions. Viruses can also replicate themselves, infecting other systems by writing themselves to any disk used in the computer or sending themselves across the network. Viruses often distribute as attachments to e-mail or as macros in word processing documents. Some viruses activate immediately on installation; others lie dormant until a specific date or time, or when a particular system event triggers them. Viruses come in thousands of varieties.They can do anything from popping up a message that says “Hi!” to erasing a computer’s entire hard disk.The proliferation of computer viruses has also led to the phenomenon of the virus hoax, which is a warning (generally circulated via e-mail or Web sites) about a virus that does not exist or does not do what the warning claims it will do. Real viruses, however, present a real threat to your network. Companies such as Symantec and McAfee make anti-virus software that is aimed at detecting and removing virus programs. Because new viruses are created daily, it is important to download new virus definition files, which contain the information required to detect each virus type on a regular basis, to ensure that your virus protection stays up-to-date.The most dangerous virus is a new, fast replicating virus for which no definition has been created. Fortunately, anti-virus companies now respond within hours of a new outbreak. Since nearly all anti-virus software has auto-update features, the new definitions are usually quickly put in place and effectively shut down the proliferation.This does not mean you are immune from infection if you have anti-virus software, it just means you are generally safe from older viruses. Both viruses and Trojans may carry a logic bomb (i.e., a bit of malicious code designed to “explode” under certain circumstances such as performing, or failing to perform an action).The bomb can do anything from delete files to wipe a computer.The “fun” part of a logic bomb for a hacker is letting the victim believe nothing is wrong and then at a much later time damage the computer, making it more difficult to determine where and when the infection occurred.

- A worm is a program that can travel across the network from one computer to another. Sometimes different parts of a worm run on different computers.Worms make multiple copies of themselves and spread throughout a network.The distinction between viruses and worms has become blurred. Originally the term worm was used to describe code that attacked multi-user systems (networks) and virus was used to describe programs that replicated on individual computers. The primary purpose of a worm is to replicate.Worm programs were initially used for legitimate purposes in performing network management duties, but their ability to multiply quickly has been exploited by hackers, who create malicious worms that replicate wildly and might also exploit OS weaknesses and perform other harmful actions. Unfortunately, nearly all these now contain a root-kit.This is a series of tools that take control of your machine and create a zombie that will do the bidding of the malicious writer. Once a root-kit is installed on your machine, your only choice is to flatten the machine and rebuild from scratch. Root-kits notoriously have subprograms that hide their presence from the OS. While there are tools such as Root-kit Revealer by SysInternals (www.sysinternals.com), there is no sure way to confirm that all pieces of the root-kit have been removed.Any remaining bits have the potential to reinstall the entire root-kit and begin transmitting information to the owner.

Buffer Overflow

In general, most data packets can be manipulated in an attempt to create a buffer overflow, which is a specific condition in an application where more data is written to an area of memory than has been allocated.The extra data then flows into the next area of memory, where it should not be. If the application design doesn’t consider this possibility, it may be possible to leverage this situation to execute the code in the second memory area.This situation can yield many unwanted results including: application hang or crash, server hang or crash, or even worst, compromise of the machine where control is given to the sender of the packet. White hat hackers, people who attempt to find vulnerabilities in software and then report them to the manufacturer for correction, are constantly working to find buffer overflow errors in software and OSs. Because humans write all software and humans are prone to error, it is highly unlikely that there will ever be “perfect software” with no vulnerability.Therefore, it is in the best interest of network administrators to protect the most valuable assets with firewalls as best we can.

Do Firewalls Have Buffer Overflows?

Short answer, “Yes.” However, there are far fewer than most other software, because firewalls are stripped down to the bare essentials. Firewall software is also scrutinized more closely due to the task the firewall is attempting to perform. Most firewall vulnerabilities result in DoS’s rather than access violations or compromise. Firewalls are designed to fail closed so that the firewall cuts off network access rather than permitting unauthorized access. Also realize that some firewalls are designed to be installed on existing OSs. If the underlying OS has vulnerabilities, your firewall will only be as good as the OS its running on. In addition, a poorly configured firewall can leave gaping holes that a malicious person could walk through with ease. This article is a good start to configure your firewall securely; however, don’t stop here. Read the manufacturer’s documentation, white papers provided by the manufacture, and blogs, newsgroups, and discussion groups related to your model of firewall. Learn from other’s mistakes and don’t make them yourself. Most of these resources are freely available on the Web; a few searches should turn up starting points that will lead you to more resources. To a determined hacker, discovering a firewall is tantamount to throwing down a gauntlet and posing the challenge of how to exploit the permitted access. The good news here is that such determined hackers are fewer than the script kiddies (less experienced hackers who rely on pre-written scripts and tools to compromise machines) who look for easy targets with well-known vulnerabilities. Therefore, be certain that the script kiddies will walk away after knocking on the door and getting no answer. Then delve into the literature mentioned above and make your network unwelcoming to even determined hackers.