#6: Choose Mature Software

Let’s say that you have a choice between two relational database management systems (RDBMSs); for our purposes, we’ll say the choices are the current release of Oracle and Joe’s Database v1.0, from Joe’s Database and Storm Door Company of Ypsilanti, Michigan. (We are not endorsing Oracle; the same rules would apply to any mature software product. As far as we know, Joe has not yet released a database.) Joe’s product has a couple of features that make it a little easier to use, and it comes in a lot cheaper than Oracle does. You like Oracle, but Joe’s sales rep got you courtside seats for the NCAA finals last year. No offense to Joe or his sales rep, but in the end, you will almost certainly be better served by going with the product from a large company like Oracle (or Sybase, IBM, or Microsoft). The large-company products have benefits that you simply cannot get from a small-company product like Joe’s. Established RDBMSs have 24 × 7 telephone support, fly-to-site support, informative web sites, user groups, user conferences, magazines, and other mature services. A small company like Joe’s is unlikely to have any of those things. Established support structures make it easier to get help from many different sources. You can find users with whom you can discuss various implementation options, performance optimization, problems you may have encountered, and all sorts of other interesting issues, real and imagined. You can also more easily find reference sites that are willing to show you what they have accomplished with their product. Mature software has been tested by its customers (and internal testers) for a long time, and problems that newer software, written by less experienced developers, may run into have probably been taken care of. In some cases, more mature software will have availability features written right into it. Oracle’s RAC features are one example of such a feature. However, like so many other good rules, this one does not apply universally. Just because a product has achieved wide acceptance, that does not ensure that it is high quality. All wide acceptance ensures is that a lot of people use it. Wide acceptance is an excellent first qualifier when examining your choices in a given marketplace, but it should never be the only factor that gets consideration. When designing for availability, sometimes you’ll have to sacrifice the latest-and-greatest technology for something established because you know you’ll have to fix it when it breaks, and because cutting-edge products are not generally as reliable as the tried-and-true.

There’s an intermediate ground, however, in the area of publicly available software. Software that has large user bases and many developers working on fixes and improvements is likely to be easily obtained, fixed, and managed, because you’ll find a reference model for its use in your application. Windows utilities, Unix tools, cross-platform languages like Perl, browsers, and even web authoring tools can be obtained by looking and evaluating carefully. This is one of the advantages that the Open Source movement provides: many eyes and hands looking at the code. You should always have reservations about dot-zero, or first major revisions of software. Not that you shouldn’t put these releases into your test bed or laboratory environments and evaluate the new features and their integration with your world. But hesitate to put them in production until you’ve done the shakedown. When dealing with Internet software and startup companies, sometimes you need the latest killer application. The enabling technology may be brand-new to the market, forcing you to deal with version 1.0 products. In that case, refer back to Key Design Principle #10 repeatedly. Of course, it’s not always easy to tell when software is brand-new. Vendors have figured out that their customers are often reluctant to use version 1.0 and so give their software all kinds of other version numbers. Some vendors hide massive rewrites of old products by giving them the next release number in sequence. Other vendors don’t use release numbers at all, preferring instead to identify versions by the year or by some other name, number, or abbreviation. And still others synchronize the releases of many of their products and give all products released together the same version numbers, regardless of their relative maturity. With a little research, though, you should be able to determine how mature different releases of software are.

#5: Choose Mature, Reliable Hardware

Statistical information about mean time between failures (MTBFs) for particular hardware units can be difficult to obtain, and the data is often considered proprietary by its vendor. What’s more, the accuracy of MTBF numbers have often been called into question; how can a vendor reliably claim that a disk drive that was developed six months previously has an MTBF of 200,000 hours (which is more than 23 years)? Nevertheless, when purchasing new hardware, make an effort to secure and compare MTBF data. The data will help you get an idea of how often the various pieces of hardware will fail. Obviously, using more reliable hardware components will result in more reliable systems.

Building for easy repair is more a matter of selecting hardware that is easy to fix and easy to access. It is much easier to swap out a client computer if its disks are located outside the box, rather than inside. If they are outside, you need only swap out the computer and plug everything back in to get the client’s user back in business. If the disks are inside the client computer, the old computer must be opened and the disks removed. Then the new client computer is opened, and its disks are removed and replaced by the older disks. All the while, the user is unable to work at his desk.

Another way to reduce MTTRs is to manage your spare parts properly. Know what spares you are likely to need, and determine how long it will take your vendor to supply them. Also consider what will happen if the vendor has certain key parts on back order. In the days that followed September 11, 2001, all air traffic in the United States was shut down. People who were waiting for shipments had to wait several days longer. Can your business handle that eventuality? Does it make sense for you to inventory certain key spare parts? Balance the extra cost of the parts and the space and inventory overhead with the convenience and the potential reduction in MTTR.

#4: Reuse Configurations

If you have success with a particular system configuration, reuse it everywhere! Replicate it throughout your enterprise. Pick a handful of configurations for different situations, and just use those. It will be necessary to revisit the specific technology choices over time, as technology evolves, but by limiting the number of different models, fewer elements will need to be revisited, and overall, fewer configurations will need to be running in your enterprise. The following are some of the advantages of using replicated configurations: Ease of support. Fewer configurations mean fewer permutations of hardware and software, fewer things to learn, and fewer things that can go wrong. Pretested configurations. If the same exact configuration works in 20 other places, how much additional testing is required before rolling it out the 21st time? And if some elements actually have changed, only those changed elements need significant testing. High degree of confidence for new rollouts. If you know that a particular configuration has worked in other rollouts, it makes it much easier to justify using it for future rollouts. Bulk purchasing. It is often easier (and cheaper) to purchase large quantities of a single component than to purchase smaller quantities of many different components. This situation is especially true when multiple vendors are involved. Fewer spare parts. Fewer different components mean fewer spare parts to stock (if you choose to stock them at all). This lowers the overall cost of the spare parts, decreases the storage space required to stock them, and simplifies the inventorying process. Less to learn. Perhaps the biggest benefit of replicating configurations is in how much less there is for incoming system administrators to learn. Giving your SAs less to learn means that they can come up to speed on the way your company does things much faster, and as a result, they can be productive much sooner.

The advantages of replicating configurations can be achieved in other ways as well. Many shops use a centralized OS server; if a new system needs to join the network, or if something happens to an old one, the OS can be reinstalled on it quickly and easily, and in a way that is just like all of its neighbors on the network. This feature, often called Jump Start, Kick Start, or Remote Installation Services, is available from most operating system vendors. Embedding more and more software, such as volume managers, databases, popular GUIs, and browsers into the Jump Start means that the post–Jump Start work is greatly reduced, and systems can rejoin the network that much more quickly.