Forecasting is, as Daniel Menasce and Virgilio Alameda put it so well "...the art and science of predicting future events" (Menasce and Almeida, Capacity Planning for Web Services, pg. 451). When it comes to capacity planning, a certain amount of forecasting for Web-server demand and related resource consumption is imperative for organizations that want to stay ahead of the demand curve, and on top of performance matters.
Here's what makes things interesting in this regard:
- When analyzing past demand and performance -- inarguably the best predictors of the future because they show where we've come from and can therefore by extension tell us where they're going -- it's nearly inevitable that demand and resource consumption trends are subjected to "curve fitting." Although this is a reasonable predictor in most cases, this approach can do nothing to predict unexpected spikes or troughs when forecasting.
- When predicting the future, particularly for Web demand, economic conditions play a significant role. They don't call this science "voodoo economics" for no good reason. In fact, trying to create accurate economic forecasts shares many of the same mathematical problems you're likely to run into for Web forecasts. Either way, the tendency is the same: when projecting trends into the future, "take a little off the top" if the economy's slowing or on the way down; "boost the slope a little" if the economy's speeding up or rising. Alas how much to take off or put on is where the art of forecasting really comes into play.
- Inevitably, boosting capacity is not a smooth function, either. At some point or another, obvious steps upward in capacity or performance make themselves felt: the impact of adding another server or cluster, or the impact of switching from a monolithic site to a load-balanced collection of geographically dispersed mirrors are obvious points where capacity and performance jump up by sizable increments (in cost, too).
That's why forecasting includes as much art as it does science: you must not only guess about where the numbers take you, but how those numbers may misrepresent the real data and also how external influences can come into play. Spurious curve smoothing and crossing performance or capacity thresholds when growing systems are good examples of the former; the impact of overall economic conditions combined with potential spikes related to the introduction of new products, publicity, or "public discovery" are good examples of the latter. Any of these influences can force you to flatten or steepen whatever trend lines emerge from your site's usage and demand histories.
When forecasting, the most important trade-off lies between providing extra capacity that will be used and extra capacity that sits idle. Therefore, you must prepare to educate management about the real tradeoff that sits beneath the costs involved: namely, the tradeoff between paying for extra capacity that goes unused but allows all incoming demand to be satisfied versus the opportunity cost of losing business when potential (or current) customers can't get to your Web site. This suggests that the tradeoff is less poignant for organizations whose Web sites may be important communication tools but where significant business isn't conducted as compared to those where availability translates into potential revenue all too readily. In the latter case, buying extra capacity going forward should be treated as form of insurance against demand spikes. How much such insurance to buy is still quite black as arts go, but "enough" will be the only magic number that satisfies management.
Ed Tittel presides over LANWrights, Inc., a company that specializes in Web markup languages and tools, with a decidedly Microsoft flavor. He is also a co-author of more than 100 books on a variety of topics, and a regular contributor to numerous TechTarget web sites. Contact Ed via e-mail at firstname.lastname@example.org.
This was first published in August 2002