The Lack of a Highly Available and Highly Scalable Operational Infrastructure

Many e-commerce enterprises fail to properly construct an inherently scalable, redundant, and reliable 24 x 7 x Forever Web enterprise system architecture. E-commerce enterprises must have speedy and highly available systems, including at least the Web server/call center frontend systems and the backend order-entry and fulfillment systems. If any of the required systems are not continuously available, nor able to gracefully handle unpredictable spikes in site activity, then the user sees this immediately and the site risks the extremely negative business impact of wandering eyeballs. Wandering eyeballs lead users to competing sites that are at most a few clicks away. These wayward forays by users can lead to a permanent loss of time spent at the site. This causes a reduction in lifetime user value that can range from a significant percentage for existing users, to a potential total loss of the lifetime user value of a sales prospect or visitor who may vow to never return again.

The Lack of a Clickstream/Callstream Data Warehouse

Many e-commerce enterprises fail to undertake the difficult design and implementation of a highly scalable clickstream/callstream data warehouse, which records the activities of all users of a particular Web site and its associated call centers. The knowledge derived from the analysis of the information in the clickstream/callstream data warehouse is the key to long-term competitive advantage of an e-commerce enterprise, making the implementation of an effective clickstream/callstream data warehouse an early priority in the life of an e-commerce enterprise.

The Alignment of Information Technology Vendor and E-Commerce Enterprise Business Goals

E-commerce enterprise employees have a natural tendency to try to become e-commerce knowledge and management superhumans, attempting to orchestrate a hodge-podge of information technology vendors, including hardware, software, and services, none of who have any real stake in the success of the overall solution. Often, a better approach is to carefully choose a single overall solution vendor, with contractually guaranteed responsibility over the multiple solution component suppliers. Such an arrangement orients the responsible vendor’s business interests and the success of the total e-commerce solution, with that of the e-commerce enterprise, creating natural, synergistic incentives for e-commerce enterprise success.

It is easy to lay blame and identify problems, but how does one construct a viable solution model that conquers these three fundamental e-commerce enterprise dilemmas? The next part of the article explores solutions to each of these important problems.

Problem One: A Highly Available and Highly Scalable E-Commerce Operational Infrastructure

Because e-commerce enterprises are information entities at their core, the problem of high availability becomes particularly acute. A highly available implementation of the information architecture of an e-commerce enterprise also needs to deliver high levels of performance, even in the face of failure, which places special performance design and implementation requirements on this highly available architecture.

Typical e-commerce enterprises take care of the first level of these requirements by sitting their Web servers at Web site-hosting service providers, such as Exodus, AboveNet, Frontier GlobalCenter, and others. These site-hosting service providers typically have multiple geographically separated, secure buildings that are sited on top of central-location Internet backbone connections for fast access. The e-commerce enterprise’s Web servers are placed at one or more of the Web-host service provider sites, creating fast, replicated Web server access for site users. In addition, the site-hosting service providers typically create a high availability environment for the hosted site servers, including such features as redundant Internet backbone connections, redundant uninterruptable power grids, nonwater-based fire suppression, and caged-system physical security measures.

Multiple replicated-content Web servers reside at these sites, providing a continuous Web presence for the e-commerce enterprise. Users enjoy fast and reliable access, and operational personnel can theoretically sleep at night.

But, not all of the critical e-commerce systems can be replicated as easily as the Web server frontends. Backend systems tend to be centralized by their very nature and this creates a new set of single-point-of-failure problems that go beyond the site and environmental redundancy provided by the site-hosting service provider.

For example, backend order entry, processing, and fulfillment systems are usually centralized. A customer who places an order from a particular Web server may later inquire about its status from another server, and all the order information needs to be stored in a centralized database so that this functionality can be delivered from any point of entry. But, the system that houses this centralized order database becomes a major single point of failure that requires a redundant architecture on top of that provided by the Web hosting service provider.

Any failure in the hardware or software of the centralized backend order system can stop an e-commerce site, and the types of “orders” can be anything from a purchase of physical goods, served-up advertisements, auction site bids, stock market buy and sell orders, to site content information. These centralized information systems must have a redundant, clustered, highly available implementation within a particular site-hosting environment, or they become a single point of failure.

In addition to a fully redundant site-hosting architecture, it is also necessary to replicate the host site systems between at least two geographically remote locations for protection against catastrophic host site disasters, such as widespread power failures, earthquakes, storms, terrorism, and war. The replicated, multisite strategy also enhances localized Internet site access performance, by shortening the pathway from the client to the nearest geographic server. In addition, the business model of many e-commerce enterprises causes the clickstream/callstream data warehouse to become so critical to operations that it, too, must be clustered and remotely replicated to ensure the viability of the enterprise.

Problem Two: The Design and Implementation of an Effective Clickstream/Callstream Data Warehouse

It is a cliché, but the Web changes everything about the design of a B2C commerce-oriented data warehouse. The wide scope of this change is best appreciated by reviewing the typical data warehouse schema of traditional brick-and-mortar retailers.

Although traditional brick-and-mortar retail data warehouses differ greatly in the details of their specific implementations, all have some version of four key dimensions: Time, Product, Geography, and Promotion, as well as a Sales Fact table containing sales transaction data. There is also no notion of a customer dimension in this old-style data warehouse. Until recently, it was so difficult to capture the identity of a specific customer and his associated market basket that this key analytical dimension, perhaps the most important of all the dimensions, was left out. Nevertheless, the information contained in this type of schema has changed the face of retailing, greatly improving inventory management, store layout, and mass-media advertising effectiveness.

By using newer IT customer identification techniques, such as loyalty cards or linkage to checking and credit card payment databases, forward-looking brick-and-mortar retailers have been able to add the Customer Dimension and associated market basket analytical capabilities to their data warehouses. This enhancement has driven the ongoing movement toward one-to-one customer relationship management (CRM).

Customer/market-basket analysis is a great advance, but there are two classes of potential customer activity that are not captured by loyalty cards or other brick-and-mortar information system tie-ins. Sales prospects are potential customers that do comparison shopping at different stores, catalogs, and assess various advertisements. They move in and out of a particular retailing environment without leaving a trace of their activity, unless, of course, they actually buy and become a customer. And casual visitors, just browsing your store, catalog, or advertisements are similarly anonymous. Web-based e-commerce is unique in that it can capture all the presales activity of prospecting potential customers as well as browsing visitors, greatly enhancing the enterprise’s overall market knowledge, and permitting much more sophisticated customer acquisition and retention strategies. As mentioned earlier, in an e-commerce environment, you lump visitors, prospects, targets, and customers into the general category of users. Let’s now start with the data warehouse schema of a forward-looking brick-and-mortar retailer and see how it changes in an electronic commerce environment.

Problem Three: Alignment of Information Technology Vendor and E-Commerce Enterprise Business Goals

E-commerce solutions have grown up in a modern, open systems technology environment. Although open systems solutions have many advantages, they also can create fundamental business goal alignment problems between the information technology product vendors and e-commerce enterprise that can thwart effective e-commerce information technology solutions. As mentioned earlier, e-commerce enterprise employees have a natural, job performance-driven tendency to try to become e-commerce knowledge and management Supermen or Superwomen, attempting to orchestrate a hodge-podge of information technology vendors, including hardware, software, and services, none of who have any real stake in the success of e-commerce enterprise.

Although a low cost, best-of-breed e-commerce solution, integrated by in-house personnel may seem attractive, there are several pitfalls. Unfortunately, the individual suppliers of information technology have no real stake in the success of the total e-commerce solution. They care only about their piece of it, and their economic incentive is focused primarily on the initial sale, not on the long-term success of the e-commerce enterprise. The myriad of best-of-breed, point-product solution vendors creates the need for deep in-house integration expertise that is vulnerable to employee knowledge, mobility, and reorganization issues. Furthermore, long-term integration costs are borne solely by the e-commerce organization, and this can be an increasingly burdensome issue as time progresses, and the complexity of integration issues grow.

A natural reaction to these problems is to outsource at least a portion of the solution to a solution integrator. Although outsourcing creates an attractive short-term management solution to the problems of an IT-lead, best-of-breed e-commerce solution integration, the motivations of the outsourcer and the e-commerce enterprise are still not properly aligned. These problems surface in a number of subtle ways.

In an outsourced solution, the integration vendor is the primary provider of the solution, and they are likely the sole source of the intellectual capital for the solution. Because the contracting e-commerce enterprise is relieved of much of the responsibility for the creation of internal solution expertise, the resulting outsourced solution is limited by the knowledge, business relationships, and integration expertise of the integration vendor. These limitations can lead to long-term solution issues that cannot be easily solved by the contracting enterprise, because the expertise required to do so is nonexistent by definition—it was outsourced.

This is bad enough, but the motivations of the integration vendor are rarely aligned with e-commerce enterprise business success. Most integration contracts are based on internally focused time and materials pricing, which has nothing to do with e-commerce business success. Furthermore, there is an implicit “forever” term to outsourcing contracts, meaning that the e-commerce enterprise will pay for this external solution expertise forever, because they are abdicating the development of the same in-house solution capabilities. When coupled with the time and materials nature of integrator pricing, this can lead to an incentive for the outsourcer to never solve any fundamental business problems, because if they did, it would only reduce their revenue.

An additional worry is the practical inability to write umbrella outsourcing contracts that address all the potential information technology modifications required to support unanticipated changes in business conditions. In an Internet environment, the contract-induced inability to rapidly adjust to changing business conditions can be fatal to the enterprise.

Finally, these insidious issues associated with totally outsourced solutions lead to a middle ground. Although this situation is not without problems, it better aligns business goals of the e-commerce enterprise and integration solution vendor. In order to solve the implied “forever” solution term with the integration vendors, the outsourcing contract must be fixed-term and nonrenewable, and it must mandate sign-off criteria and specify knowledge transfer to responsible individuals in the e-commerce information technology organization. Although this type of arrangement can still lead to higher initial solution costs and a solution limited by the knowledge of the integrator, the business drivers behind the motivations of both parties are much more aligned to the success of the e-commerce enterprise. The integration vendor takes time-limited and performance-goal-specific responsibility for a successful solution, and the resultant knowledge transfer causes the contracting e-commerce information technology organization to learn the skills required for long-term stewardship of the solution.