The Cost of Poor Cloud Performance

Although it is universally acknowledged that web site and application performance is important to the user experience, rarely do people realize -- let alone measure -- just how much of an impact sub-second latency can have on their business.

Consider this: as I was writing these lines, the GlobalProviderView service shows that the difference in response time between the fastest U.S. east coast cloud provider and the slowest provider was about 1.2 seconds. So the question is: does this latency matter? And how much does it matter to the business?

Way back in 2006 Marissa Mayer of Google told an audience at the Web 2.0 conference that when they conducted a user focus group to understand how they can improve the web search experience one of the things they were told is that users want to see more results, not just 10, come back from the search engine.

Following the rule that the customer is always right, Google immediately responded by conducting an experiment in which they increased the number of results from 10 to 30. To their dismay, traffic -- and associated revenues -- dropped by 20%. What was going on?

Well, it turned out that an unintended consequence of increasing the number of search results was also increasing the result page load time. Specifically, it rose from .4 to .9 seconds. That half a second delay could have an impact valued in the billions of dollars if it were applied to the entire population of Google users.

In 2009, both Google and Microsoft’s Bing presented at the O’Reilly Velocity conference how slow page load times have a huge impact on users -- and revenues. They even showed that even after the slowing down experiment was completed, and performance was returned to normal, those users who were exposed to the slowdown did not recover to their old behavior for weeks. Just the expectation of a 400 millisecond slower response time turned them off from conducting web searches!

Amazon also reported dramatic effects on its revenues and profitability when they conducted various A/B tests, even with 100 millisecond latencies were added -- delays that are barely noticeable by the human eye or brain.

While it is true that performance is not always on the top of the list of criteria to consider when selecting a cloud provider, these experiences suggest that when selecting a provider, one should also consider performance. And that while the cost of the provider may be what we immediately see in front of us when making the choice, there could be huge financial implications that are not immediately obvious.

To make things even more complicated, companies who sell to a global audience need to consider geographic locations of the cloud provider’s data centers as well. As again the CloudSleuth GPV shows, the difference in response times from data centers across continents can be as high as 25 seconds, clearly an unacceptable wait time for most modern web users, not to mention mobile users.

That said, not all applications are made equal. Many applications that run on the cloud are batch applications that perform massive ‘big data’ computations and are not sensitive to such latencies. In other cases, the difference when measuring your actual application may be indeed negligible and other aspects of the cloud provider -- such as ease-of-use or support -- might be more important. But all in all, you should carefully consider the difference in performance between cloud providers and its economic impact on your business.