I wanted to share some musings around performance and Content Delivery Networks.
As most (all?) readers will know, CDN acceleration is one of the more established and widely used of the many performance-centric interventions now available. A recent study by Catchpoint systems  on the top 50 (US) retail sites identified 64% of third party services being represented by one major independent CDN provider alone.
The same study pointed out that a very high proportion (some 50%) of these major retailers do not optimise their CDN utilisation through domain sharding.
This in some way leads me to my key point: CDN adoption is not a (relatively) simple matter of setting up a few test points and quoting availability and latency. It could be better than nothing, and any efforts in this area – such as the comparison portal under development by Juho Vepsäläinen and his team elsewhere should be applauded – the need certainly exists.
CDN – not “one size fits all”
CDN adoption is far from a ‘one size fits all’ decision. The market itself is both large and volatile, with the contending capabilities and claims of Independents (Akamai, CD Networks, Limelight and many others), platform based offerings (e.g. Amazon Cloudfront) and ISP based services – often themselves delivered through licencing agreements with independents, e.g. Deutsche Telecom and EdgeCast.
The broad choice is of less relevance than the inherently dynamic nature of the problem. At its most basic level, CDN usage is far from binary – a given site may well employ several CDNs either directly or indirectly (through third party content). So what are some of the core questions?
- What are you accelerating? Depending upon the nature of the site and its usage, all or part may be accelerated. Is CDN caching applied just too large static images (for example), or all content?
- Where is it going? –what are the key global markets? – CDN providers differ widely in their capacity across the globe (indeed, some are regional niche players). Which primary regional ISPs are involved in web delivery?
- What is the nature of your core content and to whom is it principally delivered?
- Web HTTP and/or Video (total download, adaptive streaming etc.), PC browser, Mobile device, other (gaming console?)
Specific considerations apply in optimising each of these areas. These may or may not be embraced by a particular product or service. As an example, delivery optimisation to mobile devices may involve test compression (to reduce overall payload), video pacing and TCP acceleration.
All of these factors should influence test design and tooling approaches.
The right strategy
In my experience, an effective CDN based strategy begins with initial selection, informed by some of the considerations above. As we have seen, determining optimal return on investment repays a thorough approach, and some form of comparative Proof of Concept trial is likely to pay major dividends. Following the purchase decision and deployment, key on-going questions remain, if best value is to be maintained.
Looking at some of the test fundamentals:
Where are the priorities for usage optimisation?
- Is there an end user monitoring strategy in place across all key (and potential) markets?
- Do target KPIs (absolute or relative to local competitors) exist for all markets and usage categories?
- Is a service level agreement – with defined interventions / remedies in place with the selected provider?
- How do these stack up – both during ‘business as usual’ and across peak demand events?
These questions should be answered by an on-going monitoring programme, supported by a best practice based test matrix. Having defined which areas (functional, geographic and demographic) would potentially benefit from the use of CDN approaches, and assuming other interventions (e.g. appliance based, FEO) have been discounted:
Is the CDN accelerating all appropriate components?
- Some ‘entry level’ providers are restricted to caching static content such as images. If other elements are required e.g. login and search, are these also included and effectively measured?
Test design. Where does caching occur? E.g. single POP /ISP domain, via multiple hosts, at cellular network mast?
- Testing should reflect any potential constraints implied by the design of a particular CDN provider’s network.
- The most effective way to do this is by pre-scheduled, synthetic testing from appropriate end users (PC and/or mobile) in the country concerned.
- A parallel testing methodology should be employed, comparing performance directly to the origin servers with that delivered locally by the CDN provider.
This approach will ensure, not only that end user performance is satisfactory, but that the extent of benefit delivered (and purchased from) the CDN is maintained. In the interests of space, one example may suffice:
Comparative monitoring of Cached content to PC end users in a key regional market (known connectivity conditions). Origin vs. local cache
Importance of test locationThe relevance is that failure or poor performance by any of the 80 hosts not detected by ISP-level monitoring would not be detected, whilst being paid for, and impacting all end users of that particular host.
Hopefully this has shed some light on the issue as I see it, but it would be great to hear what your thoughts are.