Wimpy vs. Brawny Cores

How do you know when your core is strong enough? Urs Hölzle (Google) puts a stake in the ground that cores weaker by more than a factor of two from current high end commodity cores. Going any slower, Hölzle says leads to challenges in response time and utilization. I expect that the analysis is relevant mostly to Google's workloads which, if anything, are more parallelizable than common enterprise workloads such as transaction processing or reporting.

So I agree with the general admonition against taking the throughput-oriented design too far, but not necessarily because of the general constraints around Amdahl's Law or load balancing cited by the paper. I think the key issue is that most software continues to be built, tested and tuned on fast processors with big instruction and data caches and a healthy amount of DRAM. Making this software work well on slower processors, with smaller effective cache and/or with smaller memory is like trying to squeeze the toothpaste back into the tube.

So the best systems will maintain high single thread performance. This is not likely to be a sustainable design in the long term but it's what we should expect for many years to come. Because these fast cores require more chip area and draw more power we should also, therefore expect the growth in the number of cores per chip to slow down. This doesn't mean that demand for parallel software has slackened since we will, eventually, need to contend with highly parallel systems, when we finally lose the ability to maintain high core performance.