Advanced Computing in the Age of AI | Thursday, March 28, 2024

The Carbon Footprint of Hyperscale 

<p>Microsoft CEO Steve Ballmer recently stated that the company operates over one million servers. What would it take to power an infrastructure of that magnitude? </p>

You've probably seen the headlines proclaiming Microsoft's one million-server operation. The impetus for the stories was a remark made by Microsoft CEO Steve Ballmer during the Microsoft World-Wide Partners Conference, held earlier this month in Houston.

"I claim there really are almost no companies in the world, just a handful, that are really investing in scaled public cloud infrastructure," Ballmer said in his keynote address. "We have something over a million servers in our datacenter infrastructure. Google is bigger than we are. Amazon is a little bit smaller. You get Yahoo and Facebook, and then everybody else is 100,000 units probably or less. So the number of companies that really understand the network topology, the datacenter construction, the server requirements to build this public cloud infrastructure is very, very small."

The most surprising part of this comment is that it exists at all. Of the handful of warehouse-scale datacenter companies, Microsoft is the first to confirm its server count. As for the other data points, i.e., comparisons to Google, Amazon and the like, one wonders what kind of insider knowledge Mr. Ballmer has, or if he is just speculating like everyone else.

What's really interesting to Green Computing Report, though, is the energy load of these one-million computing machines. It so happens that Amazon Web Services engineer James Hamilton was wondering the same thing. Hamilton, who left Microsoft for Amazon Web Services in 2008, extrapolates a total energy consumption of 300MW.

Hamilton provides this math trail:

Assume that each server runs 150 to 300W including all server components with a weighted average of say 250W/server. And as a Power Usage Effectiveness estimator, we will use 1.2 (only 16.7% of the power is lost to datacenter cooling and power distribution losses). With these scaling points, the implied total power consumption is over 300MW. Three hundred million watts or, looking at annual MW-h, we get an annual consumption of over 2,629,743 MWh or 2.6 terawatt hours – that's a hefty slice of power even by my standards. As a point of scale since these are big numbers, the US Energy Information Administration reports that in 2011 the average US household consumed 11.28MWh. Using that data point, 2.6TWh is just a bit more than the power consumed by 230,000 US homes.

Extending Hamilton's estimate to carbon footprint, 2.6 TWh is akin to 1,834,430 metric tons of CO2. An equivalent amount of emissions is created by 380,000 passenger vehicles in one year or the average coal-fired power plant every six months. Even if 25 percent of the datacenter power supply comes from renewables (a generous assumption), that still leaves a hefty carbon footprint of 1.35 metric tons of CO2.

Microsoft is reportedly instituting a policy of carbon neutrality beginning in fiscal year 2013. The strategy includes internal carbon fees and carbon offsets.

Working back from his 300MW figure, Hamilton attempts to determine the total number of Microsoft datacenters. Factoring out the smallest sites and assuming an average datacenter operates at 20MW, a 300MW total power envelope would give Microsoft about 15 large sites, which seems about right to this former Microsoft employee. (If you're wondering how much insider knowledge Hamilton has, he left Microsoft in 2008, two years before Azure was publicly-announced.)

Hamilton also tackles the cost implications of being in the one-million server club. If the average of all the servers comes out to $2,000 a piece, a modest estimate, then that's a total of $2 billion before factoring in what it would cost to house and manage the servers. His final analysis: an infrastructure investment of $4.25 billion total. Hamilton characterizes this as "an athletic expense even for Microsoft but certainly possible."

As for the other companies mentioned by Ballmer, Amazon and Google have never released server count data. However, the Microsoft CEO's claims aren't far off from the Webosphere's current best guesses, which put Amazon at around a half-million servers and Google at around a million. Was Ballmer just parroting these oft-repeated numbers or does he know something we don't and why does he think that Google has more servers than Microsoft? Do you know the answer? Leave your comment below or send me an email at [email protected].

EnterpriseAI