Posted by Michael Bullock on Thu, Mar 18, 2010
Computation Fluid Dynamics - Hot topic at Data Center World
Congrats to AFCOM for another fine event! More than 800 IT and facilities attendees converged in Opryland last week to learn the latest best practices for data center design and operations. Most seemed very keen on improving data center efficiency.
And like usual, there were plenty of vendors hawking their wares to scratch that itch. From floor fan tiles and cable racks - to containers and critical infrastructure, no stone seemed unturned anywhere there was a buck to be made.
So was it just me or did CFD modeling seem to take center stage at many of the vendor booths and presentations? And for good reason I might add. Computation Fluid Dynamics provides a visual illustration of your data center cooling system where you can see clear cause and effect of hot spots, inefficient design and suboptimal deployment.
Sophisticated CFD modeling has been used by Fortune 100 companies to design products ranging from automobiles to hypersonic spacecraft, and has been more recently adapted to data center cooling and airflow analysis. You can rotate a three dimensional rendering of your data center viewing it from all angles and are not limited to a fixed perspectives.
As you know, the main purpose of a data center cooling systems is to maintain equipment within acceptable operating temperature range. This helps to reduce downtime, extend equipment life and optimize energy costs associated with the air conditioning system. Achieving this requires an adequate supply of cooling air as well as effective distribution of the airflow to the inlets of all the racks.
Recent studies have shown that although most Data Centers have over twice the cooling capacity required, they still experience "hot spots" throughout the data center. This is a result of poor airflow management. And since airflow and pressure are invisible, it is difficult to develop a strategy for improving airflow management without the use of an airflow management modeling tool.
With CFD analysis, you can see these invisible temperature differences and airflow pathways. Here is a view that combines temperature and airflow in a single 3D perspective. Notice the red / hot IT racks and the cooler / blue CRACS.
CFD modeling also allows you to do "what if" planning if you want to optimize your current data center in concept, before physical implementation begins. You may believe that ultrasonic humidification will allow you to cool more efficiently, but will it go far enough in power and cooling recovery to postpone a pending data center migration?
And I caution you to keep in mind that while these tools do an excellent job in visualizing temperature and airflow, you still need to be diligent when factoring in power, efficiency and overall return on investment. For example, if you improve the efficiency of your critical infrastructure by 50% (i.e. improving PUE from 3.0 to 2.0), will you be able to immediately apply the recovered power to IT systems?
This partly depends on your infrastructure to deliver conditioned power - the load that can be sustained by your UPS and IT power distribution systems. But even if are not able to expand with this recovered power and cooling, you still will have reduced your data center electric bill by a whopping 33%!
As always, I welcome feedback, questions and comments. And if you know of other companies effectively enabling cloud computing with an impact on the enterprise you believe similar to those listed above, I'd be interested in learning more. You may reach me at mailto:cioblog@transitionaldata.com
Posted by Michael Bullock on Fri, Dec 11, 2009
Let's you and I do a little math about the National Security Agency's planned $1.5 billion data center.
Reading about the National Security Agency’s (NSA's) plan to build a $1.5 billion cyber security data center (Information Week) at the Camp Williams National Guard training base in Utah, I became curious about just how much one could glean about the project from public NSA budget information. Here's what I took away:
At first, this looks like a pretty good deal for Uncle Sam. We seem to be talking about a 1.5 million square foot data center that’s going to cost $1.5 billion, or $1,000 per square foot. That's well below what one would expect to pay for a Tier 3 class data center facility. Or is it?
For me, the math just doesn't add up. According to the budget document, the power density will be "appropriate for current state-of-the-art high-performance computing devices and associated hardware architecture." Yet if you calculate the watts per square foot by dividing the center’s total watts (65MW) by total square feet (1.5 million), you come up with a power density estimate of about 43 watts per square foot. No way that's "state of the art."
So let's say you triple the power density to a relatively modest 130 watts per square foot. That means you could support the center's full load of 65MW in about 500,000 square feet of space. At this power density, you'd probably require another 500,000 square feet for support space (generators, UPS, cooling, etc.). That's 1 million square feet. So what happens with the remaining half million square feet? According to the U.S. Army Corps of Engineers (as reported in the Information Week story) this data center "will eventually employ between 100 and 200 workers." That translates into a whopping 2,500 square feet of office space per employee. Roomy, wouldn't you say?
Now let’s suppose this facility really did support current state-of-the-art power density. That would mean about 400 watts per square foot. Given that density, the total space the data center actually would require would be just 160,000 square feet (i.e. 160,000 square feet x 400 watts per square foot = 64MW). Even assuming the same 500,000 square feet for support, that leaves us with a data center that's 660,000 square feet. Consequently, when you do the math, a 1.5 million square foot facility seems nutty and hard to reconcile with "state-of-the-art" anything. So what's going on?
My guess is that either the NSA has grossly miscalculated their space and power requirements or, perhaps, the true purpose and scale of the facility is a secret. The NSA keeping something secret? That wouldn’t be unheard of, would it?
Just for kicks, I wanted to see how this would compare to Uptime Institute estimates for this type of facility. I assumed 200 watts per square foot and used the Uptime Institute’s guidelines for a Tier 3 data center: $23,000 per KW of load and $300 per square foot. Using this approach, I came curiously close to NSA's $1.5 billion budget number ($1.495 billion, to be precise), and that made me wonder if the data center's budget is being built from the bottom up, or is simply a number tossed out to the public based on Uptime Institute estimates.
One last thought. Do you know how much money it will cost to operate a data center like the proposed Camp Williams facility? Based on a 65MW IT load, a PUE of 1.3 and Utah's cost of $.07 per kilowatt of electricity delivered (a nice rate while it lasts), it will cost $40 million per year simply to pay the electric bill.
But guess what? The reason the electricity is so cheap is because 98 percent of Utah’s electricity is produced by coal and natural gas. That's not very carbon friendly and with pending cap and trade legislation Utah's electricity costs will most definitely increase. How much? Who knows?
But whatever it is, the taxpayer (that's you) will be paying for it.
As always, I welcome feedback, questions and comments. You may reach me at cioblog@transitionaldata.com.
Posted by Eric Kraieski on Mon, Mar 02, 2009
Read Green Data Center Article
View Green Data Center Slideshow
In February, Network World published a nice piece on the Internap data center in Sommerville, MA. Here are the first few paragraphs:
"Companies looking for a green data center model should take a look at the new facility Internap built in Somerville, Mass., just outside Boston, which is so environmentally efficient the local power company wrote it a rebate check for $453,000.
"A renovated warehouse building that most recently housed a 5,000-member church, the collocation facility is optimized to economically meet demands for cooling, humidity and power consumption that are common to all data centers.
"Internap expects that the data center will save another $400,000 every year by using less power than it would have had it not built to green specifications, says Mike Higgins, vice president and general manager of Internap's data center services. That is helpful to its bottom line as well as keeping down rates it charges customers, he says."
Posted by Michael Bullock on Fri, Feb 20, 2009
If you’re running out of power, cooling or physical space in your data centers, pay attention. Not to the snake oil hype promising to solve all your problems, but to the Power-Density Paradox. If you don’t, it could ensnare you in an unappetizing manner.
The power-density paradox is this: As you increase power density in your data center, the need for support space (power, cooling and humidification) will increase disproportionately. And this in turn means that you efforts to recover space in the raised floor area of your data center may be more limited by your space outside of the data center floor. In other words, your efforts to free up space in your data center could boomerang if you don't have sufficient space for the support gear needed for these higher power densities.
Of course, it’s reasonable to want to increase capacity in a limited space by turning to higher density server and storage systems. By packing more gear into your current space, it may be possible for you to delay or avoid a costly data center relocation. Deploying high density blade servers and storage, data center containers, modular power systems, virtualization and cloud computing offer the potential for optimizing the space you already have.
But (and this is a big but) you need to plan carefully or you’ll fall victim to the power-density paradox. Drilling down, the paradox says that as you use more dense equipment (which places greater demands on power and cooling), you will quickly reach an inversion point where more floor space is consumed by support systems than is available to your IT equipment – typically between 100 and 150 watts per square foot. This translates into greater capital and operational costs, not the reductions you were hoping to achieve.
How much space will you need? At a power density of about 400 watts per square foot, plan to allocate about six times your usable data center space for cooling and power infrastructure. And if you are running your CRAC units on the raised floor space, you will lose even more precious space for these additional units and the mainenance buffer each demands.
So before you embrace high-density as a quick fix to your space problem, make sure you have adequate room to house the additional power and cooling infrastructure, sufficient raised-floor space to handle the increased airflow demands of hotter-running boxes and, of course, sufficient available power to operate the hungry systems and their support gear. If any of these resources are unavailable or inadequate, your data center will not support the increased power density. And you will have wasted your time and money.
In meeting the power-density paradox, your challenge is to balance the density of servers and other equipment in your data center with the availability of power, cooling and space so you truly gain operating efficiencies and lower net costs.
If you are already bumping up against these limits, all is not lost. There are many steps you can take to extend the life of your current facility. I’ve previously blogged about ultrasonic humidification, which is one way you can decrease your power demands while increasing your cooling capacity, letting you drive more IT equipment by recovering precious electricity wasted on inefficient humidification and cooling. You can retrofit high efficiency motors, pumps and transformers into your mechanical, electrical and plumbing infrastructure.
Besides saving on power at the point of use, the cost of these upgrades can often be recovered through utility company rebates. I’ve seen paybacks in a few months and annual power savings exceed a million dollars for a single company.
If you would like to learn more about the power-density paradox, drop me an email. As always, thank you for sending comments, tips and topic suggestions to me at CIOblog@TransitionalData.com.