Posted by Michael Bullock on Thu, Mar 18, 2010
Computation Fluid Dynamics - Hot topic at Data Center World
Congrats to AFCOM for another fine event! More than 800 IT and facilities attendees converged in Opryland last week to learn the latest best practices for data center design and operations. Most seemed very keen on improving data center efficiency.
And like usual, there were plenty of vendors hawking their wares to scratch that itch. From floor fan tiles and cable racks - to containers and critical infrastructure, no stone seemed unturned anywhere there was a buck to be made.
So was it just me or did CFD modeling seem to take center stage at many of the vendor booths and presentations? And for good reason I might add. Computation Fluid Dynamics provides a visual illustration of your data center cooling system where you can see clear cause and effect of hot spots, inefficient design and suboptimal deployment.
Sophisticated CFD modeling has been used by Fortune 100 companies to design products ranging from automobiles to hypersonic spacecraft, and has been more recently adapted to data center cooling and airflow analysis. You can rotate a three dimensional rendering of your data center viewing it from all angles and are not limited to a fixed perspectives.
As you know, the main purpose of a data center cooling systems is to maintain equipment within acceptable operating temperature range. This helps to reduce downtime, extend equipment life and optimize energy costs associated with the air conditioning system. Achieving this requires an adequate supply of cooling air as well as effective distribution of the airflow to the inlets of all the racks.
Recent studies have shown that although most Data Centers have over twice the cooling capacity required, they still experience "hot spots" throughout the data center. This is a result of poor airflow management. And since airflow and pressure are invisible, it is difficult to develop a strategy for improving airflow management without the use of an airflow management modeling tool.
With CFD analysis, you can see these invisible temperature differences and airflow pathways. Here is a view that combines temperature and airflow in a single 3D perspective. Notice the red / hot IT racks and the cooler / blue CRACS.
CFD modeling also allows you to do "what if" planning if you want to optimize your current data center in concept, before physical implementation begins. You may believe that ultrasonic humidification will allow you to cool more efficiently, but will it go far enough in power and cooling recovery to postpone a pending data center migration?
And I caution you to keep in mind that while these tools do an excellent job in visualizing temperature and airflow, you still need to be diligent when factoring in power, efficiency and overall return on investment. For example, if you improve the efficiency of your critical infrastructure by 50% (i.e. improving PUE from 3.0 to 2.0), will you be able to immediately apply the recovered power to IT systems?
This partly depends on your infrastructure to deliver conditioned power - the load that can be sustained by your UPS and IT power distribution systems. But even if are not able to expand with this recovered power and cooling, you still will have reduced your data center electric bill by a whopping 33%!
As always, I welcome feedback, questions and comments. And if you know of other companies effectively enabling cloud computing with an impact on the enterprise you believe similar to those listed above, I'd be interested in learning more. You may reach me at mailto:cioblog@transitionaldata.com
Posted by Michael Bullock on Mon, May 18, 2009
Maybe it was just wishful thinking on my part but I really believed that the Berlin Wall between IT and facilities had been torn down a long time ago and that in corporate America these groups were generally aligned. Okay, maybe IT and facilities weren’t holding hands and singing “Kumbaya,” but at least they were talking to each other and beginning to understand where their mutual interests lay. But at the IT Roadmap Conference in Boston last week, I realized how far off base I really was.
While I’ve been busy blogging about power usage efficiency (PUE), ultrasonic humidification, green data centers, the power density paradox and the like, it seems I’ve been operating on the mistaken premise that most IT execs are reasonably aware of how much running their data centers costs in electric power. I also assumed that these execs were at least somewhat motivated to make improvements, especially when doing so would entail no risk to operations and paybacks could be achieved in months, not years.
But no such luck. Most of these IT guys have no idea that they’re probably spending as much dough each year on the electricity to run their servers as they’re paying to buy new ones. Most have no clue that for every dollar they’re spending to run IT systems, another dollar (or more) may be wasted by facility inefficiencies.
According to Johna Till Johnson of Nemertes Research, a full 87% of the IT execs Nemertes interviewed (more than 2,500 from Fortune 2000 companies) said they didn’t know their data center energy costs. And that, Johnson says, includes the 97% who said they’ve consolidated or virtualized their data centers. “Basically,” she concluded, “nobody knows their data center energy costs.”
But I bet your facilities group knows. Why? They know because they’re the ones being asked to reduce the electric bill by the CFO while the CIO asks them to provision more services even though they’re fast approaching the limits of available power and cooling. And they know because they’re the ones paying for the rising maintenance costs caused by systems constantly running above their designed load limits.
In other words, the facilities guys know what you IT guys should know, what you need to know to do your jobs properly. So stop being clueless and get on the same hymn book page with facilities. And if you need more reasons, check these out:
- Much of the OPEX you’re wasting on power can be recovered and goes directly to your company’s bottom line. You can continue to ignore this at your peril.
- Whether you know it or not, you’re probably running out of power and cooling and ideas like migrating to higher density systems or containers are not going to solve your problem; they may, in fact, make it worse.
- If you think that the increased maintenance resulting from constant duty cycles of facilities infrastructure isn’t your problem, think about the fact that these systems are more susceptible to failures that may result in full data center outages.
- As your company strives to become a more responsible corporate citizen, IT will come under the microscope. There will be pressure to reduce your company’s carbon footprint and increase corporate sustainability. As these initiatives gather steam, IT and facilities will be heroes or goats. There’s no middle ground for either group.
This is a great opportunity for IT to take a leadership role and drive the changes that will save money, increase capacity and reduce your company’s carbon footprint. The only way for IT to do this is to get beyond thinking about facilities as simply square footage and maintenance. At the same time, the facilities group has to stop regarding IT as pointy-headed know-nothings. Working collaboratively is the only way IT and facilities can solve their mutual challenges.
Posted by Michael Bullock on Thu, Feb 26, 2009
I recently attended a ribbon cutting ceremony for Internap's newest data center in Boston. Besides receiving a $453,000 rebate for power efficiency from NSTAR (the largest NSTAR ever handed out), there were several other significant accomplishments achieved in building the colocation provider's new, green, Tier III-level data center:
1. It was built for under $1,200 per square foot – well below the Uptime Institute estimate of $2,500 to $5,000 for this type of facility.
2. The entire project—from permits to construction to inspection, all the way up to the facility's first tenant—took only 12 months to complete. Yet most people I speak with assume a project like this will take about 24 to 30 long, painful months.
3. The Internap data center is expected to use $400,000 less electricity per year vs. a comparable, conventionally-built facility.
You may be asking yourself, "How did Internap build its data center for a third of what the Uptime Institute said it should cost?" Not to mention, "How did it do it in half the time the prevailing wisdom says it should have taken?"
Some of those questions are answered in a recent Network World article and slide show, "What does a real green data center look like?" that also explains the project management and oversight services my company, TDS, provided in this project. Here are some other factors that led to the company's data center being completed on spec, on time and within budget. Internap:
- Worked closely with the community, including local permitting and inspection authorities to avoid late and unpleasant surprises
- Removed heat generating support systems from the raised floor area
- Eliminated steam humidification (ultrasonic is 93 percent more efficient)
- Purchased equipment directly from suppliers, getting rid of one and sometimes two layers of middle-man markups
- Used step-down transformers that are 25 percent more efficient than conventional K-factor transformers
- Used variable frequency drives (VFDs) to move air and fluids more efficiently
- Painted the roof white to reflect light and reduce heat build up
- Used 36-inch raised floors for efficient cold air supply
- Used 8-foot air return plenum in the ceilings to provide for efficient hot air exhaust
By now you may be saying, "Yeah, that's all great. But it doesn't explain how it could be built in half the time." And that, my friends, is because I'm saving the best for last.
The single decision you will make that has the greatest impact on your building costs, operating expenses and delivery time—the decision Internap addressed first—is site selection. That's why the two questions you need to ask yourself before anything else is, "Where am I putting this data center?" and "Why am I putting it there?"
If you pick a location with insufficient electrical power supply, plan on waiting 18 months for the power upgrades - on your dime. (Actually, it will be a lot more than a dime; it probably will be more on the scale of Internap's record NSTAR rebate.) If you choose a classy, A-type downtown office building, you'll undoubtedly have problems getting a floor that's sufficiently raised for cooling or enough space above the ceiling for adequate heat exhaust. If you pick an office complex, then I wish you the best of luck (which you'll need) in getting permits and approvals for roof-mounted or exterior equipment, as well as the fuel you'll need for backup power generators.
And when it comes to site selection for a new data center, do yourself a favor: Avoid relying too much on commissioned real estate brokers who often have little or no clue as to why data center real estate is different from office or manufacturing space. Also make sure your internal team is aware of the unique requirements of a state of the art, high availability and high density data center. Finally, make sure you start with your goals, cost targets, requirements and timeline established, and then plan from there.
So what space did Internap pick for its new data center? The building was previously a 5,000 member church, and before that it was a warehouse. The point is that the facility, the power grid, the zoning, the community, and Internap itself were all aligned for success. And that's why it only took a year to build.
Posted by Eric Kraieski on Sun, Apr 06, 2008
Condensed version of the EPA report to congress regarding Public Law 109-431:
Posted by Eric Kraieski on Sun, Apr 06, 2008
The United States (U.S.) Environmental Protection Agency (EPA) developed this report in response to the request from Congress stated in Public Law 109-431. This report assesses current trends in energy use and energy costs of data centers and servers in the U.S. and outlines existing and emerging opportunities for improved energy efficiency. It provides particular information on the costs of data centers and servers to the federal government and opportunities for reducing those costs through improved efficiency. It also makes recommendations for pursuing these energy-efficiency opportunities broadly across the country through the use of information and incentive-based programs. The full report may be viewed below: