Posted by Michael Bullock on Tue, May 18, 2010
AFCOM recently suveyed 436 member data center sites on eight of the hottest topics ranging from Greening and Data Center Consolidation to Performance Monitoring and Cyber Terrorism.
-
- 71% are actively engaged in greening
- Power Recovery - 60%
- Cooling Recovery - 51%
- Expansion
- 60.3% require additional data center capacity within five years
Please view the tabulated results and significant findings below:
Posted by Michael Bullock on Fri, Apr 30, 2010
Think IT and facilities operating in silos is last year's problem? In some cases, it may actually be getting worse.
Last year I wrote Four Reasons to Get Facilities and IT on the Same Page. At the time, I was surprised by how little IT leaders knew about their real data center costs, and I suggested that these leaders would do well to talk to their facilities people to understand better where their money was going. I even showed how IT could help facilities by understanding their problems. Since collaboration between IT and facilities can provide immediate financial benefits to the enterprise without layoffs or the risk of IT outages, there's a lot in it for both groups to work together.
At least one would think.
But when I bring the subject up, especially with the press, I find that most believe the problem has already been solved. "IT and facilities operating in silos is last year's problem, right?" Wrong. When I work with real world customers, I realize that the situation is bad and maybe even getting worse.
Why is there this disconnect between perception and reality? Perhaps it's because when the press looks for a story, it needs real-life case studies to make the story compelling and believable. And, of course, it is possible to find companies where, in fact, facilities and IT are on the same page. Naturally, they're willing to go public with their stories. But nobody is all that interested in going on record to talk about how they've made mistakes, how IT and facilities don't work together, or how they're pouring money down the drain by having both IT and facilities operate in a vacuum.
Don't believe there's still a problem? Here are some real life examples illustrating what can go wrong when facilities and IT don't collaborate:
• I saw a company purchase a building because it looked like an attractive value and, as a bonus, it already had a data center the company thought it could use. Because the facilities group knew IT needed a new data center, it followed the realtor's recommendation and purchased the building at a bargain. After learning that their new data center would require an additional $17 million in upgrades to make it usable for IT, plus another $500,000 a month in WAN charges, the company decided a collocation alternative would be a better approach. In the end, the company avoided that $17 million upgrade but it also ended up with a building it didn't really need. This could have been avoided with closer collaboration between facilities and IT.
• I recently visited a 9,000 square foot data center where something was amiss. IT had requested Tier 3 level (N+1) redundancy so the facility would be concurrently maintainable. Facilities actually delivered what they believed to be a full Tier 4 environment (2N) with no single point of failure. Besides costing the company over $500,000 more than it should have, the center actually did have a single point of failure, meaning it wasn't even a Tier 3 data center. Another investment of $150,000 was required to correct the problem.
• I've recently seen over 100 cabinets at one location ejecting hot exhaust right into the intake of other systems. Besides subjecting these systems to the risk of premature failure, this made it necessary to run the data center much cooler than it needed to be just to head off cooling-related problems. This unnecessarily raised the PUE, resulting in an electric bill that was 30 percent higher than it needed to be. By now everyone knows about hot aisle-cold aisle alignment, but many legacy data centers—such as this one—haven't been reconfigured to take advantage of efficiency improvements like that just because IT and facilities are not communicating and are instead holing up silently in their respective silos.
I have other stories like this; I see them every day.
Let's face it. Facilities typically reports to the CFO's office and IT typically reports to the CIO and when the CFO and CIO get together they have better things to do than discuss PUE, power density, cooling, ultrasonic humidification and data center tier ratings (although I'm sure they have a chat or two about budgets).
CFOs and CIOs expect their respective teams to do the right thing. We all understand that. But maybe these teams need some top-down guidance to help get them moving in the right direction, together.
So I open this blog up to learn about your own experiences with IT and facilities. Have you had any great successes or horror stories you'd like to share?
As always, I welcome feedback, questions and comments. And if you know of other companies effectively enabling cloud computing with an impact on the enterprise you believe similar to those listed above, I'd be interested in learning more. You may reach me at cioblog@transitionaldata.com.
Posted by Michael Bullock on Tue, Oct 27, 2009
What you don't know can hurt you...
If you're in the market for new or additional data center space, I have some advice that can help you make better decisions, save money, and avoid some lurking pitfalls.
For example, a company found what at first seemed like the one of the most affordable spaces around, but after I asked the colocation provider a few pointed questions (actually, it was more like 70), it turned out to be one of the most expensive. Once the provider was pinned down and the customer's true requirements were taken into account, it turned out that all the power charges were vastly underestimated and the simple, 1,000 square foot / 120KW facility would have cost the customer about 250% more than he originally thought, amounting to an extra $1.6 million over the first five years of tenancy.
That's a lot of money.
So my advice has two parts:
- Try to keep more than one target space in the mix until the deal is done. Even if you are actively negotiating with your first choice, it pays to have a backup option. At minimum, this will allow you to negotiate without feeling trapped and provides a solid plan ‘B' plan should the preferred space become unavailable - or too costly upon full discovery.
-
My second tip is that you should go into negotiations armed with a trusty sword of skepticism. As Ronald Reagan said many years ago, "Trust, but verify." And since you and your company will be living with this decision for at least 5-10 years, it's very important to understand the details and long term impact of your agreement with your data center provider. It's always a mistake to view them as IT consultants who are there to help you. Their goal is to get you to commit to their space and lock you in while they retain the flexibility to increase their rates over the long term. This is not because they're bad guys. This is simply their business. You'd do the same.
Here are some of the collocation gambits that will end up costing you:
The Shell Game -- As you know, total facility fees include rent, power (utility) and a power surcharge that covers cooling, humidification, UPS (to avoid spikes and dips), etc. An inefficient facility will look very expensive and risky based on this surcharge, which could amount to one-to-two times as much as the cost of the actual metered power usage. So as not to scare off prospective customers, some operators may try to hide a percentage of this cost (that comes from their own inefficiency) by burying it in your lease charge. This would not be an issue were this a fixed cost, but more often than not the lease is subject to annual escalations. So you need to make sure you understand your provider's true operating expenses, pass through charges and cost escalation terms in order to have some degree of cost predictability going forward.
Convenient Laziness -- I've seen this happen this way: You put out an RFP with very specific questions, such as, "Does your proposal include redundant power feeds?" The bidder provides a one page summary of its expected charges and then replies to your question about backup power by writing, "See quote." When you look at the quote, you see a line item for redundant power. So does that answer your question? Maybe yes, maybe no. The quote may refer only to the UPS battery backup on the primary line and not to the second power feed into the building that's required for Tier 3 facilities. So, at best, it takes a lot of digging and hard work on your side to interpret the provider's response when he writes "See quote," and there's always the risk that you won't get it right. On the other hand, this convenient laziness in the response to your RFP allows the provider to crank out many quotes for many RFPs using an Excel spreadsheet with fixed assumptions that have nothing to do with your unique business and its unique requirements, not to mention the specifications contained in your RFP-the only one you care about. So when issuing an RFP for data center space, you should make it clear that specific, detailed responses are required or the bid will be rejected as noncompliant.
The Old Bait and Switch -- In this case, the bidder may provide detailed responses to your questions but he may not have the capacity or architecture to live up to his promises or your requirements. Recently, I saw a colo provider assure a customer of Tier 3 redundancy even though its facility's backup power feed was unconditioned street power. This meant that it was unprotected by UPS power and therefore represented a single point of failure and vulnerability to power spikes whenever the primary went down for maintenance. The tenant was paying market rate (which seemed like a good deal) for a true Tier 3 facility but the truth was that it wasn't getting one. In other cases, I've seen mismatches in what a given facility is capable of providing and what is actually quoted and provisioned to the customer. In these cases, the provider hopes first to get the new tenant in the space and plans to address the requirements later, after the relocation has been completed. I wouldn't feel comfortable with that set-up. Would you?
Rounding Down -- In this scenario, the customer asks for a specific power capacity he requires to operate his IT equipment. The colocation bidder immediately discounts this by 20%, claiming, "You shouldn't run the system above 80% of design load." So the facility provides you with 120 KW of breakered service where only 80% (or 96KW) is available for IT use. Of course, it could have provided 150 KW of breakered service (120 IT usable), but then its price would look too high. All the power and cooling estimates you get after that are based on this assumed 20% reduction to your requirements. If you don't look at the details, you may end up with a space you thought was 20% more efficient than others but really has 20% less capacity than the alternatives.
The message is that when evaluating new data center space, make sure you're diligent, and be on your guard. Take advantage of the people in your organization (or outside experts) who have the experience to know how large data centers are operated and where to look for the traps and gaps that will cause your requirements to go underserved. And beware of real estate brokers who may not understand what data centers really require and simply try to sell you on sites that pay them a higher commission. I'm not saying all real estate brokers are like that . . . I'm just saying . . .
As always, I welcome feedback, questions and comments. You may reach me at cioblog@transitionaldata.com.
Posted by Michael Bullock on Mon, Apr 06, 2009
With all the bad news about the economy, rising unemployment, corruption in high places and mind-boggling billion dollar bailouts, wouldn’t it be great to read about something positive once in a while? Taking the direct approach, I Googled “positive news” and was surprised to see how many websites are dedicated to it. Unfortunately, they’re not all that meaty and they certainly don’t target IT and the data center, which is why we’re all here, right?
So I tuned into Len Eckhaus (founder of AFCOM and board member of the Data Center Institute) and his recently updated “Five Bold Predictions for the Data Center Industry that will Change Your Future” (originally published in 2006).
Here’s a quick summary:
- Original: By 2010, more than half of all data centers will have to relocate to new facilities or outsource some applications.
Update: He has revised this prediction downward, but not by a specific amount and probably because most organization that were planning in 2006 to move by 2010 are already deep into the process, shrinking the pool a bit.
- Original: Within the next five years, one out of every four data centers will experience a business disruption serious enough to affect the entire company’s ability to continue business as usual.
Update: Actually, Ekhaus says that things are even worse than that due to the widespread budget cuts enforced by the economic crisis and the consequent delay in making upgrades to data center infrastructure, hardware, software, and support systems.
Positive news? That’s positive news? You may be thinking, “Has Mike lost his mind? This all sounds like horrible news!”
Well, everything comes down to one’s perspective, right? If our starting point is the world of 2006, which we viewed through rose-tinted lenses, of course everything after that looks and sounds like bad news. So now that we’ve removed the glasses (or had them smashed for us), I’ve decided to take a fresh look and find the realistic upside for IT and for your career.
Accordingly, here are three predictions you can use to leverage Eckhaus’ somewhat dire analysis to your advantage:
1. Highly motivated to improve productivity and quality while delivering greater business value, the IT industry will show remarkable adaptability and resilience over the next five years. IT organizations, their suppliers, employees and external partners will all work together like never before to get the most out of IT budgets, technology and data center resources. This will usher in an age of pragmatism, eliminating the waste and inefficiency that all too often characterized corporate IT. Employees, vendors and VARs who do not adapt will be replaced by more competent alternatives. (See “Avoid the Hidden Cost of Bias and Preference in IT”). The upside for you is that the faster you embrace the concept of cooperation, collaboration, and efficient resource usage, the more valuable you will become to your current or future employer.
2. Driven by more efficient software, hardware, and facilities infrastructure “sustainable IT” will become much more than a catch phrase in the next decade. In the data center market, this means Tier IV (2N+1 redundancy) will be recognized widely as inefficient and wasteful while Tier III (concurrently maintainable) will be seen as perfectly adequate, not to mention much more cost effective. By focusing on efficiency and sustainability in IT operations, many companies will be able to defer costly consolidations and relocations. For those organizations that have no alternative but to move soon, there’s a new generation of colocation data centers coming online (like the new Internap facility near Boston) that were built from the ground up with a green and sustainable approach.
3. I predict that 98% of organizations that fail to plan ahead will be blindsided by power availability or cost issues over the next five years (one in 50 may get lucky). This is the consequence of the combined effect of the growth of information and power consumption, and the constraints imposed by power availability and inefficiency. Since half the power in a typical data center goes toward support systems (power conversion and cooling), those systems act as a perpetual (and unnecessary) tax on your operations. Luckily, technologies already exist that can help: ultrasonic humidification, variable frequency drives, high efficiency transformers and Energy Star servers, just to name a few. So if you want to make a good career decision, now is the time to begin implementing power saving initiatives that have fast paybacks and can help you avoid a power crisis in the future. Or, like I said, maybe you’ll get lucky. (Me, I don’t advise banking on luck to see you and your company through to the turnaround.)
I hope this gives you some ideas on how you can prioritize your future and develop greater value for your employer. If this advice comes too late for your present job situation, here are some other CIO.com career links you may find useful: “5 tips to get on a recruiter’s good side,” “how not to piss off a recruiter,” and “calculating the risk you’ll lose your job.”