Posted by Michael Bullock on Fri, Apr 30, 2010
Think IT and facilities operating in silos is last year's problem? In some cases, it may actually be getting worse.
Last year I wrote Four Reasons to Get Facilities and IT on the Same Page. At the time, I was surprised by how little IT leaders knew about their real data center costs, and I suggested that these leaders would do well to talk to their facilities people to understand better where their money was going. I even showed how IT could help facilities by understanding their problems. Since collaboration between IT and facilities can provide immediate financial benefits to the enterprise without layoffs or the risk of IT outages, there's a lot in it for both groups to work together.
At least one would think.
But when I bring the subject up, especially with the press, I find that most believe the problem has already been solved. "IT and facilities operating in silos is last year's problem, right?" Wrong. When I work with real world customers, I realize that the situation is bad and maybe even getting worse.
Why is there this disconnect between perception and reality? Perhaps it's because when the press looks for a story, it needs real-life case studies to make the story compelling and believable. And, of course, it is possible to find companies where, in fact, facilities and IT are on the same page. Naturally, they're willing to go public with their stories. But nobody is all that interested in going on record to talk about how they've made mistakes, how IT and facilities don't work together, or how they're pouring money down the drain by having both IT and facilities operate in a vacuum.
Don't believe there's still a problem? Here are some real life examples illustrating what can go wrong when facilities and IT don't collaborate:
• I saw a company purchase a building because it looked like an attractive value and, as a bonus, it already had a data center the company thought it could use. Because the facilities group knew IT needed a new data center, it followed the realtor's recommendation and purchased the building at a bargain. After learning that their new data center would require an additional $17 million in upgrades to make it usable for IT, plus another $500,000 a month in WAN charges, the company decided a collocation alternative would be a better approach. In the end, the company avoided that $17 million upgrade but it also ended up with a building it didn't really need. This could have been avoided with closer collaboration between facilities and IT.
• I recently visited a 9,000 square foot data center where something was amiss. IT had requested Tier 3 level (N+1) redundancy so the facility would be concurrently maintainable. Facilities actually delivered what they believed to be a full Tier 4 environment (2N) with no single point of failure. Besides costing the company over $500,000 more than it should have, the center actually did have a single point of failure, meaning it wasn't even a Tier 3 data center. Another investment of $150,000 was required to correct the problem.
• I've recently seen over 100 cabinets at one location ejecting hot exhaust right into the intake of other systems. Besides subjecting these systems to the risk of premature failure, this made it necessary to run the data center much cooler than it needed to be just to head off cooling-related problems. This unnecessarily raised the PUE, resulting in an electric bill that was 30 percent higher than it needed to be. By now everyone knows about hot aisle-cold aisle alignment, but many legacy data centers—such as this one—haven't been reconfigured to take advantage of efficiency improvements like that just because IT and facilities are not communicating and are instead holing up silently in their respective silos.
I have other stories like this; I see them every day.
Let's face it. Facilities typically reports to the CFO's office and IT typically reports to the CIO and when the CFO and CIO get together they have better things to do than discuss PUE, power density, cooling, ultrasonic humidification and data center tier ratings (although I'm sure they have a chat or two about budgets).
CFOs and CIOs expect their respective teams to do the right thing. We all understand that. But maybe these teams need some top-down guidance to help get them moving in the right direction, together.
So I open this blog up to learn about your own experiences with IT and facilities. Have you had any great successes or horror stories you'd like to share?
As always, I welcome feedback, questions and comments. And if you know of other companies effectively enabling cloud computing with an impact on the enterprise you believe similar to those listed above, I'd be interested in learning more. You may reach me at cioblog@transitionaldata.com.
Posted by Michael Bullock on Mon, Jul 20, 2009
I’m going to stay away from the global warming and climate change debate; it looks (for now) like that train has left the station. Instead, I’d like to focus on the potential tax exposure created by "Cap and Trade” as it relates to the data center.
Last week I wrote about the Five Myths of Data Center Optimization, or Power Usage Effectiveness (PUE). However, a concept that is not a myth is that power distribution and cooling expenses act as a multiplier to the total cost to operate a data center. In other words, a data center operating at a 2.0 PUE level is generally wasting 1/3 more of the electricity it purchases to support IT operations than one operating at a more efficient PUE of 1.3. If you can buy power cheaply enough, it may seem like the penalty created by that 2.0 PUE rating is inconsequential. Trust me, it’s not.
If you look more closely, you’ll see that cheap electricity does not necessarily translate to a reduced carbon footprint. In fact, some electricity that’s cheap by today’s standards may increase substantially in total expense to you in the near future as Cap-and-Trade takes full effect. So, in the long run, making data center site selections based on today’s lowest cost of electricity may not be a wise strategy.
For example, the CO2 produced by nuclear and hydro-electric sources is minimal when compared to coal and petroleum-based plants, which are expected to produce 2.1 and 1.9 pounds of carbon dioxide gas respectively per kWh of electricity. Over time, as taxes work their way through the value chain and increase prices, you can expect the cost of electricity to reflect these differing methods of generation and their consequent emissions more accurately.
According to the EPA, the average emission rate of CO2 per kWh across all US regions is about 1.4 pounds. NY, CA, the Pacific Northwest and New England are all on the good side at about 0.9; the South averages around 1.5 and the Rockies come in at just over 2.0. And in the absence of some complicated, yet-to-be-defined measurement and auditing system, it seems logical that the federal government will use the EPA’s sub-region map for determining Cap and Trade allocations.
Here are two examples based on regional PUEs and local power grids:
• A data center in New England leveraging free cooling can achieve a sustainable PUE of about 1.3. At about .9 pounds of CO2, this facility generates about 1.17 pounds of CO2 per kWh.
• A data center in Texas with constant cooling demands will have a hefty PUE of about 2.0. Producing about 1.4 pounds of CO2 (per kWh), this facility will contribute 2.8 pounds of CO2 per kWh – for an increased carbon impact 140% above the New England example.
If you’re wondering just what impact this increased carbon footprint will have on global warming, join the club. There’s only one thing on which we can all probably agree – this increased carbon footprint will most assuredly be taxed. Bet on it. I don’t know about you, but avoiding an increased tax exposure of around 140% sounds like a pretty good idea to me.
By now you might be trying to figure out how cogeneration and alternative energy can help. Put down your pencil. Of the fossil fuel alternatives (eco-unfriendly, but reliable), natural gas is best at about 10 pounds per kWh (with diesel generators topping out over 22 pounds per kWh). Clearly, solar and wind power generation are carbon friendly, and may help your organization one of these days, but right now (and for the foreseeable future) they don’t scale to the magnitude and constant power demands required by corporate data centers.
I wish I could tell you how this will all play out in terms of specific tax impact, but your guess is as good as mine. In the meantime, the best advice is to focus on the stuff you can control. Optimize your PUE and don’t rush into a new facility without first taking a close look at your potential carbon exposure.
Listen carefully. Are those (carbon) footsteps you’re hearing? Or the tax man’s?
As always, I welcome your comments, tips, insights and topic suggestions. You can reach me at cioblog@transitionaldata.com.
Posted by Michael Bullock on Thu, Jun 18, 2009
In the U.S., it takes weeks to count ballots. Not in Iran. The answer must be cloud.
The recent Iranian election demonstrates the mind boggling power of cloud computing, social networking, and the role technology can play to improve the predictability of politics.
Think about it. Here in the U.S., it takes weeks to count millions of ballots in Florida and a few months to count thousands of ballots in Minnesota. But in Iran the vote counting is nearly instantaneous.
Consider how much more efficient things are in Iran where they can count millions of hand-written ballots – with fully consolidated reporting – in a matter of hours. Obviously, they must have some serious Persian/Farsi-based character recognition software in addition to computing capacity equal to Google’s and the NSA’s. Combined.
Now while I don’t claim to be an insider (and—full disclosure—the Iranian government is not one of my company’s clients), I’m almost certain this astounding speed was achieved with the aid of cloud computing. There is simply is no other reasonable explanation. So there you have it: today, Iran, unlike the U.S., is able to leverage cloud computing.
And consider the apparently ubiquitous adoption of social networking in Iran. It appears that those Iranians unhappy with the voting process (not to mention the result) are relying heavily on Facebook, Twitter and YouTube to voice their displeasure, using proxy servers to get around government interference.
Now the last thing any self-respecting despotic government needs is improved, real-time social communication networks that can empower citizens. But what, as any aspiring dictator might ask, can you do? Well, Iran is not taking its citizens’ exercise of digital freedom lying down. According to ComputerWorld, Iran has declared cyberwar (Iran's leaders fight Internet; Internet wins - so far ) in order to try to shut up those voters who don’t seem to appreciate the government’s technological advancements in vote-counting in the cloud.
Alas, as ComputerWorld reports, the Internet is a tough enemy “because of the resilience of a communications grid originally designed to be both resilient and pervasive. In fact, [the government’s] actions may also be crippling banking systems and hindering commerce in what is a technologically advanced nation. Cutting off Internet access affects more than Web sites or Twitter and Facebook. Credit card and ATM systems could be affected, as could critical infrastructures.”
This all goes to prove that cloud computing is alive and well in Iran. I’m not so sure about the instantaneous vote-counting application (it could be vaporware), but clearly social applications like Facebook, Twitter and YouTube are operating in a cloud/SaaS model.
So there you have it: rigged elections, a killer application to drive cloud computing and social networking. The U.S. better get on the stick. Clearly, we’re facing a cloud computing gap.
Posted by Michael Bullock on Mon, May 18, 2009
Maybe it was just wishful thinking on my part but I really believed that the Berlin Wall between IT and facilities had been torn down a long time ago and that in corporate America these groups were generally aligned. Okay, maybe IT and facilities weren’t holding hands and singing “Kumbaya,” but at least they were talking to each other and beginning to understand where their mutual interests lay. But at the IT Roadmap Conference in Boston last week, I realized how far off base I really was.
While I’ve been busy blogging about power usage efficiency (PUE), ultrasonic humidification, green data centers, the power density paradox and the like, it seems I’ve been operating on the mistaken premise that most IT execs are reasonably aware of how much running their data centers costs in electric power. I also assumed that these execs were at least somewhat motivated to make improvements, especially when doing so would entail no risk to operations and paybacks could be achieved in months, not years.
But no such luck. Most of these IT guys have no idea that they’re probably spending as much dough each year on the electricity to run their servers as they’re paying to buy new ones. Most have no clue that for every dollar they’re spending to run IT systems, another dollar (or more) may be wasted by facility inefficiencies.
According to Johna Till Johnson of Nemertes Research, a full 87% of the IT execs Nemertes interviewed (more than 2,500 from Fortune 2000 companies) said they didn’t know their data center energy costs. And that, Johnson says, includes the 97% who said they’ve consolidated or virtualized their data centers. “Basically,” she concluded, “nobody knows their data center energy costs.”
But I bet your facilities group knows. Why? They know because they’re the ones being asked to reduce the electric bill by the CFO while the CIO asks them to provision more services even though they’re fast approaching the limits of available power and cooling. And they know because they’re the ones paying for the rising maintenance costs caused by systems constantly running above their designed load limits.
In other words, the facilities guys know what you IT guys should know, what you need to know to do your jobs properly. So stop being clueless and get on the same hymn book page with facilities. And if you need more reasons, check these out:
- Much of the OPEX you’re wasting on power can be recovered and goes directly to your company’s bottom line. You can continue to ignore this at your peril.
- Whether you know it or not, you’re probably running out of power and cooling and ideas like migrating to higher density systems or containers are not going to solve your problem; they may, in fact, make it worse.
- If you think that the increased maintenance resulting from constant duty cycles of facilities infrastructure isn’t your problem, think about the fact that these systems are more susceptible to failures that may result in full data center outages.
- As your company strives to become a more responsible corporate citizen, IT will come under the microscope. There will be pressure to reduce your company’s carbon footprint and increase corporate sustainability. As these initiatives gather steam, IT and facilities will be heroes or goats. There’s no middle ground for either group.
This is a great opportunity for IT to take a leadership role and drive the changes that will save money, increase capacity and reduce your company’s carbon footprint. The only way for IT to do this is to get beyond thinking about facilities as simply square footage and maintenance. At the same time, the facilities group has to stop regarding IT as pointy-headed know-nothings. Working collaboratively is the only way IT and facilities can solve their mutual challenges.
Posted by Emad Omar on Tue, Apr 28, 2009
Is it just me or did anyone else ROFL when they learned about the pending Snoracle merger? On one hand, you have a company (Sun) that pioneered and embraced open source (UNIX, Java and MySQL). On the other (Oracle), you have something more along the lines of Star Wars’ interstellar crime lord Jabba the Hutt: "As long I get my cut, everything's good."
Oracle chairman Larry Ellison says: "The acquisition of Sun transforms the IT industry, combining best-in-class enterprise software and mission-critical computing systems. Oracle will be the only company that can engineer an integrated system - applications to disk - where all the pieces fit and work together so customers do not have to do it themselves. Our customers benefit as their systems integration costs go down while system performance, reliability and security go up."
Sun CEO Jonathan Schwartz had this to add: "This is a fantastic day for Sun's customers, developers, partners and employees across the globe, joining forces with the global leader in enterprise software to drive innovation and value across every aspect of the technology marketplace."
Oh, really? So Ellison and Schwartz want us to believe that their primary motivation was to make everything better for their customers and the industry. Oracle and Sun simply saw an opportunity to drive value and innovation across the technology marketplace. That's nice, but I'm not buying it.
CIOs, you may want to stop doing that happy dance now.
During an April 20 analyst call, Ellison called Java "the single most important software asset we ever acquired." Now we’re getting warmer. I agree with Citigroup analyst Brent Thill who said Oracle looked at Java as a $1 billion business even though it contributed just $220 million of Sun’s 2008 $14 billion revenue. Now you tell me: Will Oracle get Java from $220 million to $1 billion by increasing sales, or is it more likely that innovative re-pricing, like the Death Star, is heading our way?
Thrill also noted that Oracle expects half its 2009 revenue to come from the support and maintenance of products its customers have already licensed – support contracts that carry profit margins approaching 90 percent. Given that interesting expectation, can open source maintenance and support fees be far behind?
And, in this brave new world, what happens to MySQL, the leading open source database alternative? How will Oracle monetize MySQL, and will its attempt to extract value from this market backfire into a mass exodus to other open source database alternatives like PostgreSQL?
When I made my five hype-free predictions for 2009, two trends appear directly relevant to the Snoracle merger. Here’s what I said then:
- Open source enterprise software will continue to displace expensive royalty-based and seat license-based services.
- Facing slumping sales, many technology suppliers will try to increase revenue by raising their service fees and trying to force their clients to upgrade to new hardware and software.
If I may say so, the Force seems to have been with me. It looks like Oracle is building their game plan straight from this playbook.
And, to be fair, it all makes business sense. Sure, Oracle will slash the workforce on the server side of the Sun line, and Ellison will milk it as long as he can. That's not a bad idea and probably it’s the same thing IBM would have done if it had been able to snatch Sun itself.
On the other hand, I don't see Oracle emerging as a leading server company nor as the next Jedi guardian of open source software. As they say, a Wookie can't help being shaggy, and servers and opensource are two things missing from Larry "Java the Hutt" Ellison's DNA.
Posted by Michael Bullock on Wed, Feb 11, 2009
Cisco’s jump into the server market has certainly put the spotlight on the battle for the data center and HP was the first competitor to respond by cutting prices 30 to 50 percent on its network equipment -- a shot across Cisco’s bow. This is war, and IT departments will benefit from more options and/or better prices. However, as I take a closer look, I’m not even sure that Cisco and the traditional vendors in the data center space (HP, IBM, EMC and Juniper Networks) are fighting on the same battlefield. But one thing is certain: with $30 billion in the bank, Cisco is a force to be reckoned with and the other companies will have to get busy to refine their own go-to-market strategies.
How, exactly, will the new Cisco server be differentiated? We’ll know soon enough. Probably the most astounding aspect of this is how well Cisco has kept its real strategy and many of the important details out of the mainstream.
The New York Times coverage last month only touches the surface: “A Cisco-branded server with virtualization.” Trust me, there’s got to be much more to it than that because several questions immediately leap to mind: Will their servers be sufficiently differentiated to command a premium in the cutthroat server market? Will these systems be cheap enough to manufacture to be priced competitively against the incumbent vendors, who know all too well about the need for massive scalability and squeezing costs out at every opportunity?
Except for its consumer-oriented acquisitions like Linksys, Cisco has never been known for its low cost of goods or optimized manufacturing. After all, with the 65 percent margins it’s been able to maintain in its core networking business, it really hasn’t had to think about that stuff.
So, while I don’t see this as a profitable move for Cisco in the short term, it’s most certainly looking for a long-term payoff. After all, if there’s one place Cisco should be able to succeed in the server market, it’s with central IT, and the data center is a good place to start. This is where Cisco is strongest and expanding from a position of strength certainly makes sense.
Now suppose Cisco succeeds in the server market. What will that mean? Its gross sales should increase but most likely its profit margin will go down. Maybe this is Cisco’s tax shelter strategy for the next few years … make less money (which is probably not a bad idea based on the current administration’s apparent plans for redistributing the wealth). If Cisco can remain strong in their core, this is probably the best time for it to expand into the server market. But grabbing a chunk of this market will not be quick and it will not be easy; it will be a long slog.
The Empires Strike Back. Or Do They?
Was it a knee-jerk response by HP to reduce its prices on network equipment that competes with Cisco’s? Actually, this follows HPs encroachment on Cisco territory with the launch of the ProCurve line of switches. So this is not HP’s first shot at attempting to take share away from Cisco, and I don’t see this one succeeding either. It does have the potential, however, to chip away at Cisco’s profit margins as the two companies compete on price. And this illustrates the clash of approach: Cisco wants the battle to be fought on features; HP wants to bring everything back to cost.
So what about IBM, EMC and the other gorillas in the data center? I don’t see any real advantage for IBM or EMC to join this battle. IBM would prefer to continue its march toward applications and services. EMC will hunker down in its sweet spots of storage, information management and virtualization. EMC’s component approach to security (through the RSA division) will be unaffected by Cisco’s initiative as IT department generally avoid “all in” deployment strategies.
In fact, this is a huge hurdle for Cisco’s main challenger in high-end networking, Juniper Networks, which is betting the market will buy into a centralized approach to security. This is not a good bet.
If there’s one place where compartmentalization, independent systems and overlap makes sense, it’s in security. History has proven that no company can stay on top of the market across all security areas – firewalls, encryption, remote access, compliance, information security, etc. And even if one could, it would be a bad IT decision. Security professionals don’t want one system for the bad guys to defeat; having many is better. And being committed to a sole source supplier also virtually assures that prices eventually will increase for a captive customer with no way out. This is one big concern for CIOs with the emerging Juniper approach.
In the end, I’ll wait to see whether Cisco can be a game changer in the server space. I am, however, convinced that Cisco ultimately will develop unique differentiation and that it hasn’t played or revealed all its cards yet. Whether Cisco’s servers end up delivering the necessary advances in performance, TCO or leapfrog benefits to take market share is the unanswerable question. But regardless of how it all plays out, more competition will translate into better value for CIOs, their IT departments and their enterprises. And if that isn’t part of Cisco’s strategy, then it really isn’t worth bothering about.
Posted by Michael Bullock on Wed, Dec 17, 2008
I recently was interviewed by Francis Rose of WFED radio in Washington D.C. His show, “In Depth with Francis Rose,” focuses on topics of interest to federal IT leaders and professionals. (You can listen to the interview online.)
Now, Francis is not a technology guy so I was curious as to why he was interested in dedicating 20 minutes of his show to data center relocation. Why did he think that was newsworthy? Of course, folks in the know understand that aging facilities are forcing companies to consolidate and relocate data centers all over the country. But why did Francis care, and what did it have to do with the government?
I quickly saw where he was heading. Francis had read an article I wrote about the five pitfalls of data center relocation and how the State of Oregon managed (disastrously) to fall into every one. Francis wanted me to expand on how IT managers could avoid repeating Oregon’s mistakes.
You can read about all five pitfalls in the article, but I wanted to focus on one in particular because it’s a seductive trap that’s snaring a lot of organizations: thinking about the move as an opportunity to introduce new technologies (particularly virtualization) and methodologies. As I told Francis, we strongly advise against introducing any new moving parts during data center relocations. Doing is so like performing open heart surgery while throwing in a hip replacement on the side. It can be done, but it’s an enormous risk—and one not worth taking. Francis agreed; it seemed like a no-brainer to him.
So why do so many companies do it? They usually do so for two basic reasons:
1. Pent-up demand. When everything in the IT function (including the data center) is working smoothly, there aren’t a lot of CIOs willing to introduce infrastructure changes (or CFOs willing to fund them) unless the ROI is overwhelming. Especially in this economy, the guiding principle seems to be, if it ain’t broke, don’t fix it. Therefore, projects (including virtualization) tend to be pushed off even when they make business sense. Over time, this creates a project backlog—an itch that IT organizations want to scratch.
In the excitement of a data center relocation, the organization discovers a new enthusiasm for IT improvement and innovation. With all its resources allocated to the relocation, the organization convinces itself that the staff, energy, and budget is available to do anything and everything. In other words, it gets to scratch that backlog itch. Instead of “If it ain’t broke, don’t fix it,” the order of the day becomes, “Let’s take advantage of this focus on IT to fix everything!”
If you find yourself in a similar place, and the pressure to virtualize is mounting, at least make sure you have the time, money, staff, and expertise to do it right. If you’re not absolutely convinced, my advice is to finish the relocation first, satisfy yourself that your new data center is functioning optimally, and then think about virtualization.
2. Vendor hype. Let’s face it, most vendors know the best time to sell you anything new is when it aligns with changes that are already taking place. That’s the easiest and best time to capture your attention, not to mention your budget. Vendors specialize in taking advantage of any fears and uncertainties you may have to tell you that they have the tools and expertise to make them all go away. So if your virtualization software provider says virtualizing during a relocation is a good idea, and your hardware vendor says it’s a good idea, and the press and the analysts agree (as they often do as they spend a lot of time with and get a lot of their information from those same vendors), then it must be a good idea, right?
Well, not exactly. Performing two complex IT operations at the same time is almost always a recipe for disaster.
If you’re ready to make a move to a virtualized environment, you need to accept that fact that no matter what your vendors say, you will be beginning a complex, resource-intensive project. It’s important to be sure that all your systems will be able to support appropriate service levels throughout the transition. Unless those service levels can be maintained to the business’ satisfaction, it doesn’t matter how smoothly the virtualization initiative goes.
And if you’re also scheduled to relocate your data center at the same time, please, stop and reflect upon the risk you’ll be assuming by attempting to pull off two complex projects at the same time, essentially under a stop watch held by a watchful business.
Me, I wouldn’t want to do it.