Overview
Recognized by the U.S. Environmental Protection Agency and other energy organizations for reducing its carbon footprint, the accounting firm looks to all parts of its business to support its energy-reduction efforts — including its data centers.
Its traditional data center that supports all the applications used by the firm’s professionals in the United States generates almost half of its own electricity on site using a low-emission, natural-gas power generation system. Instead of venting into the environment, the generator’s exhaust heat is distributed to chillers — providing continuously chilled water to the air conditioning system that maintains optimal temperatures for the IT equipment. Solar power is even used to provide electricity to the firm’s headquarters campus that houses the data center and company offices.
Energy efficiency steps inside the data center include virtualizing servers, and eliminating hotspots, over cooling and other energy inefficiencies. The firm also uses perforated tiles and blanking panels to prevent hot air from creeping into cabinets or flowing into cold aisles.
With business growing — and the completion recently of several company acquisitions — the firm made the decision to expand its data center operations to a Tier III colocation (colo) facility. The data center team took the opportunity to exam its cabinet configurations and power distribution strategy, and to introduce new efficiency tools to both data centers.
Challenges
The accounting firm’s new colo site will serve dual roles — both production and disaster recovery. Critical applications from the legacy data center will be replicated in the colo site. The goal over time is to have equal production in both data centers.
In the same timeframe as the move to the colo data center, the firm had been migrating towards blade servers and virtualization — reducing the compute per square footage of its physical real estate while increasing the power density of its data centers. Higher voltage power would be required in the denser-configured cabinets filled with blade computers. Older low-density 2 to 4 kW cabinets were being replaced with 6 to 8 kW cabinets filled with blade servers tightly packed in each cabinet. The move to virtualization freed up substantial cabinet space in the data center at headquarters, but power capacity constraints were a concern.
Whereas in the new colo there was plenty of floor space and power capacity for future expansions — almost a megawatt of power for the 5,000-square-foot space. However, managing power consumption and cooling were a concern in the new site. Another challenge was determining power distribution and capacity to support the different cabinet configurations that would be deployed at the colo site.
Designed to streamline and speed infrastructure deployments, the firm created standard cabinet configurations for different applications. One cabinet configuration, for example, consists of a Cisco UCS chassis, primary flash storage and backup storage. Another cabinet configuration could have three to six UCS chassis with primary flash storage and backup storage.
The firm also wanted a single-pane-of-glass visibility of its power chain and data center environment for planning purposes and to proactively eliminate potential problems.
Solution — Intelligent Power Chain & Metering
The firm was looking for power distribution and monitoring solutions for its new colo data center.
Historically the firm had opted for basic power distribution units at the rack and a branch circuit monitoring solution to measure power up to the remote power panel. But the solution in place doesn’t provide details on energy usage and capacity — such as where you may be oversubscribed.
For the new colo site, the firm wanted finer metering granularity of power downstream — at the cabinet and device level. The firm’s IT team conducted a bakeoff of different vendors’ rack PDU products. Raritan’s PDU was selected for its ability to provide accurate outlet-level power information and for the smart temperature and humidity sensors that plug into the PDU.
The firm purchased Raritan’s 50 amp 3-phase intelligent rack PDUs. With three-phase power distribution directly to the rack the firm could effectively double rack density without doubling the cost. Three-phase PDUs are also more energy efficient as it eliminates energy losses with each voltage step down.
The real-time information gathered from Raritan PDUs is analyzed by the Power IQ DCIM Monitoring Software. In addition to monitoring power capacity and energy consumption, Power IQ provides data center health maps, power analytics, cooling charts, and alerts of potential trouble.
“The ability to be able to visualize and see what your infrastructure is doing helps when somebody says, ‘Hey, we’re going to deploy 12 more cabinets in the colo site. Can you handle it?’ You can do a quick ‘what if’ analysis using accurate information — instead of nameplate information from server manufacturers — to make decisions on our infrastructure, says the accounting firm’s director of data center operations.
“Nameplate information may be fine in some circumstances. But if you go with a full-width blade, a large number of cores, and a lot of memory, you’re going to start pulling power. You will want to know what is really happening under the load. So, outlet-level metering tools are very helpful,” he added.
Benchmarking
Power draw from cabinet cells varies; even among the same cell configuration. “There are so many variables, including what applications are running, what CPU is being used,” says the director. “Having the Power IQ information is very useful because we can say with some certainty that these HP storage arrays on a regular basis are consuming three kW, not one as we originally had thought.”
Insights gleaned from Power IQ on the various cabinet configurations in the new colo site are also helping the legacy data center. “If we put six UCSs in one cabinet in the colo and the cabinet consumes X amount of kilowatts, we can use the data point to help with deployments at headquarters,” adds the director.
“We generate weekly reports on power consumption and track where we are on that power curve. For the first time in many years we’ve seen power consumption go down,” says the director. “The results are from the combination of using more efficient computers, efficient power distribution, and real-time monitoring that keeps us informed on when there may be issues and how to improve efficiencies.”
To safeguard IT equipment, the colo data center also was outfitted with sensors to manage air flow, temperature, and humidity. Temperature sensors have been placed to measure the return air temperature coming into the tops of the cabinets. By monitoring temperature near CRAC units, the sensors showed that the firm didn’t have to run all the CRACs — saving cooling cost. The sensors also help the firm to eliminate hotspots.
As more applications are deployed in the colo, the firm’s IT team keeps an eye on infrastructure constraints — such as sufficient switch port capacity, network connections to support a new cell, backup connections. To conserve ports and IP addresses — and reduce costs, the IT team uses the cascading features of Raritan PDUs. Eight PDUs are cascaded using one single IP address.
The firm uses a number of the reports generated by Power IQ, including the power capacity in cabinets. “We can easily see what needs our attention — yellow needs attention, red means reaching a critical threshold. We also use the reports with management to demonstrate why we should update a cabinet from 30 amps to 50 amps,” says the director.
KVM and Serial Access over IP
A long-time Raritan KVM user, the firm started over a decade ago to use Raritan’s KVM switches for out-of-band access to servers. Over time analog switches were migrated to Raritan’s digital KVM-over-IP switches at the headquarters data center.
The new colo data center has added Raritan’s Dominion KX III KVM-over-IP switches and Dominion SX serial solutions to remotely access and manage IT equipment.
In turn, all the IT equipment connected to Dominion KX and SX solutions at both data centers can be controlled through a single portal using Raritan’s CommandCenter Secure Gateway. It provides logical views of the entire enterprise, as well as BIOS-level access to all servers via a browser. Through a single sign-on to a single IP-address, the accounting firm has real-time views of all physical and virtual servers in the headquarters and colo data centers.
Summary
“We now have a single pane of glass monitoring platform to manage our distributed data centers. Raritan solutions help us visualize our data center infrastructure so that we can manage and meet the demands of our business. We can deliver reliability by understanding our power capacity at any given time no matter what the computing configuration. It’s like having a gas gauge in a car. There’s no excuse for running out of gas. We have visibility in real time to power usage and infrastructure performance.”
