Data center and other mission critical commissioning efforts require levels of rigor not normally applied to the average commercial facility. Bluerithm’s robust, cloud-based platform can help you stay organized and ensure every detail is addressed, and every problem is resolved!
Guide to Data Center Commissioning
Solution:
Data center commissioning is a process that ensures industrial-scale, complex facilities (that may require as much energy as a small town) are delivered according to the Owner’s Project Requirements (OPR). The functional requirements for these facilities are more extensive than average commercial facilities. We all rely on computing resources and expect 24/7 uninterrupted availability of internet services. Downtime at data centers can have disastrous financial implications (For example, if Amazon servers go down, they lose about $1,104 in sales for every second of downtime).
These data center facilities require design, construction, and commissioning processes that are more rigorous than what’s usually deployed in commercial construction. This guide focuses on the facilities themselves – the power, cooling, support systems, the physical structures that keep the environment for the computing infrastructure inside them running, and what it takes to implement a successful commissioning program for a data center.
Table of Contents
Functional Requirements for Data Centers
There are several functional requirements that are relevant to the scope of data center commissioning including:
- Power
- Cooling
- Location
- Control Systems
- Physical Security
- General
Power
Power is usually the largest cost in data center operations. The fault tolerance and resiliency needed to maintain uninterrupted power requires a multi-faceted approach that includes:
- Parallel electric utility sources
- Backup power sources, such as generators that run on a consumable fuel source that can be stored and delivered to the site
- Uninterruptable power supplies (UPS) through the use of batteries, flywheels, etc. to carry the load of the building during a transition from one power source to another
- System design as it relates to redundancy, fault tolerance, failover, and maintenance
Cooling
The computing resources in a data center generate a lot of heat. Keeping equipment at optimal temperatures ensures it works properly and doesn’t fail. Reliable and redundant cooling systems are another aspect of maintaining uptime and include the following considerations:
- Climate – The equipment and its configuration should be appropriate for the climate and take advantage of “free cooling” when available
- The operating ranges of the computing resources that are planned to be used
- The density (or lack thereof) of computing resources in server racks. This can determine whether a traditional row (hot/cold aisle) air distribution design, or a more direct rack distribution system may work best
- Server rack density as it relates to a need continuous cooling
- System design as it relates to redundancy, fault tolerance, failover, and maintenance
Location
Not every region is suitable for data centers. Commissioning requirements related to location are less relevant than those for primary power and mechanical systems, but like any system, it will have unique characteristics and performance based on its environment. The following considerations may be used when deciding where a data center will be built:
- Extreme weather patterns and geological risks
- Flight paths
- Climate – Data centers require large amounts of cooling, making some climates better than others for building data centers
- Availability, reliability, and cost of regional electric utilities – Many data centers have incoming power from multiple utilities.
- Availability, reliability, and cost of the backhaul network connections – Without reliable network connections to the facility, the computing resources inside the data center would be of no value to the systems that rely on them
- Relative location to other data centers – Many computing systems that data centers will support are critical enough to require geo-redundancy, where a failure in one data center will cause mirrored computing resources in another coordinated data center to take on the workload
- Proximity to users of the computing resources – Some systems require extremely low latency and the distance network traffic has to travel to end users can be a consideration
- Regulatory requirements – Some government agencies require data to be stored within certain geographical boundaries
- Political stability of the region
Control Systems
Control systems and automation are what bring equipment together and help them to operate in unison, and respond to events and actions from other equipment and systems, environmental conditions, or operators. The following items must be carefully coordinated among engineering disciplines and construction trades:
- Sequences of operation for all systems
- Control system software design practices and process
- Power monitoring system hardware and software
- Cooling systems control hardware and software
- Control system networking hardware and configuration
- Interconnectedness of control systems and components from different manufacturers – Open and common protocols don’t always guarantee proper interface – it’s still up to each manufacturer to implement protocol specifications correctly, and then up to the engineers and technicians from the installer to program and configure them properly
- Electrical power for control systems – The power sources for control system and networking hardware must be on uninterruptable power sources
- Availability and responsiveness of local support for control systems
- Competency of data center staff for the installed control systems
- A common interface for all control systems – This common interface doesn’t need to bring ALL data into one place, but it should have dashboards and key, critical operating data and alarms, with links and ways to quickly access each individual system when full reports and detailed data are required
Physical Security
Access to a data center must be controlled and limited to only qualified, authorized individuals. Depending on the systems running on the servers in a data center, these access restrictions may even be governed by laws and regulations by governments or their agencies.
General
There are several general considerations that will dictate all functional requirements including:
- Budget – Each incremental improvement in redundancy and reliability doesn’t come without higher initial investment, or potentially higher operational expenses
- Tier certification – The Uptime Institute publishes what’s considered the international standard for data center performance
- The four tiers of certification “match a particular business function and define criteria for maintenance, power, cooling and fault capabilities”
- Each tier will add reliability to the data center, but will also increase complexity and initial costs
Commissioning for Data Centers
Commissioning for highly resilient facilities like data centers may vary in the level of rigor applied compared to a commissioning program for any other facility. There are five levels of commissioning usually implemented within data center commissioning programs (these levels are not limited to data center commissioning, but are more commonly used in this context).
A few of the differences in data center commissioning include:
- Higher witness test rates, less sampling
- Multiple tests and multiple iterations of these tests – every potential scenario or condition must be tested
- Higher level of test complexity
- Witness testing of all operation, failure and maintenance scenarios
- Integrated Systems Test
The five levels cover parts of a commissioning program that mainly occur during what’s considered the construction phase of commissioning. It should be noted that a complete commissioning program includes design phase activities as well. Sometimes, the five level approach is used with a sixth level, which is usually identified as level zero, that covers design reviews, OPR development, and other early-stage engagement. What’s important is that the commissioning program spans all design, construction, and post-occupancy phases.
Level One Commissioning
Level one commissioning covers submittal reviews and factory witness testing. Factory witness testing involves mockups of equipment or systems in controlled environments, often at the manufacturer or vendor’s factory – hence the name. Testing in this environment makes it easier and less expensive to find and fix design and implementation issues with equipment and software. It also allows asynchronous testing of many different systems while the data center itself is still under construction, aiding in overall timeline efficiency.
Level Two Commissioning
Level two commissioning is site acceptance inspection. Equipment is inspected to ensure it complies with all design criteria, that it is not damaged, and that there is a proper storage plan in place.
Level Three Commissioning
Level three covers installation inspections, sometimes referred to as pre-functional testing. Installation inspections involve the inspection of the installation of all equipment. Both the contractors and commissioning agent will verify that all equipment is installed properly, and that installation meets design and operational standards. Equipment is started for the first time to check proper, independent operation. Testing is repeated after corrections are made to any equipment that fails testing.
Level Four Commissioning
Functional performance testing happens during level four commissioning. This often applies to either individual components or equipment, or tightly coupled components and equipment (i.e. a chiller, its pumps, and its cooling tower(s)). During this phase, each control loop is checked, actual operation is compared to designed sequences of operation, and performance is observed. Setpoint adjustments may be made as necessary. Operational issues are uncovered during this phase.
This level of commissioning begins to involve more people and firms, disciplines, and entities working together than other phases, at least with respect to testing.
Level Five Commissioning
This is the final phase of testing (aside from any seasonal testing that was not performed earlier), and usually focuses on building-wide response to major events, like suddenly disconnecting utility power. It’s at this point where the response from all systems – power, cooling, general HVAC, etc. must be proven to work together in unison and prevent any interruptions to the operation of the data center. In summary, level five commissioning demonstrates the performance of the facility as a whole against all design criteria. Systems are operated at various loads and in various modes to demonstrate proper response to equipment failures and utility problems.
See How Much Bluerithm Can Save You on Your Data Center Commissioning Projects Using These Calculators
Bluerithm Demo
Play Video about Bluerithm demo
How to Set Up a Bluerithm Project
Play Video about How to Set Up a Bluerithm Project
Move your Excel-based processes to the cloud
Start saving time and money today
