What Do AI Rack Density & Back To The Future Have In Common?

By Marc Cram

First, it is important to consider what data center rack density really is and how it is measured. Data center density is measured in the number of kilowatts consumed per cabinet. As computing needs increase, so too must the power draw per cabinet. Following the Association for Computer Operations Management (AFCOM) scale, data center density has a Low category of up to 4kW per rack, and in the Extreme category over 16kW per rack. As AI applications are deployed, the computational power needed starts to increase as well as power usage and heat generation. This places AI’s workload in the High category of AFCOM scale, i.e. 8kW to 15kW per rack.

Given these facts, it seems that Moore’s Law is clearly in play as the demands for higher compute and the need for additional power are creating denser racks. Of course, there are always exceptions that tend to keep Moore in check, and one of them is increasing the average computer per watt. A good example of this is NVIDIA’s DGX-1 server that can learn 140 times faster than typical CPU servers—the net result is fewer servers, less density and lower power needed for AI workloads. Furthermore, data center providers are now considering a change to bring 415V PDUs to the rack level to increase power efficiency and cut costs. Switching current requirements can cut the power losses that data centers experience every time electricity passes through a power converter or transformer.

MOVE-IN READY INFRASTRUCTURE FOR AI WORKLOADS

An unfortunate by-product of AI and machine learning is the fact that many server racks are being pushed to their density brink, and power costs have risen significantly over the years. Colocation providers are feeling the pressure as well. According to Mike Killory, Associate Principal Engineer at Southland Engineering, the typical power density in the colocation provider’s data hall has risen from 150W per square foot to 250 – 300W per square foot, with some providers going up to 400W. Note that “power in watts per square foot” measurement has been switched to power density measured in “kilowatts per rack” due to the advent of high-density environments.

Although there are many facility and IT challenges for colocation providers, rack density, heat, and power monitoring are among several of the top issues to contend with. However, there are some examples of providers overcoming density challenges presented by AI workloads.