Improving the Thermal Design of Open Compute's Intel-Based Servers

30 May, 2018


When the University of Texas at Arlington (UTA) were looking to improve the thermal design of the Open Compute Project’s Intel Based servers, they used thermal simulation to help them find a solution. UTA used the 6SigmaET software throughout their project to create, simulate and fine-tune their proposed solutions to the server’s thermal design issues.

Introduction

Two different methods were chosen to improve the server’s thermal design: one improved the ducting inside the server, while the other utilised warm water cooling.


Solution 1: Improved Ducting

*please download case study for details (below)*


Solution 2: Liquid Cooling

The university then investigated whether a liquid-cooled system would improve the server’s thermal performance. To achieve a completely liquid-cooled system, the aircooled heat sinks were replaced with liquid-cooled cold plates, and the system was sealed from the ambient air to prevent gaseous and component contamination. A heat exchanger was incorporated to cool the air inside the server, as the remaining components are not directly liquid cooled. A combination of warm water and recirculated air would be used to cool the server. This solution required custom ducting to direct the recirculated air over the server’s DIMMs, PCH, HDD and other heat generating components.

A range of candidate ducts were designed, with the goal of ensuring adequate airflow through the system and keeping component temperatures within the server’s critical limits. These designs were modelled and simulated in 6SigmaET, and the results used to determine the optimum duct design

A prototype was created from this design, which was then subjected to thermal testing. It was tested with water inlet temperatures from 27.5-45°C in increments of 2.5°C. The server was exercised computationally at idle, 40%, 60%, 80% and 100% CPU loading, and one test was performed with maximum CPU and memory power levels to provide continuous heat dissipation. 

The thermal testing results showed a correlation between the server’s performance and the increased water inlet temperatures. The server’s cooling power consumption, radiator fan speeds, CPU temperatures and IT power consumption increased as the water temperature increased. However, the CPU temperatures remained below the critical die temperature of 80°C throughout the experiment. 

The prototyped duct regulated the air flow through the DIMMs, HDD and other auxiliary components as expected, maintaining component temperatures below critical. The university concluded that there was ample incentive to operate at higher water temperatures, up to 45°C.


Conclusions

Both projects used thermal simulation to analyse airflow and temperatures in the original server and determine where improvements could be made. This allowed the university to create and test a range of designs, iteratively improve them, and determine which performed best. The design with the best results was then prototyped and physically tested, with good agreement between the CFD model and the experimental results. Thermal simulation using 6SigmaET reduced the time and cost of the design stage, and meant that a wide range of potential solutions could be investigated.

 

To download the full case study, please fill out the form below.