Jülich Supercomputing Centre Receives GreenICE System From EXTOLL
March 3 — Within the scope of the EU-funded DEEP project, a highly innovative immersion cooling system for compute hardware was installed at Jülich Supercomputing Centre: EXTOLL’s GreenICE. The machine complements the DEEP prototype with a second Booster system testing the EXTOLL ASIC TOURMALET. The latter allows for running a node without a host CPU. Thus, a GreenICE node consists of an Intel XEON Phi 7120D (Knights Corner: KNC) coprocessor and an EXTOLL NIC. This enables world-leading compute-power density integrating 32 compute nodes within a 19” wide and 10U high basin and yielding a compute power of 38 TFLOP/s peak performance. A 2-phase liquid cooling with a secondary hot-water cooling circuit is able to take away thermal power of up to 12 kW without the need for moving parts or active external cooling. A Power usage efficiency of PUE=1.01 is game changing. For even more compute power, up to 4 GreenICE systems could be stacked within a standard 19” rack resulting in 150 TFLOP/s.
The road to Exascale Computing demands an increase in compute power while at the same time reducing volume and energy consumption. Hence, systems with high power density, low energy consumption and low cooling effort are needed.
GreenICE System
Recently, EXTOLL released its immersion cooled system GreenICE. Within a basin of 19” x 10U power supplies, compute electronics and cooling is arranged in 3 separate levels: Power supplies are placed at the lowest level, compute electronics fill the mid level and cooling devices are placed in the top level.
Both the lower and the mid level are filled with 3M’s NOVEC-649 fluid having its boiling point at 49°C. Hence, compute electronics as well as power supplies are completely immersed.
In lieu of a tightly integrated Booster board, a passive PCI Express backplane connects eight Intel Xeon Phi 7120D cards and eight EXTOLL TOURMALET NICs in a pairwise manner. Thanks to EXTOLL technology the Booster nodes can be scaled independently from the scalar cluster nodes.
To ease integration, four assembled backplanes are completely immersed in NOVEC. In operation, the heat produced by these components evaporates the NOVEC fluid. The NOVEC vapor is then cooled down by loops of specially tailored copper pipes (with maximized surface), with water as cooling liquid. The condensed vapor drops back into the basin.
System management is performed by a Raspberry PI system via I2C connections to the Backplane and power supplies. The EXTOLL links are carried via copper cables that attach to the NICs using standard HDI6 connectors.
Within the DEEP project this GreenICE system serves as an evaluator for exploring the suitability of EXTOLL’s network chip TOURMALET for the realisation of the DEEP Cluster-Booster architecture. This GreenICE booster is attached to the FPGA-based Booster of the DEEP project.
The network topology of the GreenICE booster is designed as a 3D torus of 32 nodes in a 4x4x2 logical arrangement.
The DEEP project will be present at this year’s CeBIT and showcase amongst others a small version of the GreenICE Booster. Find the team at ‘Innovationsland Nordrhein-Westfalen’ booth C30 in hall 6 from Monday, March 14, to Friday, March 16, 2016.
About EXTOLL GmbH
EXTOLL GmbH is a Mannheim, Germany, based privately held company dedicated to high-performance computing (HPC). Its core product is an HPC networking solution including in-house designed ASICs, PCIe Boards, cabling solutions and software stack. EXTOLL also offers design IP related to its core products. Additionally, EXTOLL GmbH provides extremely efficient and dense 2-phase immersion cooling solutions.
—
Source: EXTOLL
