Line 14: | Line 14: | ||
* UltraMemory - A firm that designs [[ThruChip Interface|TCI]]-based [[DRAM]] | * UltraMemory - A firm that designs [[ThruChip Interface|TCI]]-based [[DRAM]] | ||
+ | == Systems == | ||
=== ZettaScaler-1.x === | === ZettaScaler-1.x === | ||
'''ZettaScaler-1.x''' ('''ZS-1.x''') is based on the {{pezy|PEZY-SC}}. Each node (brick) was constructed using two {{intel|Xeon E5}} connected using [[Intel]]'s {{intel|QPI}} along with 4 set of dual PEZY-SC chips in PCIe card slots using InfiniBand FDR HCA. The use of the general-purpose Intel chips was to reduce cost and speedup development cycle. The original ZS-1.0 used an off-the-shelf SuperMicro board in order to speed-up the development cycle. Starting with the ZS-1.4, PEZY moved to a custom "brick". | '''ZettaScaler-1.x''' ('''ZS-1.x''') is based on the {{pezy|PEZY-SC}}. Each node (brick) was constructed using two {{intel|Xeon E5}} connected using [[Intel]]'s {{intel|QPI}} along with 4 set of dual PEZY-SC chips in PCIe card slots using InfiniBand FDR HCA. The use of the general-purpose Intel chips was to reduce cost and speedup development cycle. The original ZS-1.0 used an off-the-shelf SuperMicro board in order to speed-up the development cycle. Starting with the ZS-1.4, PEZY moved to a custom "brick". |
Latest revision as of 19:39, 13 January 2018
ZettaScaler is a series of Japanese supercomputers using processors designed by PEZY and liquid cooling systems designed by ExaScaler.
Contents
Overview[edit]
ZettaScaler supercomputers are constructed using dense server aggregates called 'Bricks'. The system is cooled using enclosures containing a number of bricks. The cooling system is designed by ExaScaler and makes use of 3M's Fluorinert Electronic liquid which is an electrically insulating fluorocarbon-based inert liquid.
ZettaScalers use a host CPU (typically a Xeon E5) which is then used to offload work to the PEZY-SCx processors which act as GPDSP/GPGPU accelerators using OpenCL-like programming called PZCL.
Collaboration[edit]
The systems are a joint collaboration between three related start-ups:
- PEZY - Microprocessor design firm that designs the PEZY-SCx series.
- Exascaler - The firm that designs the water cooling solutions
- UltraMemory - A firm that designs TCI-based DRAM
Systems[edit]
ZettaScaler-1.x[edit]
ZettaScaler-1.x (ZS-1.x) is based on the PEZY-SC. Each node (brick) was constructed using two Xeon E5 connected using Intel's QPI along with 4 set of dual PEZY-SC chips in PCIe card slots using InfiniBand FDR HCA. The use of the general-purpose Intel chips was to reduce cost and speedup development cycle. The original ZS-1.0 used an off-the-shelf SuperMicro board in order to speed-up the development cycle. Starting with the ZS-1.4, PEZY moved to a custom "brick".
With the introduction of the ZS-1.6, PEZY redesigned the package of the PEZY-SC. The new chip, PEZY-SCnp (NP for New Package), addressed a number of issues relating to signal quality (DRAM, and PCIe frequency failure).
System | ZettaScaler-1.0 | ZettaScaler-1.4 | ZettaScaler-1.5 | ZettaScaler-1.6 |
---|---|---|---|---|
Introduction | October 2014 | April 2015 | October 2015 | April 2016 |
Board | SuperMicro Board | Brick | ||
Host | 2x Xeon E5-2660 v2 | 2x Xeon E5-2618L v3 | ||
Chip | PEZY-SC | PEZY-SCnp | ||
Memory | DDR3 256 GB | DDR4 64 GB | DDR4 128 GB | |
TOP500 | Suiren: 187 TFLOPS (Rank 369) 2014/11 206 TFLOPS (Rank 366) 2015/06 |
Shoubu: 412 TFLOPS (Rank 169) 2015/06 SuirenBlue: 194 TFLOPS (Rank 392) 2015/06 |
Shoubu: 1,001 TFLOPS (Rank 94) 2016/06 Satsuki: 290 TFLOPS (Rank 486) 2016/06 | |
Green500 | Suiren: 4.95 GFLOPS/W (Rank 2) 2014/11 6.22 GFLOPS/W (Rank 3) |
Shoubu: 7.03 GFLOPS/W (Rank 1) 2015/06 SuirenBlue: 6.84 GFLOPS/W (Rank 2) 2015/06 |
Shoubu: 6.67 GFLOPS/W (Rank 1) 2016/06 Satsuki 6.20 GFLOPS/W (Rank 2) 2016/06 |
ZettaScaler-2.x[edit]
With the introduction of the ZettaScaler-2.x, PEZY has redesigned the Brick. PEZY moved to higher integration through advanced 3D packaging technology using ThruChip Interface (TCI) to interconnect the DRAM to the processor which uses wireless near-field inductive coupling instead of the traditional TSV. Powering the ZS-2.x computers is the PEZY-SC2 which incorporates 2,048 cores.
On the ZS-2.x, a single module contains a beefier 48V power supply. The module is measured 160mm (W) x 100mm (D) x 28 mm (H). Each brick contains 32 modules.
16 such bricks go into a single tank.
ZettaScaler-2.2[edit]
The ZettaScaler-2.2 (ZS-2.2) architecture was introduced to the Top500 list in November 2017 with multiple systems being upgraded.
The highest-performing system was the Gyoukou supercomputer which officially peaked at an efficiency of 14.173 GFLOPS/W, surpassing the previous record of 14.11 held by the TSUBAME 3.0. The systems features 10,000 PEZY-SC2 chips, however in order to improve yield they each have 1,984 cores enabled instead of the expected 2,048 for a total of 19,840,000 cores. In the ZS-2.2, the SC2 chips are clocked at 700 MHz instead of their announced frequency of 1 GHz. For every eight SC2 chips, there is a single 16-core Xeon D processor for a combined total of 1250 Xeon processors and 20,000 cores. The Gyoukou ZS-2.2 has a Linpack performance of 19,135.8 TFLOPS with a theoretical peak performance of 28.192.0 PFLOPS consuming 1,350 kW. With 7,056 PEZY-SC2 chips, only 19.53 immersion tanks are filled out of the 26 tanks available, leaving a lot of room to grow. Additionally, with 19,860,000 cores (19,840,000 PEZY + 20,000 x86), the ZettaScaler-2.2 surpassed the Chinese supercomputer Sunway TaihuLight as the highest core-count supercomputer by over 3 million cores.
In addition to the Gyoukou supercomputer, three other smaller Japanese supercomputers based on the ZettaScaler-2.2 were introduced - all three beating the Gyoukou in efficiency, reaching the top three spot on the Green500 list:
Top500/Green500 November 2017 Ranking | ||||||
---|---|---|---|---|---|---|
Green500 Rank |
Top500 Rank |
System | Cores | Rmax | Power | Efficiency |
1 | 259 | Shoubu system B | 794,400 | 842.0 TFLOPS | 50 kW | 17.009 GFLOPS/W |
50x (8x PEZY-SC2 + 1x Xeon D-1571) | ||||||
2 | 307 | Suiren2 | 762,624 | 788.2 TFLOPS | 47 kW | 16.759 GFLOPS/W |
48x (8x PEZY-SC2 + 1x Xeon D-1571) | ||||||
3 | 276 | Sakura | 794,400 | 824.7 TFLOPS | 50 kW | 16.657 GFLOPS/W |
50x (8x PEZY-SC2 + 2x Xeon E5-2618L v3) | ||||||
5 | 4 | Gyoukou | 19,860,000 | 19,135.8 TFLOPS | 1,350 kW | 14.173 GFLOPS/W |
1250x (8x PEZY-SC2 + 1x Xeon D-1571) |
References[edit]
- Some information was obtained directly from PEZY
- 3M Fluorinert Helps PEZY/ExaScaler Rise to the Top of the Green500 List