|Introduction|| May 23, 2018 (announced)|
|Process|| 28 nm|
0.028 μm, 16 nm
0.016 μm, 10 nm
|Package||PCIe x16 Gen 3 Card, OCP OAM, M.2|
Neural network processors (NNP) is a family of neural processors designed by Intel for the acceleration of artificial intelligence workloads. The name and original architecture originated with the Nervana startup prior to its acquisition by Intel in 2016. Although the first product was announced in 2017, it never made it past customer sampling which eventually served as a learning product. Intel eventually productized those chips starting with their second-generation designs in late 2019.
The NNP family comprises two separate series - NNP-I for inference and NNP-T for training. The two series use entirely different architectures. The training chip is a direct descendent of Nervana's original ASIC design. Those chips use the PCIe and OAM form factors that have high TDPs designed for maximum performance at the data center and for workstations. Unlike the NNP-T, NNP-I inference chips are the product of Intel IDC which, architecturally, are very different from the training chips. They use Intel's low-power client SoC has the base SoC and build the AI architecture from there. The inference chips use low-power PCIe, M.2, and ruler form factors designed for servers, workstations, and embedded applications.
On January 31, 2020, Intel announced that it has discontinued the Nervana NNP product line in favor of the unified architecture it has acquired from Habana Labs a month earlier.
|20171||Training||Lake Crest||28 nm|
|2019||Training||Spring Crest||16 nm|
|2019||Inference||Spring Hill||10 nm|
1 - Only sampled
- Main article: Lake Crest µarch
The first generation of NNPs were based on the Lake Crest microarchitecture. Manufactured on TSMC's 28 nm process, those chips were never productized. Samples were used for customer feedback and the design mostly served as a software development vehicle for their follow-up design.
T-1000 Series (Spring Crest)
- Main article: Spring Crest µarch
Launched in late 2019, second-generation NNP-Ts are branded as the NNP T-1000 series and are the first chips to be productized. Fabricated TSMC's 16 nm process based on the Spring Crest microarchitecture, those chips feature a number of enhancements and refinments over the prior generation including a shift from Flexpoint to Bfloat16 and considerable performance uplift. Intel claims that these chips have about 3-4x the training performance of first generation. All NNP-T 1000 chips come with 32 GiB of four HBM2 stacks in a CoWoS package and come in two form factors: PCIe Gen 3 and an OCP OAM accelerator card.
- Proc 16 nm process
- Mem 32 GiB, HBM2-2400
- TDP 300-400 W (150-250 W typical power)
- Perf 108 TOPS (bfloat16)
|List of NNP-T 1000-based Processors|
|Model||Launched||TDP||EUs||Frequency||HBM2||Peak Perf (bfloat16)|
|NNP-T 1300||12 November 2019||300 W|
|NNP-T 1400||12 November 2019||375 W|
POD Reference Design
Along with the launch of the NNP-T 1000 series, Intel also introduced the POD reference design. Those systems were intended for large-scale out systems for the processing of very large neural networks. The POD reference design featured 10 racks with 6 nodes per rack. Each of the nodes features eight interconnected OAM cards, producing a system with a total of 480 NNP-Ts.
I-1000 Series (Spring Hill)
- Main article: Spring Hill µarch
The NNP I-1000 series is Intel's first series of devices designed specifically for the acceleration of inference workloads. Fabricated on Intel's 10 nm process, these chips are based on Spring Hill and incorporate a Sunny Cove core along with twelve specialized inference acceleration engines. The overall SoC design borrows considerable amount of IP from Ice Lake. Those devices come in M.2 and PCIe form factors.
- Proc 10 nm process
- Mem 4x32b LPDDR4x-4200
- TDP 10-50 W
- Eff 2.0-4.8 TOPs/W
- Perf 48-92 TOPS (Int8)
|List of NNP-I-1000-based Processors|
|Model||Launched||TDP||EUs||Peak Perf (Int8)|
|NNP-I 1100||12 November 2019||12 W|
|NNP-I 1300||12 November 2019||75 W|
Intel also announced NNP-I in an EDSFF (ruler) form factor which was designed to provide the highest compute density possible for inference. Intel hasn't announced specific models. The rulers were planned t come with a 10-35W TDP range. 32 NNP-Is in a ruler form factor can be packed in a single 1U rack.
|first announced||May 23, 2018 +|
|first launched||2019 +|
|full page name||nervana/nnp +|
|instance of||integrated circuit family +|
|main designer||Intel +|
|manufacturer||Intel + and TSMC +|
|package||PCIe x16 Gen 3 Card +, OCP OAM + and M.2 +|
|process||28 nm (0.028 μm, 2.8e-5 mm) +, 16 nm (0.016 μm, 1.6e-5 mm) + and 10 nm (0.01 μm, 1.0e-5 mm) +|