From WikiChip
Editing habana/microarchitectures/gaudi
Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.
The edit can be undone.
Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.
This page supports semantic in-text annotations (e.g. "[[Is specified as::World Heritage Site]]") to build structured and queryable content provided by Semantic MediaWiki. For a comprehensive description on how to use annotations or the #ask parser function, please have a look at the getting started, in-text annotation, or inline queries help pages.
Latest revision | Your text | ||
Line 19: | Line 19: | ||
==Architecture == | ==Architecture == | ||
=== Block Diagram === | === Block Diagram === | ||
− | :[[File:habana gaudi block diagram.svg| | + | :[[File:habana gaudi block diagram.svg|500px]] |
== Overview == | == Overview == | ||
Gaudi was designed as a microarchitecture for the [[acceleration]] of training in the data center. It is offered as a PCIe-based [[accelerator card]] or, alternatively, as an [[OCP]] [[Open Accelerator Module]] (OAM). | Gaudi was designed as a microarchitecture for the [[acceleration]] of training in the data center. It is offered as a PCIe-based [[accelerator card]] or, alternatively, as an [[OCP]] [[Open Accelerator Module]] (OAM). | ||
− | The design itself is based on the company's first inference chip, {{\\|Goya}}, but adds additional components to | + | The design itself is based on the company's first inference chip, {{\\|Goya}}, but adds additional components to facilitates efficient scale-out capabilities. To that end, Gaudi features eight Tensor Processing Cores (TPCs), a General Matrix Multiply (GMM) engine, and a large pool of shared memory. In order to facilitate large scale-out capabilities, Gaudi integrates a large set of ethernet ports and [[high-bandwidth memory]]. |
− | |||
− | |||
− | |||
=== Tensor Processing Cores (TPC) === | === Tensor Processing Cores (TPC) === | ||
− | There are eight TPCs integrated on Gaudi, each with its own local memory and without caches | + | There are eight TPCs integrated on Gaudi, each with its own local memory and without caches. The on-die caches and memory can be either hardware-managed or fully software-managed, allowing the compiler to optimize the residency of data and reducing movement. Each of the individual TPCs is a VLIW DSP design that has been optimized for AI applications. This includes AI-specific instructions and operations. The design itself is actually an enhanced version of the TPCs found in the company's prior inference accelerator design, {{\\|Goya}}. |
− | The TPC supports mixed- | + | The TPC supports mixed-prevision operations including 8-bit, 16-bit, and 32-bit SIMD vector operations for both integer and floating-point. This was done in order to allow accuracy loss tolerance to be controlled on a per-model design by the programmer. Goya offers both coarse-grained precision control and fine-grained down to the tensor level. Compared to {{\\|Goya}}, the TPC in Gaudi also adds supports for [[bfloat16]] and adds additional operations and functionality more desired in training. |
=== High-Bandwidth Memory (HBM2) === | === High-Bandwidth Memory (HBM2) === | ||
− | {{ | + | {{also|CoWoS|High-Bandwidth Memory}} |
Gaudi is fabricated on [[TSMC]] [[16 nm process]] (16FF+) and utilizes its [[2.5D]] {{tsmc|CoWoS}} interposer technology in order to integrate four stacks of [[HBM2]] memory. Each stack has 8 GiB in capacity and operates at a [[signaling rate]] of 2 GT/s for a total bandwidth of 1 TiB/s. | Gaudi is fabricated on [[TSMC]] [[16 nm process]] (16FF+) and utilizes its [[2.5D]] {{tsmc|CoWoS}} interposer technology in order to integrate four stacks of [[HBM2]] memory. Each stack has 8 GiB in capacity and operates at a [[signaling rate]] of 2 GT/s for a total bandwidth of 1 TiB/s. | ||
Line 42: | Line 39: | ||
From a programmability point of view, Gaudi supports parameters, tensor, and sub-tensor transfers over Ethernet. For [[quality of service]], there are hardware hooks for supporting congestion control and congestion avoidance. Additionally, the fabric has both lossless and lossy support. | From a programmability point of view, Gaudi supports parameters, tensor, and sub-tensor transfers over Ethernet. For [[quality of service]], there are hardware hooks for supporting congestion control and congestion avoidance. Additionally, the fabric has both lossless and lossy support. | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
== See also == | == See also == | ||
* {{\\|Goya}} | * {{\\|Goya}} | ||
* {{habana|HL}} series | * {{habana|HL}} series |
Facts about "Gaudi - Microarchitectures - Habana"
codename | Gaudi + |
designer | Habana + |
first launched | 2019 + |
full page name | habana/microarchitectures/gaudi + |
instance of | microarchitecture + |
manufacturer | TSMC + |
name | Gaudi + |
process | 16 nm (0.016 μm, 1.6e-5 mm) + |
processing element count | 8 + |