Latest revision |
Your text |
Line 9: |
Line 9: |
| | | |
| | | |
− | NVLink is designed to replace the inter-GPU-GPU communication from going over the PCIe lanes. It's worth noting that NVLink was also designed for CPU-GPU communication with higher bandwidth than PCIe. Although it's unlikely that NVLink would be implemented on an x86 system by either [[AMD]] or [[Intel]], [[IBM]] has collaborated with Nvidia to support NVLink on their [[POWER]] microprocessors. For supported microprocessors, the NVLink can eliminate PCIe entirely for all links. | + | NVLink is designed to replace the inter-GPU-GPU communication from going over the PCIe lanes. It's worth noting that NVLink was also designed for CPU-GPU communication with higher bandwidth than PCIe. Although it's unlikely that NVLink would be implemented on an x86 system by either [[AMD]] or [[Intel]], [[IBM]] has collaborate with Nvidia to support NVLink on their [[POWER]] microprocessors. For support microprocessors, the NVLink can eliminate PCIe entirely for all links. |
| | | |
| | | |
Line 15: |
Line 15: |
| | | |
| === Links === | | === Links === |
− | An NVLink channel is called a '''Brick''' (or an ''NVLink Brick''). A single NVLink is a bidirectional interface which comprises 8 differential pairs in each direction for a total of 32 wires. The pairs are DC coupled an use an 85Ω differential termination with an embedded clock. To ease routing, NVLink supports [[lane reversal]] and [[lane polarity]], meaning the physical lane ordering and their polarity between the two devices may be reversed.
| + | A single NVLink is a bidirectional interface which comprises 8 differential pairs in each direction for a total of 32 wires. The pairs are DC coupled an use an 85Ω differential termination with an embedded clock. To ease routing, NVLink supports [[lane reversal]] and [[lane polarity]], meaning the physical lane ordering and their polarity between the two devices may be reversed. |
| | | |
| | | |
Line 22: |
Line 22: |
| | | |
| ==== Packet ==== | | ==== Packet ==== |
− | A single NVLink [[packet]] ranges from a one to eighteen [[flits]]. Each flit is 128-bit, allowing for the transfer of 256 bytes using a single header flit and 16 payload flits for a peak efficiency of 94.12% and 64 bytes using a single header flit and 4 data payload flits for an efficiency of 80% unidirectional. In bidirectional traffic, this is slightly reduced to 88.9% and 66.7% respectively.
| + | {{empty section}} |
− | | |
− | A packet comprises of at least a header, and optionally, an address extension (AE) flit, a byte enable (BE) flit, and up to 16 data payload flits. A typical transaction has at least request and response with posted operations not necessitating a response.
| |
− | | |
− | :[[File:nvlink packet.svg|600px]]
| |
− | ==== Header flit ====
| |
− | The header flit is 128-bit. It comprises a 25-bit CRC field (discussed below), 83-bit transaction field, and a 20-bit data link (DL) layer field. The transaction field includes the request type, address, flow control bits, and tag identifier. The data link field includes things such as the packet length, application number tag, and acknowledge identifier.
| |
− | | |
− | The address extension (AE) flit is reserved for fairly static bits and is usually transmitted only bits change.
| |
− | ==== Error Correction ====
| |
− | Nvidia specifies the error rate at 1 in 1×10<sup>12</sup>. Error detection is done through the 25-bit [[cyclic redundancy check]] header field. The receiver is responsible for keeping the data in a [[replay buffer]]. Transmitted packets are sequenced and a positive acknowledgment is sent back to the source upon a good CRC. A missing acknowledgment following a timeout will initiate a reply sequence, retransmitting all subsequent packets.
| |
− | | |
− | The CRC field consists of 25 bits, allowing up to 5 random bits in error for the largest packet or alternatively, for differential pair bursts it can support up to 25 sequential bit errors. CRC is actually calculated over the header and the previous payload, eliminating the need for a separate CRC field for the data payload. Note that since the header also incorporates the packet length, it is also gets included in the CRC check.
| |
− | | |
− | For example, consider a sequence of two data payload (32 bytes) flits and their accompanying header. The next packet will have the CRC done over the current header as well as the two data payload from the prior transaction. If this is the first transaction, the CRC assumes the prior transaction was a [[NULL]] transaction.
| |
− | | |
− | === Data Rates ===
| |
− | <table class="wikitable">
| |
− | <tr><th> </th><th>[[#NVLink 1.0|NVLink 1.0]]</th><th>[[#NVLink 2.0|NVLink 2.0]]</th><th> [[#NVLink 3.0|NVlink 3.0]]</th><th> [[#NVLink 4.0|NVlink 4.0]]</th></tr>
| |
− | <tr><th>Signaling Rate</th><td>20 GT/s</td><td>25 GT/s</td><td>50 GT/s</td><td>100 GT/s</td></tr>
| |
− | <tr><th>Lanes/Link</th><td>8</td><td>8</td><td>4</td><td>2</td></tr>
| |
− | <tr><th>Rate/Link</th><td>20 GB/s</td><td>25 GB/s</td><td>25 GB/s</td><td>25 GB/s</td></tr>
| |
− | <tr><th>BiDir BW/Link</th><td>40 GB/s</td><td>50 GB/s</td><td> 50 GB/s</td><td> 50 GB/s</td></tr>
| |
− | <tr><th>Links/Chip</th><td>4 (P100)</td><td>6 (V100)</td><td> 12 (A100)</td><td> 18 (H100)</td></tr>
| |
− | <tr><th>BiDir BW/Chip</th><td>160 GB/s (P100)</td><td>300 GB/s (V100)</td><td>600 GB/s (A100)</td><td>900 GB/s (H100)</td></tr>
| |
− | </table>
| |
| | | |
| == NVLink 1.0 == | | == NVLink 1.0 == |
− | NVLink 1.0 was first introduced with the {{nvidia|P100}} [[GPGPU]] based on the {{nvidia|Pascal|l=arch}} microarchitecture. {{nvidia|P100}} comes with its own [[HBM]] memory in addition to being able to access system memory from the CPU side. The P100 has four NVLinks, which supports up to 20 GB/s for a bidrectional bandwidth of 40 GB/s for a total aggregated bandwidth of 160 GB/s. In the most basic configuration, all four links are connected between the two GPUs for 160 GB/s GPU-GPU bandwidth in addition to the PCIe lanes connected to the CPU for accessing system [[DRAM]]. | + | NVLink 1.0 was first introduced with the {{nvidia|P100}} [[GPGPU]] based on the {{nvidia|Pascal|l=arch}} microarchitecture. {{nvidia|P100}} comes with its own [[HBM]] memory in addition to being able to being able to access system memory from the CPU side. The P100 has four NVLinks, which supports up to 20 GB/s for a bidrectional bandwidth of 40 GB/s for a total aggregated bandwidth of 160 GB/s. In the most basic configuration, all four links are connected between the two GPUs for 160 GB/s GPU-GPU bandwidth in addition to the PCIe lanes connected to the CPU for accessing system [[DRAM]]. |
| | | |
| | | |
Line 69: |
Line 44: |
| === DGX-1 Configuration === | | === DGX-1 Configuration === |
| [[File:nvidia dgx-1 nvlink hybrid cube-mesh.svg|right|300px]] | | [[File:nvidia dgx-1 nvlink hybrid cube-mesh.svg|right|300px]] |
− | In 2017 Nvidia introduced the DGX-1 system which takes full advantage of NVLink. The DGX-1 consists of eight Tesla P100 GPUs connected in a hybrid cube-mesh NVLink network topology along with dual-socket {{intel|Xeon}} CPUs. The two Xeons communicate with each other over [[Intel]]'s {{intel|QPI}} while the GPUs communicate via the NVLink. | + | In 2017 Nvidia introduced the DGX-1 system which takes full advantage of NVLink. The DGX-1 consists of eight Tela P100 GPUs connected in a hybrid cube-mesh NVLink network topology along with dual-socket {{intel|Xeon}} CPUs. The two Xeons communicate with each other over [[Intel]]'s {{intel|QPU}} while the GPUs communicate via the NVLink. |
| | | |
| :[[File:nvidia dgx-1 nvlink-gpu-xeon config.svg|600px]] | | :[[File:nvidia dgx-1 nvlink-gpu-xeon config.svg|600px]] |
| | | |
| == NVLink 2.0 == | | == NVLink 2.0 == |
− | NVLink 2.0 was first introduced with the {{nvidia|V100}} [[GPGPU]] based on the {{nvidia|Volta|l=arch}} microarchitecture along with [[IBM]]'s {{ibm|POWER9|l=arch}}. Nvidia added CPU mastering support, allowing both the GPU and CPU to access each others memory (i.e., direct load and stores) in a flat address space. The flat address space is supported through new address translation services. Additionally, there native support for [[atomic operations]] was added for both the CPU and GPU. With the addition of flat address space, NVLink now has [[cache coherence]] support, allowing the CPU to efficiently cache GPU memory, significantly improving latencies and thus performance. NVLink 2.0 has improved signaling rate to 25 Gbps per wire (25 GT/s) for 50 GB/s bidirectional bandwidth. The V100 also increased the number of NVLinks on-die to 6 for a total aggregated bandwidth of 300 GB/s. It's worth noting that additional power saving features were added such as deactivating lanes during idle. | + | NVLink 2.0 was first introduced with the {{nvidia|V100}} [[GPGPU]] based on the {{nvidia|Volta|l=arch}} microarchitecture along with [[IBM]]'s {{ibm|POWER9|l=arch}}. |
− | | |
− | NVLink 2.0 was introduced with the second-generation {{nvidia|DGX-1}}, but the full topology change took place with the {{nvidia|DGX-2}}. Nvidia also introduced the {{nvidia|NVSwitch}} with the DGX-2 which is an 18 NVLink ports switch. The 2-billion transistor switch can route traffic from nine ports to any of the other nine ports. With 50 GB/s per port, the switch is capable of a total of 900 GB/s of bandwidth.
| |
− | | |
− | For the DGX-2, Nvidia uses six {{nvidia|NVSwitches}} to fully connect every one of the eight GPUs to all the other seven GPUs on the same baseboard.
| |
− | | |
− | | |
− | :[[File:dgx2 nvswitch baseboard diagram.svg|800px]]
| |
− | | |
− | | |
− | Two baseboards are then connected to each other to fully connect all 16 GPUs to each other.
| |
− | | |
− | | |
− | :[[File:dgx2 nvswitch baseboard diagram with two boards connected.svg|800px]]
| |
− | | |
− | == NVLink 3.0 ==
| |
− | NVLink 3.0 was first introduced with the {{nvidia|A100}} [[GPGPU]] based on the {{nvidia|Ampere|l=arch}} microarchitecture. NVLink 3.0 uses 50 Gbps signaling rate
| |
− | | |
− | == References ==
| |
− | * IEEE HotChips 34 (HC28), 2022
| |
− | * IEEE HotChips 30 (HC28), 2018
| |
− | * IEEE HotChips 29 (HC29), 2017
| |
− | * IEEE HotChips 28 (HC28), 2016
| |
− | | |
− | | |
| | | |
| [[category:nvidia]] | | [[category:nvidia]] |
| [[Category:interconnect_architectures]] | | [[Category:interconnect_architectures]] |