From WikiChip
Difference between revisions of "nvidia/tegra/xavier"
< nvidia

(→‎Overview)
(→‎GPU: Fixed tensor core MACs per cycle numbers.)
 
(19 intermediate revisions by 4 users not shown)
Line 15: Line 15:
 
|microarch=Carmel
 
|microarch=Carmel
 
|microarch 2=Volta
 
|microarch 2=Volta
 +
|core name=Carmel
 
|process=12 nm
 
|process=12 nm
 
|transistors=9,000,000,000
 
|transistors=9,000,000,000
Line 22: Line 23:
 
|core count=8
 
|core count=8
 
|thread count=8
 
|thread count=8
 +
|max cpus=4
 
|tdp=30 W
 
|tdp=30 W
 
|tdp typical=20 W
 
|tdp typical=20 W
Line 29: Line 31:
 
== Overview ==
 
== Overview ==
 
[[File:xavier overview.png|right|thumb|Overview (HC 30)]]
 
[[File:xavier overview.png|right|thumb|Overview (HC 30)]]
−
Xavier is an autonomous machine process designed by [[Nvidia]] and introduced at CES 2018. Silicon came back in the last week of December 2017 with sampling started in the first quarter of 2018. NVIDIA plans on mass production by the end of the year. NVIDIA reported that the product is a result of $2 billion R&D and 8,000 engineering years. The chip is said to have full redundancy and diversity in its functional blocks.
+
Xavier is an autonomous machine processor designed by [[Nvidia]] and introduced at CES 2018. Silicon came back in the last week of December 2017 with sampling started in the first quarter of 2018. NVIDIA plans on mass production by the end of the year. NVIDIA reported that the product is a result of $2 billion R&D and 8,000 engineering years. The chip is said to have full redundancy and diversity in its functional blocks.
  
 
:[[File:xavier block.svg|500px]]
 
:[[File:xavier block.svg|500px]]
  
 
The design targets and architecture started back in [[2014]]. Fabricated on [[TSMC]] [[12 nm process]], the chip itself comprises an eight-core CPU cluster, GPU with additional inference optimizations, [[neural processor|deep learning accelerator]], vision accelerator, and a set of multimedia accelerators providing additional support for machine learning (stereo, LDC, optical flow). The ISP has been enhanced to provide native HDR support, higher precision math without offloading work to the GPU. Xavier features a large set of I/O and has been designed for safety and reliability supporting various standards such as Functional safety [[ISO-26262]] and [[ASIL]] level C. The CPU cluster is fully [[cache coherent]] and the coherency is extended to all the other [[accelerators]] on-chip.
 
The design targets and architecture started back in [[2014]]. Fabricated on [[TSMC]] [[12 nm process]], the chip itself comprises an eight-core CPU cluster, GPU with additional inference optimizations, [[neural processor|deep learning accelerator]], vision accelerator, and a set of multimedia accelerators providing additional support for machine learning (stereo, LDC, optical flow). The ISP has been enhanced to provide native HDR support, higher precision math without offloading work to the GPU. Xavier features a large set of I/O and has been designed for safety and reliability supporting various standards such as Functional safety [[ISO-26262]] and [[ASIL]] level C. The CPU cluster is fully [[cache coherent]] and the coherency is extended to all the other [[accelerators]] on-chip.
 +
 +
At the platform level, one of the bigger changes took place at the I/O subsystem. Xavier features [[NVLink]] 1.0 supporting 20 GB/s in each direction for connecting a [[discrete graphics processor]] to Xavier in a [[cache coherent]] manner. Xavier has PCIe Gen 4.0 support (16 GT/s). It's worth noting that Xavier added support for an end-point mode in addition to the standard root complex support. This support meant they can connect two Xaviers directly one to another (2-way multiprocessing) without going through a PCIe switch or alike.
  
 
== Architecture ==
 
== Architecture ==
 
=== CPU ===
 
=== CPU ===
 +
[[File:xavier cpu complex.svg|right|thumb|300px|CPU Block Diagram]]
 
{{main|nvidia/microarchitectures/carmel|l1=Carmel core}}
 
{{main|nvidia/microarchitectures/carmel|l1=Carmel core}}
−
The chip features eight control/management {{nvidia|Carmel|l=arch}} cores, Nvidia's own custom {{arch|64}} [[ARM]] cores. Those cores implement [[ARMv8.2]] with [[RAS]] support and safety built-in, including dual-execution mode. All cores are fully [[cache coherent]] which is extended to {{nvidia|Volta|the GPU|l=arch}} and all the other accelerators in the chip.
+
The chip features eight control/management {{nvidia|Carmel|l=arch}} cores, Nvidia's own custom {{arch|64}} [[ARM]] cores. Those cores implement [[ARMv8.2]] with [[RAS]] support and safety built-in, including dual-execution mode. The cluster consists of 4 duplexes, each sharing 2 MiB of L2 cache. All cores are fully [[cache coherent]] which is extended to {{nvidia|Volta|the GPU|l=arch}} and all the other accelerators in the chip. Compared to {{\\|Parker}} which was based on {{nvidia|Denver 2|l=arch}}, Nividia reports around 2x the multithreaded performance.
  
 
=== GPU ===
 
=== GPU ===
 +
[[File:xavier gpu.svg|right|thumb|300px|GPU Block Diagram]]
 
{{main|nvidia/microarchitectures/volta|l1=Volta}}
 
{{main|nvidia/microarchitectures/volta|l1=Volta}}
−
Xavier implements a derivative of their {{nvidia|Volta|l=arch}} GPU with a set of finer changes to address the machine earning market. It has eight Volta stream multiprocessors along with their standard 128 KiB of L1 cache and a 512 KiB of shared L2. Compared to Parker, Nvidia claims this GPU has 2.1x the graphics performance. Whereas their desktop parts (e.g., GV100) are a very powerful GPU that is used for training, the GPU here is optimized for inference. The most obvious change is that they added int8 support for lower precision to the CUDA tensor cores and those operate at the full 2x throughput of the FP16 ops. There is also 512 CUDA tensor cores, a number that's comparable to Nvidia's top-end models for machine learning (e.g., the GV100 has 672). All of this yields 22.6 tera-operations (int8) per second.
+
Xavier implements a derivative of their {{nvidia|Volta|l=arch}} GPU with a set of finer changes to address the machine learning market, particularly improving inference performance over training. It has eight Volta stream multiprocessors along with their standard 128 KiB of L1 cache and a 512 KiB of shared L2. Compared to Parker, Nvidia claims this GPU has 2.1x the graphics performance. Whereas their desktop parts (e.g., GV100) are a very powerful GPU that is used for training, the GPU here is optimized for inference. The most obvious change is that each Volta multiprocessor contains eight tensor cores, each of which can perform 64x FP16 MACs or 128x INT8 MACs per cycle. All of this yields a maximum 22.6 tera-operations (int8) per second.
 +
 
 +
{| class="wikitable"
 +
|-
 +
! Throughput
 +
|-
 +
| 22.6 DL TOPS 8-bit
 +
|-
 +
| 2.8 CUDA TFLOPS FP16
 +
|-
 +
| 1.4 CUDA TFLOPS FP32
 +
|}
  
 
=== Accelerators ===
 
=== Accelerators ===
 
Xavier incorporates a set of accelerators designed to augment the functionality offered by the GPU and CPU in order to provide added flexibility and perhaps offer a way to implement some of the more common set of algorithms slightly more efficiently.
 
Xavier incorporates a set of accelerators designed to augment the functionality offered by the GPU and CPU in order to provide added flexibility and perhaps offer a way to implement some of the more common set of algorithms slightly more efficiently.
 
==== Programmable Vision Accelerator ====
 
==== Programmable Vision Accelerator ====
−
Xavier incorporates a Programmable Vision Accelerator (PVA) for processing computer vision. There are actually two exact instances of the PVA on-chip, each can be used in lock-step or independently and are capable of implementing some of the common filter loop and other detection algorithms (e.g. [[Harris corner]]). For each of the PVAs, there are two dedicated vector processing units, each with its own memory and DMA. This 7-stock VLIW architecture made of 2 scalar slots, 2 vector slots, and 3 memory operations. The pipe is 256 bit wide and all types can operate at full throughput (32x8b, 16x16b, and 8x32b vector math). Additionally, since the chip is expected to be connected to a network of cameras (e.g., side, front, inside), the chip is capable of doing real-time [[encoding]] for all camera in [[high dynamic range]].  
+
[[File:xavier pva block.svg|right|thumb|300px|PVA Block Diagram]]
 +
Xavier incorporates a Programmable Vision Accelerator (PVA) for processing computer vision. There are actually two exact instances of the PVA on-chip, each can be used in lock-step or independently and are capable of implementing some of the common filter loop and other detection algorithms (e.g. [[Harris corner]], [[fast Fourier transform|FFTs]]). For each of the PVAs, there is a {{armh|Cortex-R5|l=arch}} core along with two dedicated vector processing units, each with its own memory and DMA. The DMA on the PVA is designed to operate on tiles of memory. To that end, the DMA performs the address calculation and can perform prefetching while the processing pipes operate. This is 7-slot VLIW architecture made of 2 scalar slots, 2 vector slots, and 3 memory operations. The pipe is 256 bit wide (slightly wider because of the [[guard bits]] keeping the precision for the operation) and all types can operate at full throughput (32x8b, 16x16b, and 8x32b vector math). The pipe supports additional operations beyond vector such as custom logic for table lookup and hardware looping.
 +
 
 +
{| class="wikitable"
 +
|-
 +
! Throughput
 +
|-
 +
| 1.7 CV TOPS
 +
|}
 +
 
 +
It's worth noting that since the chip is expected to be connected to a network of cameras (e.g., side, front, inside), the PVA is capable of doing real-time [[encoding]] for all camera in [[high dynamic range]].  
  
 
{| class="wikitable"
 
{| class="wikitable"
Line 61: Line 88:
  
 
==== Deep Learning Accelerator ====
 
==== Deep Learning Accelerator ====
−
The chip incorporates a deep learning accelerator (DLA) that implements a number of specific set of deep learning functions common to many applications. This allows them to read the highest possible energy efficiency for those operations. The DLA has a peak performance of 5 [[TOPS]] for 16-bit integers or 10 [[TOPS]] for 8-bit integer.
+
The other accelerators on-die is the deep learning accelerator (DLA) which is actually a physical implementation of the open source Nvidia NVDLA architecture. Xavier has two instances of NVDLA which can offer a peak theoretical performance of 5.7 [[teraFLOPS]] (half precision FP) or twice the throughput at 11.4 TOPS for int8.
 +
 
 +
{| class="wikitable"
 +
|-
 +
! Throughput
 +
|-
 +
| 11.4 TOPS (int8)
 +
|-
 +
| 5.7 TFLOPS (FP16)
 +
|}
 +
 
 +
==== Stereo & Optical Flow Engine ====
 +
Xavier has dedicated engines for stereo and optical flow
 +
 
 +
{| class="wikitable"
 +
|-
 +
! Throughput
 +
|-
 +
| 12.4 TOPS 8-bit
 +
|-
 +
| 6.2 TOPS 16-bit
 +
|}
  
 
== Memory controller ==
 
== Memory controller ==
Line 73: Line 121:
  
 
== I/O ==
 
== I/O ==
−
* 16 [[Camera Serial Interface|CSI]] channels
+
* [[NVLINK]]
−
** 109 Gbps total bandwidth
+
** NVLINK 1.0 (20 GB/s)
−
* 1x gE interface
+
* [[PCIE]]
−
* 1x 10gE interface
+
** Multiple 16GT/s gen4 controllers
 +
** x8, x4, x2, x1 configurations
 +
** Root port + Endpoint
 +
* USB
 +
** 3x USB3.1 (10 GT/s) ports
 +
** 4x USB2.0 ports
 +
* DISPLAY
 +
** 4x DP/HDMI/eDP
 +
** 4K @ 60 Hz
 +
** DP HBR3
 +
** HDMI 2.0
 +
* CAMERA
 +
** 16 CSI lanes
 +
** 40 Gbps in DPHY 1.2 Mode
 +
** 109 Gbps in CPHY 1.1 Mode
 +
* OTHER
 +
** Ethernet
 +
** CAN
 +
** UART
 +
** UFS
 +
** SDMMC
 +
** SPIO
 +
** I2C
 +
** I2S
 +
** GPIO
  
 
== Die ==
 
== Die ==
Line 92: Line 164:
 
* ~89.2 mm² silicon area
 
* ~89.2 mm² silicon area
  
−
:[[File:xavier die volta gpu.png|500px]]
+
:[[File:xavier die volta gpu.png|600px]]
  
  
−
:[[File:xavier die volta gpu (annotated).png|500px]]
+
:[[File:xavier die volta gpu (annotated).png|600px]]
  
 
=== CPU ===
 
=== CPU ===
Line 101: Line 173:
  
 
=== PVA ===
 
=== PVA ===
−
:[[File:xavier die pva.png|650px]]
+
:[[File:xavier die pva.png|750px]]
  
−
:[[File:xavier die pva (annotated).png|650px]]
+
:[[File:xavier die pva (annotated).png|750px]]
  
 
=== MM Engine / DLA ===
 
=== MM Engine / DLA ===
 
* ~21.75 mm² silicon area
 
* ~21.75 mm² silicon area
  
−
:[[File:xavier die mm-dl accel.png|650px]]
+
:[[File:xavier die mm-dl accel.png|750px]]
  
−
== Borads ==
+
== Board ==
 
<gallery mode=packed-hover heights="300px" widths="300px">
 
<gallery mode=packed-hover heights="300px" widths="300px">
 
jetson_xavier_(front).png|Jetson Xavier, front
 
jetson_xavier_(front).png|Jetson Xavier, front
Line 118: Line 190:
 
== Documents ==
 
== Documents ==
 
* [[:File:ces2018 - nvidia drive xavier.pdf|CES 2018: Nvidia Drive Xavier]]
 
* [[:File:ces2018 - nvidia drive xavier.pdf|CES 2018: Nvidia Drive Xavier]]
 +
 +
== See also ==
 +
* Tesla {{teslacar|FSD Chip}}
  
 
== Bibliography ==
 
== Bibliography ==
 
* IEEE Hot Chips 30 Symposium (HCS) 2018.
 
* IEEE Hot Chips 30 Symposium (HCS) 2018.
 +
* Schor, David. (September, 2018). "[https://fuse.wikichip.org/news/1618/hot-chips-30-nvidia-xavier-soc/ Hot Chips 30: Nvidia Xavier SoC]"

Latest revision as of 02:08, 9 December 2019

Edit Values
Xavier
xavier soc chip.png
General Info
DesignerNvidia
ManufacturerTSMC
Model NumberTegra194
MarketArtificial Intelligence, Embedded
IntroductionJanuary 8, 2018 (announced)
June, 2018 (launched)
General Specs
FamilyTegra
Microarchitecture
ISAARMv8 (ARM)
MicroarchitectureCarmel, Volta
Core NameCarmel
Process12 nm
Transistors9,000,000,000
TechnologyCMOS
Die350 mm²
Word Size64 bit
Cores8
Threads8
Multiprocessing
Max SMP4-Way (Multiprocessor)
Electrical
TDP30 W
TDP (Typical)20 W

Tegra Xavier is a 64-bit ARM high-performance system on a chip for autonomous machines designed by Nvidia and introduced in 2018. Xavier is incorporated into a number of Nvidia's computers including the Jetson Xavier, Drive Xavier, and the Drive Pegasus.

Overview[edit]

Overview (HC 30)

Xavier is an autonomous machine processor designed by Nvidia and introduced at CES 2018. Silicon came back in the last week of December 2017 with sampling started in the first quarter of 2018. NVIDIA plans on mass production by the end of the year. NVIDIA reported that the product is a result of $2 billion R&D and 8,000 engineering years. The chip is said to have full redundancy and diversity in its functional blocks.

xavier block.svg

The design targets and architecture started back in 2014. Fabricated on TSMC 12 nm process, the chip itself comprises an eight-core CPU cluster, GPU with additional inference optimizations, deep learning accelerator, vision accelerator, and a set of multimedia accelerators providing additional support for machine learning (stereo, LDC, optical flow). The ISP has been enhanced to provide native HDR support, higher precision math without offloading work to the GPU. Xavier features a large set of I/O and has been designed for safety and reliability supporting various standards such as Functional safety ISO-26262 and ASIL level C. The CPU cluster is fully cache coherent and the coherency is extended to all the other accelerators on-chip.

At the platform level, one of the bigger changes took place at the I/O subsystem. Xavier features NVLink 1.0 supporting 20 GB/s in each direction for connecting a discrete graphics processor to Xavier in a cache coherent manner. Xavier has PCIe Gen 4.0 support (16 GT/s). It's worth noting that Xavier added support for an end-point mode in addition to the standard root complex support. This support meant they can connect two Xaviers directly one to another (2-way multiprocessing) without going through a PCIe switch or alike.

Architecture[edit]

CPU[edit]

CPU Block Diagram
Main article: Carmel core

The chip features eight control/management Carmel cores, Nvidia's own custom 64-bit ARM cores. Those cores implement ARMv8.2 with RAS support and safety built-in, including dual-execution mode. The cluster consists of 4 duplexes, each sharing 2 MiB of L2 cache. All cores are fully cache coherent which is extended to the GPU and all the other accelerators in the chip. Compared to Parker which was based on Denver 2, Nividia reports around 2x the multithreaded performance.

GPU[edit]

GPU Block Diagram
Main article: Volta

Xavier implements a derivative of their Volta GPU with a set of finer changes to address the machine learning market, particularly improving inference performance over training. It has eight Volta stream multiprocessors along with their standard 128 KiB of L1 cache and a 512 KiB of shared L2. Compared to Parker, Nvidia claims this GPU has 2.1x the graphics performance. Whereas their desktop parts (e.g., GV100) are a very powerful GPU that is used for training, the GPU here is optimized for inference. The most obvious change is that each Volta multiprocessor contains eight tensor cores, each of which can perform 64x FP16 MACs or 128x INT8 MACs per cycle. All of this yields a maximum 22.6 tera-operations (int8) per second.

Throughput
22.6 DL TOPS 8-bit
2.8 CUDA TFLOPS FP16
1.4 CUDA TFLOPS FP32

Accelerators[edit]

Xavier incorporates a set of accelerators designed to augment the functionality offered by the GPU and CPU in order to provide added flexibility and perhaps offer a way to implement some of the more common set of algorithms slightly more efficiently.

Programmable Vision Accelerator[edit]

PVA Block Diagram

Xavier incorporates a Programmable Vision Accelerator (PVA) for processing computer vision. There are actually two exact instances of the PVA on-chip, each can be used in lock-step or independently and are capable of implementing some of the common filter loop and other detection algorithms (e.g. Harris corner, FFTs). For each of the PVAs, there is a Cortex-R5 core along with two dedicated vector processing units, each with its own memory and DMA. The DMA on the PVA is designed to operate on tiles of memory. To that end, the DMA performs the address calculation and can perform prefetching while the processing pipes operate. This is 7-slot VLIW architecture made of 2 scalar slots, 2 vector slots, and 3 memory operations. The pipe is 256 bit wide (slightly wider because of the guard bits keeping the precision for the operation) and all types can operate at full throughput (32x8b, 16x16b, and 8x32b vector math). The pipe supports additional operations beyond vector such as custom logic for table lookup and hardware looping.

Throughput
1.7 CV TOPS

It's worth noting that since the chip is expected to be connected to a network of cameras (e.g., side, front, inside), the PVA is capable of doing real-time encoding for all camera in high dynamic range.

Video Processing
Encode Decode
1.2 GPIX/s 1.8 GPIX/s

The chip has an ISP with native full-range HDR support and tile rendering capable of processing at 1.5 GPIX/s.

Deep Learning Accelerator[edit]

The other accelerators on-die is the deep learning accelerator (DLA) which is actually a physical implementation of the open source Nvidia NVDLA architecture. Xavier has two instances of NVDLA which can offer a peak theoretical performance of 5.7 teraFLOPS (half precision FP) or twice the throughput at 11.4 TOPS for int8.

Throughput
11.4 TOPS (int8)
5.7 TFLOPS (FP16)

Stereo & Optical Flow Engine[edit]

Xavier has dedicated engines for stereo and optical flow

Throughput
12.4 TOPS 8-bit
6.2 TOPS 16-bit

Memory controller[edit]

[Edit/Modify Memory Info]

ram icons.svg
Integrated Memory Controller
Max TypeLPDDR4X-4266
Supports ECCYes
Channels8
Width32 bit
Max Bandwidth127.1 GiB/s
130,150.4 MiB/s
136.473 GB/s
136,472.586 MB/s
0.124 TiB/s
0.136 TB/s

I/O[edit]

  • NVLINK
    • NVLINK 1.0 (20 GB/s)
  • PCIE
    • Multiple 16GT/s gen4 controllers
    • x8, x4, x2, x1 configurations
    • Root port + Endpoint
  • USB
    • 3x USB3.1 (10 GT/s) ports
    • 4x USB2.0 ports
  • DISPLAY
    • 4x DP/HDMI/eDP
    • 4K @ 60 Hz
    • DP HBR3
    • HDMI 2.0
  • CAMERA
    • 16 CSI lanes
    • 40 Gbps in DPHY 1.2 Mode
    • 109 Gbps in CPHY 1.1 Mode
  • OTHER
    • Ethernet
    • CAN
    • UART
    • UFS
    • SDMMC
    • SPIO
    • I2C
    • I2S
    • GPIO

Die[edit]

SoC[edit]

  • 9,000,000,000 transistors
  • 350 mm² die size
  • TSMC's 12FFN

nvidia xavier die shot.png


nvidia xavier die shot (annotated).png

GPU[edit]

  • ~89.2 mm² silicon area
xavier die volta gpu.png


xavier die volta gpu (annotated).png

CPU[edit]

See Carmel § CPU Complex.

PVA[edit]

xavier die pva.png
xavier die pva (annotated).png

MM Engine / DLA[edit]

  • ~21.75 mm² silicon area
xavier die mm-dl accel.png

Board[edit]

  • Jetson Xavier, front

  • Jetson Xavier, back

Documents[edit]

See also[edit]

Bibliography[edit]

Retrieved from "https://en.wikichip.org/w/index.php?title=nvidia/tegra/xavier&oldid=94785"
Facts about "Tegra Xavier - Nvidia"
RDF feed
core count8 +
designerNvidia +
die area350 mm² (0.543 in², 3.5 cm², 350,000,000 µm²) +
familyTegra +
first announcedJanuary 8, 2018 +
first launchedJune 2018 +
full page namenvidia/tegra/xavier +
has ecc memory supporttrue +
instance ofmicroprocessor +
isaARMv8 +
isa familyARM +
ldateJune 2018 +
main imageFile:xavier soc chip.png +
manufacturerTSMC +
market segmentArtificial Intelligence + and Embedded +
max memory bandwidth127.1 GiB/s (130,150.4 MiB/s, 136.473 GB/s, 136,472.586 MB/s, 0.124 TiB/s, 0.136 TB/s) +
max memory channels8 +
microarchitectureCarmel + and Volta +
model numberTegra194 +
nameXavier +
process12 nm (0.012 μm, 1.2e-5 mm) +
supported memory typeLPDDR4X-4266 +
tdp30 W (30,000 mW, 0.0402 hp, 0.03 kW) +
tdp (typical)20 W (20,000 mW, 0.0268 hp, 0.02 kW) +
technologyCMOS +
thread count8 +
transistor count9,000,000,000 +
word size64 bit (8 octets, 16 nibbles) +