x86
Instruction Set Architecture

General

• History
• Families

Variants

• x86-16
• x86-32
• x86-64

Topics

• Instructions
• Addressing Modes
• Registers
• Model-Specific Register
• Assembly
• Interrupts
• Micro-Ops
• Timer
• Calling Convention
• Microarchitectures
• CPUID

CPUIDs

• AMD's CPUIDs
• Intel's CPUIDs

Modes

• Real
• Protected
• Long

Extensions(all)

• 3DNow!
• ABM
• ADX
• AES
• AMX
• AVX
• AVX2
• AVX-512
• BMI1
• BMI2
• CLMUL
• E3DNow!
• EMMX
• F16C
• FMA3
• FMA4
• FPU
• MKTME
• MMX
• MPX
• PMEM
• PREFETCH
• RdRAND
• SEV
• SGX
• SHA
• SME
• SMM
• SMX
• SSE
• SSE2
• SSE3
• SSE4.1
• SSE4.2
• SSE4a
• SSE5
• SSSE3
• TBM
• TME
• TSME
• TSX
• XOP

AVX-512 Vector Neural Network Instructions (AVX512 VNNI) is an x86 extension, part of the AVX-512, designed to accelerate convolutional neural network-based algorithms.

Overview

The AVX512 VNNI x86 extension extends AVX-512 Foundation by introducing four new instructions for accelerating inner convolutional neural network loops.

• VPDPBUSD - Multiplies the individual bytes (8-bit) of the first source operand by the corresponding bytes (8-bit) of the second source operand, producing intermediate word (16-bit) results which are summed and accumulated in the double word (32-bit) of the destination operand.
  • VPDPBUSDS - Same as above except on intermediate sum overflow which saturates to 0x7FFF_FFFF/0x8000_0000 for positive/negative numbers.
• VPDPWSSD - Multiplies the individual words (16-bit) of the first source operand by the corresponding word (16-bit) of the second source operand, producing intermediate word results which are summed and accumulated in the double word (32-bit) of the destination operand.
  • VPDPWSSDS - Same as above except on intermediate sum overflow which saturates to 0x7FFF_FFFF/0x8000_0000 for positive/negative numbers.

Motivation

The major motivation behind the AVX512 VNNI extension is the observation that many tight convolutional neural network loops require the repeated multiplication of two 16-bit values or two 8-bit values and accumulate the result to a 32-bit accumulator. Using the foundation AVX-512, for 16-bit, this is possible using two instructions - VPMADDWD which is used to multiply two 16-bit pairs and add them together followed a VPADDD which adds the accumulate value.

Likewise, for 8-bit values, three instructions are needed - VPMADDUBSW which is used to multiply two 8-bit pairs and add them together, followed by a VPMADDWD with the value 1 in order to simply up-convert the 16-bit values to 32-bit values, followed by the VPADDD instruction which adds the result to an accumulator.

To address those two common operations, two new instructions were added (as well as two saturated versions): VPDPBUSD fuses VPMADDUBSW, VPMADDWD, and VPADDD and VPDPWSSD fuses VPMADDWD and VPADDD.

Detection

Support for these instructions is indicated by the AVX512_VNNI feature flag. 128- and 256-bit vectors are supported if the AVX512VL flag is set as well.

The AVX-VNNI extension adds AVX (VEX encoded) versions of these instructions operating on 128- and 256-bit vectors.

CPUID		Instruction Set
Input	Output
EAX=07H, ECX=0	EBX[bit 31]	AVX512VL
EAX=07H, ECX=0	ECX[bit 11]	AVX512_VNNI

Microarchitecture support

Designer	Microarchitecture	Year	Support Level
			F	CD	ER	PF	BW	DQ	VL	FP16	IFMA	VBMI	VBMI2	BITALG	VPOPCNTDQ	VP2INTERSECT	4VNNIW	4FMAPS	VNNI	BF16
Intel	Knights Landing	2016	✔	✔	✔	✔	✘	✘	✘	✘	✘	✘	✘	✘	✘	✘	✘	✘	✘	✘
	Knights Mill	2017	✔	✔	✔	✔	✘	✘	✘	✘	✘	✘	✘	✘	✔	✘	✔	✔	✘	✘
	Skylake (server)	2017	✔	✔	✘	✘	✔	✔	✔	✘	✘	✘	✘	✘	✘	✘	✘	✘	✘	✘
	Cannon Lake	2018	✔	✔	✘	✘	✔	✔	✔	✘	✔	✔	✘	✘	✘	✘	✘	✘	✘	✘
	Cascade Lake	2019	✔	✔	✘	✘	✔	✔	✔	✘	✘	✘	✘	✘	✘	✘	✘	✘	✔	✘
	Cooper Lake	2020	✔	✔	✘	✘	✔	✔	✔	✘	✘	✘	✘	✘	✘	✘	✘	✘	✔	✔
	Tiger Lake	2020	✔	✔	✘	✘	✔	✔	✔	✘	✔	✔	✔	✔	✔	✔	✘	✘	✔	✘
	Rocket Lake	2021	✔	✔	✘	✘	✔	✔	✔	✘	✔	✔	✔	✔	✔	✘	✘	✘	✔	✘
	Alder Lake	2021	✔	✔	✘	✘	✔	✔	✔	✔	✔	✔	✔	✔	✔	✔	✘	✘	✔	✔
	Ice Lake (server)	2021	✔	✔	✘	✘	✔	✔	✔	✘	✔	✔	✔	✔	✔	✘	✘	✘	✔	✘
	Sapphire Rapids	2023	✔	✔	✘	✘	✔	✔	✔	✔	✔	✔	✔	✔	✔	✘	✘	✘	✔	✔
AMD	Zen 4	2022	✔	✔	✘	✘	✔	✔	✔	✘	✔	✔	✔	✔	✔	✘	✘	✘	✔	✔
Centaur	CHA		✔	✔	✘	✘	✔	✔	✔	✘	✔	✔	✘	✘	✘	✘	✘	✘	✘	✘

Intrinsic functions

// VPDPBUSD
__m128i _mm_dpbusd_epi32(__m128i, __m128i, __m128i);
__m128i _mm_mask_dpbusd_epi32(__m128i, __mmask8, __m128i, __m128i);
__m128i _mm_maskz_dpbusd_epi32(__mmask8, __m128i, __m128i, __m128i);
__m256i _mm256_dpbusd_epi32(__m256i, __m256i, __m256i);
__m256i _mm256_mask_dpbusd_epi32(__m256i, __mmask8, __m256i, __m256i);
__m256i _mm256_maskz_dpbusd_epi32(__mmask8, __m256i, __m256i, __m256i);
__m512i _mm512_dpbusd_epi32 (__m512i src, __m512i a, __m512i b);
__m512i _mm512_mask_dpbusd_epi32 (__m512i src, __mmask16 k, __m512i a, __m512i b);
__m512i _mm512_maskz_dpbusd_epi32 (__mmask16 k, __m512i src, __m512i a, __m512i b);
// VPDPBUSDS
__m128i _mm_dpbusds_epi32(__m128i, __m128i, __m128i);
__m128i _mm_mask_dpbusds_epi32(__m128i, __mmask8, __m128i, __m128i);
__m128i _mm_maskz_dpbusds_epi32(__mmask8, __m128i, __m128i, __m128i);
__m256i _mm256_dpbusds_epi32(__m256i, __m256i, __m256i);
__m256i _mm256_mask_dpbusds_epi32(__m256i, __mmask8, __m256i, __m256i);
__m256i _mm256_maskz_dpbusds_epi32(__mmask8, __m256i, __m256i, __m256i);
__m512i _mm512_dpbusds_epi32 (__m512i src, __m512i a, __m512i b);
__m512i _mm512_mask_dpbusds_epi32 (__m512i src, __mmask16 k, __m512i a, __m512i b);
__m512i _mm512_maskz_dpbusds_epi32 (__mmask16 k, __m512i src, __m512i a, __m512i b);
// VPDPWSSD
__m128i _mm_dpwssd_epi32(__m128i, __m128i, __m128i);
__m128i _mm_mask_dpwssd_epi32(__m128i, __mmask8, __m128i, __m128i);
__m128i _mm_maskz_dpwssd_epi32(__mmask8, __m128i, __m128i, __m128i);
__m256i _mm256_dpwssd_epi32(__m256i, __m256i, __m256i);
__m256i _mm256_mask_dpwssd_epi32(__m256i, __mmask8, __m256i, __m256i);
__m256i _mm256_maskz_dpwssd_epi32(__mmask8, __m256i, __m256i, __m256i);
__m512i _mm512_dpwssd_epi32 (__m512i src, __m512i a, __m512i b);
__m512i _mm512_mask_dpwssd_epi32 (__m512i src, __mmask16 k, __m512i a, __m512i b);
__m512i _mm512_maskz_dpwssd_epi32 (__mmask16 k, __m512i src, __m512i a, __m512i b);
// VPDPWSSDS
__m128i _mm_dpwssds_epi32(__m128i, __m128i, __m128i);
__m128i _mm_mask_dpwssds_epi32(__m128i, __mmask8, __m128i, __m128i);
__m128i _mm_maskz_dpwssds_epi32(__mmask8, __m128i, __m128i, __m128i);
__m256i _mm256_dpwssds_epi32(__m256i, __m256i, __m256i);
__m256i _mm256_mask_dpwssds_epi32(__m256i, __mmask8, __m256i, __m256i);
__m256i _mm256_maskz_dpwssds_epi32(__mmask8, __m256i, __m256i, __m256i);
__m512i _mm512_dpwssds_epi32 (__m512i src, __m512i a, __m512i b);
__m512i _mm512_mask_dpwssds_epi32 (__m512i src, __mmask16 k, __m512i a, __m512i b);
__m512i _mm512_maskz_dpwssds_epi32 (__mmask16 k, __m512i src, __m512i a, __m512i b);

Bibliography

• Intel, IEEE Hot Chips 30 Symposium (HCS) 2018.
• Rodriguez, Andres, et al. "Lower numerical precision deep learning inference and training." Intel White Paper (2018).
• "Intel® Architecture Instruction Set Extensions and Future Features", Intel Order Nr. 319433, Rev. 047, December 2022