Flexpoint is a compact number encoding format developed by Intel's Nervana used to represent standard floating point values. Flexpoint combines the advantages of fixed point and floating point by splitting up the mantissa and the exponent part which is shared across all arithmetic execution elements. By only passing the integer value, both memory and bandwidth requirements are reduced. Additionally, this lowers hardware complexity, lowering both power and area requirements.

Overview

Flexpoint was initially proposed by Intel's Nervana for their neural processor in 2017. Flexpoint splits up a floating point value into its two fundamental components: mantissa and exponent. An N-bit mantissa and M-bit exponent is represented as flexN+M, for example flex32+8 refers to a flexpoint value with 8-bit exponent and a 32-bit mantissa.

Unlike standard IEEE 754 floating point-based values, the M-bit exponent is not propagated down as part of the value in flexpoint. Instead, the exponent is shared across all the applicable execution units. Note that this is different from a fixed point because the exponent in flexpoint can be updated automatically.

The premise behind the format is the reduction of memory and bandwidth. Additionally, since the exponent is identical across all the relevant compute elements, operations such as addition and multiplication can be done as a fixed point operation.

Bibliography

• Köster, Urs, et al. "Flexpoint: An adaptive numerical format for efficient training of deep neural networks." Advances in Neural Information Processing Systems. 2017.