From WikiChip
Difference between revisions of "brain floating-point format"

(→‎Bibliography)
(→‎CPUs)
 
(10 intermediate revisions by 4 users not shown)
Line 1: Line 1:
 
{{title|Brain floating-point format (bfloat16)}}
 
{{title|Brain floating-point format (bfloat16)}}
−
'''Brain floating-point format''' ('''bfloat16''' or '''BF16''') is an [[number encoding format]] occupying 16 bits representing a floating-point number. It is equivalent to a standard [[single-precision floating-point]] value with a truncated [[mantissa field]]. Bfloat16 is designed to be used in hardware [[accelerating]] machine learning algorithms. Bfloat was first proposed and implemented by [[Google]] with [[Intel]] supporting it in their FPGAs, Nervana [[neural processors]], and CPUs.
+
'''Brain floating-point format''' ('''bfloat16''' or '''BF16''') is a [[number encoding format]] occupying 16 bits representing a floating-point number. It is equivalent to a standard [[single-precision floating-point]] value with a truncated [[mantissa field]]. Bfloat16 is designed to be used in hardware [[accelerating]] machine learning algorithms. Bfloat was first proposed and implemented by [[Google]] with [[Intel]] supporting it in their FPGAs, Nervana [[neural processors]], and CPUs.
  
 
== Overview ==
 
== Overview ==
−
Bfloat16 follows the same format as a standard IEEE 754 [[single-precision floating-point]] but truncates the [[mantissa field]] from 23 bits to just 7 bits. Preserving the exponent bits keeps the format to the same range as the 32-bit single precision FP (~1e<sup>-38</sup> to ~3e<sup>38</sup>). This allows for relatively simpler conversion between the two data types. In other words, while some resolution is lost, numbers can still be represented.
+
Bfloat16 follows the same format as a standard IEEE 754 [[single-precision floating-point]] but truncates the [[mantissa field]] from 23 bits to just 7 bits. Preserving the exponent bits keeps the format to the same range as the 32-bit single precision FP (~1e<sup>-38</sup> to ~3e<sup>38</sup>). This allows for relatively simpler conversion between the two data types. In other words, while some resolution is lost, numbers can still be represented. [[Microsoft]] developed a similar format for an {{microsoft|msfp8|8-bit floating point}} based on the float16 range.
  
  
Line 32: Line 32:
  
 
== Hardware support ==
 
== Hardware support ==
 +
=== CPUs ===
 +
* [[Intel]] {{intel|Cooper Lake|l=arch}}, {{intel|Ice Lake (Server)|l=arch}}
 +
* [[IBM]] {{ibm|POWER10|l=arch}}
 +
* [[Arm]] {{armh|Neoverse V1|l=arch}}, {{armh|Neoverse N2|l=arch}}
 +
 +
=== NPUs ===
 +
{{see also|neural processors}}
 +
* [[Arm]] {{arm|Project Trillium|l=arch}}
 +
* [[Centaur Technology]] {{centtech|CHA|CHA (NCORE)|l=arch}}
 +
* [[Flex Logix]] [[InferX]]
 
* [[Google]] {{google|TPU}}
 
* [[Google]] {{google|TPU}}
−
* [[Intel]] {{nervana|NPU}}, {{intel|Cooper Lake|l=arch}}
+
* [[Habana]] {{habana|HL}}
 +
* [[Intel]] {{nervana|NNP}}
 
* [[Wave Computing]] {{wave|DPU}}
 
* [[Wave Computing]] {{wave|DPU}}
−
* [[Flex Logix]] [[InferX]]
+
 
 +
== ISAs ==
 +
* Power: [[Power ISA v3.1]]
 +
* x86-64: [[AVX512_BF16]] (part of ''{{intel|DL Boost}} technology'')
 +
* AArch64: [[ARMv8.6-BF16]]
  
 
== See also ==
 
== See also ==
 +
* {{microsoft|MSFP8|MSFP8-11}}
 
* {{intel|Flexpoint}}
 
* {{intel|Flexpoint}}
  

Latest revision as of 12:35, 26 April 2021

Brain floating-point format (bfloat16 or BF16) is a number encoding format occupying 16 bits representing a floating-point number. It is equivalent to a standard single-precision floating-point value with a truncated mantissa field. Bfloat16 is designed to be used in hardware accelerating machine learning algorithms. Bfloat was first proposed and implemented by Google with Intel supporting it in their FPGAs, Nervana neural processors, and CPUs.

Overview[edit]

Bfloat16 follows the same format as a standard IEEE 754 single-precision floating-point but truncates the mantissa field from 23 bits to just 7 bits. Preserving the exponent bits keeps the format to the same range as the 32-bit single precision FP (~1e-38 to ~3e38). This allows for relatively simpler conversion between the two data types. In other words, while some resolution is lost, numbers can still be represented. Microsoft developed a similar format for an 8-bit floating point based on the float16 range.


float32: (range ~1e-38 to ~3e38)
float32 encoding format.svg


float16: (range ~5.96e-8 to 65,504)
float16 encoding format.svg


bfloat16: (range ~1e-38 to ~3e38)
bfloat16 encoding format.svg

Motivation[edit]

The motivation behind the reduced mantissa is derived from Google's experiments that showed that it is fine to reduce the mantissa so long it's still possible to represent tiny values closer to zero as part of the summation of small differences during training. Smaller mantissa brings a number of other advantages such as reducing the multiplier power and physical silicon area.

  • float32: 242=576 (100%)
  • float16: 112=121 (21%)
  • bfloat16: 82=64 (11%)

From the above, it can be seen that there is a factor of two and a factor of ten in terms of the number of bits that will flip (or components). From an efficiency standpoint, the gain was worth it for Google and their TPU implementation. At the system level, the benefits in memory capacity saving and bandwidth can also be realized if desired.

There are a number of other advantages from a software standpoint. From a debugging standpoint, since both bfloat16 and float32 share almost the same dynamic range, the software error behavior of bfloat16 tends to be almost the same as those of float32. Since bfloat16 maintains the same dynamic range, using lower precisions works well with bfloat16 without loss scaling. Getting training to converge on a standard 16-bit float requires multiplying the loss by around 210 and then adjusting the learning rate to get the same numerical result. This complexity is entirely eliminated with bfloat16 where there is no need to compensate for the loss of dynamic range.

Hardware support[edit]

CPUs[edit]

NPUs[edit]

See also: neural processors

ISAs[edit]

See also[edit]

Bibliography[edit]

  • Cliff Young, Google AI. (October 2018). "Codesign in Google TPUs: Inference, Training, Performance, and Scalability". Keynote speech, Processor Conference 2018.
  • Carey Kloss, Intel VP Hardware and AI Products Group. (April 2019). "Deep Learning By Design; Building silicon for AI." Processor Conference 2019.
Retrieved from "https://en.wikichip.org/w/index.php?title=brain_floating-point_format&oldid=98765"