From WikiChip
Editing brain floating-point format
Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.
The edit can be undone.
Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.
This page supports semantic in-text annotations (e.g. "[[Is specified as::World Heritage Site]]") to build structured and queryable content provided by Semantic MediaWiki. For a comprehensive description on how to use annotations or the #ask parser function, please have a look at the getting started, in-text annotation, or inline queries help pages.
Latest revision | Your text | ||
Line 1: | Line 1: | ||
{{title|Brain floating-point format (bfloat16)}} | {{title|Brain floating-point format (bfloat16)}} | ||
− | '''Brain floating-point format''' ('''bfloat16''' or '''BF16''') is | + | '''Brain floating-point format''' ('''bfloat16''' or '''BF16''') is an [[number encoding format]] occupying 16 bits representing a floating-point number. It is equivalent to a standard [[single-precision floating-point]] value with a truncated [[mantissa field]]. Bfloat16 is designed to be used in hardware [[accelerating]] machine learning algorithms. Bfloat was first proposed and implemented by [[Google]] with [[Intel]] supporting it in their FPGAs, Nervana [[neural processors]], and CPUs. |
== Overview == | == Overview == | ||
− | Bfloat16 follows the same format as a standard IEEE 754 [[single-precision floating-point]] but truncates the [[mantissa field]] from 23 bits to just 7 bits. Preserving the exponent bits keeps the format to the same range as the 32-bit single precision FP (~1e<sup>-38</sup> to ~3e<sup>38</sup>). This allows for relatively simpler conversion between the two data types. In other words, while some resolution is lost, numbers can still be represented | + | Bfloat16 follows the same format as a standard IEEE 754 [[single-precision floating-point]] but truncates the [[mantissa field]] from 23 bits to just 7 bits. Preserving the exponent bits keeps the format to the same range as the 32-bit single precision FP (~1e<sup>-38</sup> to ~3e<sup>38</sup>). This allows for relatively simpler conversion between the two data types. In other words, while some resolution is lost, numbers can still be represented. |
Line 32: | Line 32: | ||
== Hardware support == | == Hardware support == | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
* [[Google]] {{google|TPU}} | * [[Google]] {{google|TPU}} | ||
− | * [[ | + | * [[Intel]] {{nervana|NPU}}, {{intel|Cooper Lake|l=arch}} |
− | |||
* [[Wave Computing]] {{wave|DPU}} | * [[Wave Computing]] {{wave|DPU}} | ||
− | + | * [[Flex Logix]] [[InferX]] | |
− | |||
− | * | ||
− | |||
− | |||
== See also == | == See also == | ||
− | |||
* {{intel|Flexpoint}} | * {{intel|Flexpoint}} | ||