Jump to content

Talk:Bfloat16 floating-point format

Page contents not supported in other languages.
From Wikipedia, the free encyclopedia

Error in bit layout illustration

[edit]

The file Bfloat16_format.svg shows the wrong bit indices (first field is 0-7 when it should be 0-6). I tried uploading a newer version with that fixed. It shows up. But the SVG itself is the old version.

Maybe it's because I used the exact same filename. I still believe this behavior is incorrect. Tried changing the filename and reuploading. That version is not even showing up. Now I can't even upload newer versions. Maybe this was flagged for moderation.

I don't know. I give up. Uploading experience is really bad. --nachokb (talk) 09:21, 27 September 2018 (UTC)[reply]

It seems to be correct, now. I suppose that your issue was due to caching. Vincent Lefèvre (talk) 09:49, 27 September 2018 (UTC)[reply]

History

[edit]

It would be good to have a section explaining where this format was introduced and a history of its adoption. — Steven G. Johnson (talk) 20:04, 3 June 2021 (UTC)[reply]

Add additional information about rounding

[edit]
  • Specific text to be added or removed

1. Use Shortened instead of Truncated

To avoid confusion with the rounding mechanism, the first change is to replace “truncated” with “shortened” when it comes to describing bfloat16. “The bfloat16 format, being a shortened IEEE 754 single-precision 32-bit float, …”. In such a way, the text won’t be misunderstood with the truncation rounding in conversion.

2. Add a Format Conversion Section Add a format conversion section to detail the rounding mechanism. The proposed text is as follows:


Rounding and Conversion

The most common use case is the conversion between binary32 and bfloat16. The following section describes the conversion process and its rounding scheme in the conversion. Note that there are other possible scenarios of format conversions to or from bfloat16. For example, int16 and bfloat16.

a. From IEEE754 binary32 (32-bit floating point) to bfloat16

When bfloat16 was first introduced as a storage format[1], the conversion from IEEE754 binary32 (32-bit floating point) to bfloat16 is truncation (round-to-zero). Later on, when it becomes the input of matrix multiplication units, the conversion can have various rounding mechanisms depending on the hardware platforms. For example, Google TPU uses round-to-nearest-evenfor Google TPU, the rounding scheme in the conversion is round-to-nearest-even[2]; ARM uses round-to-nearest-odd the non-IEEE Round-to-Odd mode.[3] [4]; for NVIDIA, it supports supports four rounding schemes outlined in IEEE754, including round-to-nearest-evenconverting float number to bfloat16 precision in round-to-nearest-even mode[5].

b. From bfloat16 to IEEE754 binary32 (32-bit floating point)

Since IEEE binary32 can represent all exact values in bfloat16, the conversion simply pads 16 zeros in the mantissa bits[2].


  • Reason for the change: To reflect the recent status that the industry vendors have supported various rounding mechanisms other than truncation.
  • References supporting change: NVIDIA, ARM

2A00:79E1:ABC:12B:BDC7:5327:147A:E174 (talk) 22:09, 18 July 2023 (UTC)[reply]

Disclosure: the requestor works at Google, and some content of the edit request is related to Google's product. 2A00:79E1:ABC:12B:BDC7:5327:147A:E174 (talk) 22:09, 18 July 2023 (UTC)[reply]

Reply 18-JUL-2019

[edit]

  Unable to review  
Your edit request could not be reviewed because the request is not formatted correctly.

  1. The citation style predominantly used by the bfloat16 floating-point format article appears to be Citation Style 1. The citation style used in the edit request consists of bare URL's.[a] Any requested edit of yours which may be implemented will need to resemble the current style already in use in the article – in this case, CS1. (See WP:CITEVAR.)
  2. Citation ref tags have not been placed within the requested text indicating which portions of the text the source is referencing. (See WP:INTEGRITY.)
  3. The ref note for the Dillon source is only used once, and it is placed in a part of the sentence which confirms no information apart from the fact that bloat16 was once a storage format: "When bloat16 was first introduced as a storage format, ....." The reference entry for Dillon (the only one out of four from the request that uses citation style #1) is also missing the |page= parameter.
  4. One of the web references provided, from Google, confirms 12% of the information provided. The other two web sources do not confirm the text provided. At this link you will see text from two websites The text on the right side of the pane is from the Google source that you provided as a reference. The text on the left side of the pane is from your edit request proposal. Besides the 12% which does match, kindly point out the text from the right side of the pane that matches the text in your proposal on the left side of the pane. The comparison of your edit request proposal with the nvidia source can be viewed here. The comparison of your request"s text with the community.arm source can be viewed here.
  5. Significant portions of text have no ref tags.

In the collapsed section below titled Request edit examples, I have illustrated two: The first shows how the edit request was submitted; the second shows how requests should be submitted in the future.

Request edit examples
INCORRECT


The Sun's diameter is 864,337.3 miles, while the Moon's diameter is 2,159 miles. The Sun's temperature is 5,778 Kelvin.

https://www.booksource.com
https://www.journalsource.com
https://www.websource.com

In the example above there are three URL's provided with the claim statements, but these URL's have not been placed using Citation Style 1, which is the style predominantly used by the bfloat16 floating-point format article. Additionally, ref tags have not been placed within the text at the exact positions where the information they reference resides. Using the correct style and the correct positioning of the ref tags, the WikiFormatted text should resemble the following:

CORRECT


The Sun's diameter is 864,337.3 miles,<ref>{{cite book|last1=Sjöblad|first1=Tristan|title=The Sun|url=http://www.booksource.com|publisher=Academic Press|date=2020|page=1}}</ref> while the Moon's diameter is 2,159 miles.<ref>{{cite journal|last1=Harinath|first1=Prisha|title=Size of the Moon|journal=Science|issue=78|volume=51|url=http://www.journalsource.com|date=2020|page=46}}</ref> The Sun's temperature is 5,778 Kelvin.<ref>{{cite book|last1=Uemura|first1=Shu|title=The Sun's Heat|url=http://www.websource.com|publisher=Academic Press|date=2020|page=2}}</ref>

Which displays as:


The Sun's diameter is 864,337.3 miles,[1] while the Moon's diameter is 2,159 miles.[2] The Sun's temperature is 5,778 Kelvin.[3]

References

  1. ^ Sjöblad, Tristan. The Sun. Academic Press, 2020, p. 1.
  2. ^ Harinath, Prisha. "Size of the Moon", Science, 51(78):46. 2020.
  3. ^ Uemura, Shū. The Sun's Heat. Academic Press, 2020, p. 2.

In the example above the references have been formatted according to Citation Style 1, which shows the author, the source's name, date, etc. Also, the ref tags are placed in the exact location where the text which they reference resides. As Wikipedia is a volunteer project, edit requests such yours are generally expected to have this formatting done before the request is submitted for review.

Kindly rewrite your edit request so that it aligns more with the second example shown in the collapsed section above, and feel free to re-submit that edit request at your earliest convenience. If you have any questions about this formatting please don't hesitate to ask myself or another editor. Regards,  Spintendo  22:53, 18 July 2023 (UTC)[reply]

Notes

  1. ^ The use of bare URLs as references is a style which is acceptable for use in Wikipedia. However, general practice dictates that the style already in use for an article be the one that is subsequently used for all future additions unless changed by editorial consensus.

References

  1. ^ Joshua V. Dillon, Ian Langmore, Dustin Tran, Eugene Brevdo, Srinivas Vasudevan, Dave Moore, Brian Patton, Alex Alemi, Matt Hoffman, Rif A. Saurous (2017-11-28). TensorFlow Distributions (Report). p. 3. arXiv:1711.10604. Bibcode:2017arXiv171110604D. Accessed 2018-05-23. All operations in TensorFlow Distributions are numerically stable across half, single, and double floating-point precisions (as TensorFlow dtypes: tf.bfloat16 (truncated floating point), tf.float16, tf.float32, tf.float64). Class constructors have a validate_args flag for numerical asserts{{cite report}}: CS1 maint: multiple names: authors list (link)
  2. ^ a b "The bfloat16 numerical format". Google Cloud. Retrieved 2023-07-11. On TPU, the rounding scheme in the conversion is round to nearest even and overflow to inf.
  3. ^ "BFloat16 processing for Neural Networks on Armv8-A". community.arm.com. Retrieved 2023-07-11. Only one rounding mode is supported, the non-IEEE Round to Odd mode.
  4. ^ "Arm A64 Instruction Set Architecture". developer.arm.com. Retrieved 2023-07-26. Uses the non-IEEE Round-to-Odd rounding mode.
  5. ^ "1.3.5. Bfloat16 Precision Conversion and Data Movement" (PDF). docs.nvidia.com. p. 199. Retrieved 2023-07-26. Converts float number to nv_bfloat16 precision in round-to-nearest-even mode and returns nv_bfloat16 with converted value.
Thanks for the advices. I've addressed the comments:
For #3 regarding the Dillon source, it is also used in the original article, so the final change will be using the already existing citation (and the original article doesn't have the |page= parameter. 2620:15C:2D1:206:95E1:3368:FE14:B48E (talk) 21:24, 19 July 2023 (UTC)[reply]
@2620:15C:2D1:206:95E1:3368:FE14:B48E Thank you for your reply. To add the information, please provide the source that you stated was already in the article that can be used as a reference for the proposed information here on the talk page. Also, if no page number is provided, please use the |quote= parameter to give the information from the source that confirms the proposed information here. Thank you! Regards,  Spintendo  05:47, 25 July 2023 (UTC)[reply]
All the references, including the Dillon source, already have the "|quote=" parameter. The information of the Dillon source is also in the quote: "tf.bfloat16 (truncated floating point)". 2620:15C:2D1:206:C5D:D294:1BF7:E3B2 (talk) 16:57, 25 July 2023 (UTC)[reply]
The Dillon source appears to be available as a pdf (which contains page numbers.) The docs.nvidia.com source also appears to be available in pdf format (with page numbers). The community.arm.com source lists this reference and this reference as its sources. Per WP:SWYGT, if a source is getting its information from another source, both (or just the original) should be listed as the reference of record. That would be preferred to listing community.arm.com, which is a blog. WP:BLOGS states "Self-published expert sources may be considered reliable when produced by an established subject-matter expert, whose work in the relevant field has previously been published by reliable, independent publications". It is not known if Nigel Stephens, the author of the community.arm.com source, fits that description. Regards,  Spintendo  23:05, 25 July 2023 (UTC)[reply]
Add |page= in both the Dillon and the nvidia source.
Nigel Stephens is an ARM employee (https://uk.linkedin.com/in/nigelstephens) talking about ARM's product (specific hardware instructions can be found in ARM's product specification). Anyway, the ARM source is updated with the product specification from developer.arm.com. 2620:15C:2D1:206:B191:EE78:6202:30BF (talk) 19:51, 26 July 2023 (UTC)[reply]
Thank you for your reply. To use Stephens' blog as something coming from an expert, his work in the relevant field would have to of been previously published by reliable, independent publications. Would you know which one's these would be? Also, the information after letter (b) is not referenced. Regards,  Spintendo  17:28, 31 July 2023 (UTC)[reply]
The blog is no longer used. The reference changes to the product specification from developer.arm.com.
What is the information after letter (b)? 73.162.81.124 (talk) 19:11, 31 July 2023 (UTC)[reply]
From your edit request: a. From IEEE754 binary32 (32-bit floating point) to bfloat16. When bfloat16 was first introduced as a storage format, the conversion from IEEE754 binary32 (32-bit floating point) to bfloat16 is truncation (round-to-zero). Later on, when it becomes the input of matrix multiplication units, the conversion can have various rounding mechanisms depending on the hardware platforms. For example, for Google TPU, the rounding scheme in the conversion is round-to-nearest-even ARM uses the non-IEEE Round-to-Odd rounding mode. for NVIDIA, it supports converting float number to bfloat16 precision in round-to-nearest-even mode
b. From bfloat16 to IEEE754 binary32 (32-bit floating point)
Also, you said the blog is no longer used, the reference changes to the product discussion. It doesn't matter if the blog is no longer used or no longer published by the blogger, It was used as a reference in the request. Also the reference changes references aren't like .gif images that change appearance, they are what they are entered to be. So I'm not entirely sure what that means. Please clarify.  Spintendo  07:44, 7 August 2023 (UTC)[reply]
1. The reference for (b) is added in the edit request. 2. The reference for the ARM source has been updated: instead of a blog, it is now pointing to a product specification. Since the reference is no longer based on a blog, do I still need to follow guidelines in WP:BLOGS?
FYI, the reply button doesn't work anymore, the page asks me to edit directly. — Preceding unsigned comment added by 73.162.81.124 (talk) 17:08, 7 August 2023 (UTC)[reply]
@73.162.81.124 Where has it been updated? Please note that any changes you make to previous talk page posts of yours need to be following the guidelines according to WP:REDACT.  Spintendo  17:54, 7 August 2023 (UTC)[reply]
thanks for pointing me to the guidelines. I've tried my best to capture the changes in the talk edit request.
The talk edit request should have the updated parts now. 2620:15C:2D1:206:2560:20C8:AF:4692 (talk) 17:18, 8 August 2023 (UTC)[reply]
to clarify: I've tried my best to capture the *past* changes in the talk edit request that otherwise wasn't being captured. 2620:15C:2D1:206:B5D3:2F44:C0C2:7170 (talk) 21:39, 8 August 2023 (UTC)[reply]
Retrieved from "https://wiki.riteme.site/w/index.php?title=Talk:Bfloat16_floating-point_format&oldid=1236032364"