floating point – NVIDIA Technical Blog News and tutorials for developers, data scientists, and IT admins 2025-07-25T02:22:39Z http://www.open-lab.net/blog/feed/ Karin Sevegnani <![CDATA[Floating-Point 8: An Introduction to Efficient, Lower-Precision AI Training]]> http://www.open-lab.net/blog/?p=101197 2025-06-12T18:50:43Z 2025-06-04T16:27:30Z With the growth of large language models (LLMs), deep learning is advancing both model architecture design and computational efficiency. Mixed precision...]]> With the growth of large language models (LLMs), deep learning is advancing both model architecture design and computational efficiency. Mixed precision...A decorative image.

With the growth of large language models (LLMs), deep learning is advancing both model architecture design and computational efficiency. Mixed precision training, which strategically employs lower precision formats like brain floating point 16 (BF16) for computationally intensive operations while retaining the stability of 32-bit floating-point (FP32) where needed, has been a key strategy for��

Source

]]>
0
Mark Harris <![CDATA[Everything You Ever Wanted to Know About Floating Point but Were Afraid to Ask]]> http://www.parallelforall.com/?p=1028 2024-05-04T00:19:42Z 2011-06-07T21:04:37Z This post was written by Nathan Whitehead A few days ago, a friend came to me with a question about floating point.?Let me start by saying that my friend knows...]]> This post was written by Nathan Whitehead A few days ago, a friend came to me with a question about floating point.?Let me start by saying that my friend knows...Diagram of fused multiply-add method for floating point.

This post was written by Nathan Whitehead A few days ago, a friend came to me with a question about floating point. Let me start by saying that my friend knows his stuff, he doesn��t ask stupid questions. So he had my attention. He was working on some biosciences simulation code and was getting answers of a different precision than he expected on the GPU and wanted to know what was up.

Source

]]>
2
���˳���97caoporen����