In .NET 5 Preview 7, Microsoft has now introduced another floating-point datatype alongside the standard Float and Double. Dubbed Half, the new datatype is equivalent to binary16, which is specified in the IEEE 754 standard for floating-point arithmetic.
Unlike Float and Double in C#, which occupy 32 bits and 64 bits respectively, Half will be based on 16 bits. These bits will be split as follows: one sign bit, five exponent bits, and 10 mantissa bits. Taken together, Half can represent and store values in the range ±65504. This decrease in precision is where the datatype gets its name from in the first place. One rather strange implication of this format is that a Half can store both positive zero and negative zero:
- 1 00000 0000000000 = -0
- 0 00000 0000000000 = +0
Computations in machine learning take advantage of the decreased precision to speed up calculations and cut down on storage space. So, this would probably be the primary use case for a Half. Furthermore, you will be able to store a Half using a Float or Double, but the inverse won't be true without a loss of precision since you will be transitioning to lesser bits. Moreover, it is to be noted that:
In .NET 5.0, the Half type is primarily an interchange type with no arithmetic operators defined on it. It only supports parsing, formatting and comparison operators. All arithmetic operations will need an explicit conversion to a float/double. Future versions will consider adding arithmetic operators directly on Half.
It is expected that the Half will find its way into many codebases and will soon be supported in C# as a half datatype as it promises faster computation and efficiency in ML workflows. Further details can be found here.
1 Comment - Add comment