Module ml::tensor_half
fp16 / bf16 tensors and mixed-precision ops.
SafeC has no native 16-bit float scalar (see float16.h's header comment), so half-precision tensors are carried the way float16.h carries single values: raw IEEE binary16 / bfloat16 bits in 'unsigned short', converted with float16.h's round-to-nearest-even routines.
Two layers:
-
HalfTensor (fp16) and BF16Tensor (bf16) — tensor.h's DenseTensor stored in 16 bits (half the memory of a FloatTensor). Every op, its autograd and the backward pass work on them the same as on a FloatTensor: the op loads its operands to float32, runs the float32 kernel and rounds its result once (tensor_matmul accumulates every dot product in float32, like tensor cores / Apple's simdgroup matrices do); gradients are float32. This header adds conversions between element types, the batch helpers mixed-precision training uses, and gemv for decoding with 16-bit weights.
-
Mixed precision on float32 tensors: FloatTensor::cast() rounds a tensor's values to fp16/bf16 precision in the forward pass and passes gradients through unchanged in the backward pass (the straight- through estimator), so a graph can compute in reduced precision while its master weights and gradients stay float32. tensor_matmul_lowp() is the common case built from it. LossScaler is fp16 training's dynamic loss scaling: scale the loss up so small gradients survive fp16's narrow range, then unscale and skip the step when any gradient overflowed. (bf16 shares float32's exponent range and doesn't need it.)
GPU half-precision compute lives in gpu_mps.h / gpu_cuda.h / gpu_webgpu.h (MpsBuf::upload_bits / upload_f16 etc.); a HalfTensor's or BF16Tensor's 'data' buffer is exactly the bit layout those functions read and write.
Structs
| LossScaler |
Functions
| half_round | Round 'v' to the nearest value representable in 'dtype' (returned as float32). Any dtype other than TENSOR_FP16/TENSOR_BF16 returns 'v'. |
| half_encode | |
| half_decode | |
| tensor_convert_into | dst's values = src's, rounded to dst's element type (any pair of FloatTensor / HalfTensor / BF16Tensor, including fp16 <-> bf16). Shapes may differ; sizes must match (0 otherwise). Not a graph node -- for that, see DenseTensor::to_float / from_float. |
| tensors_convert | Appends one converted copy of each tensor in 'src' to 'out' (heap, requiresGrad 0; free with tensors_free). |
| tensors_convert_into | Refresh copies made by tensors_convert, or write them back; 0 if the lists differ in length or any pair in size. |
| tensors_free | Frees every tensor in the list (DenseTensor::free and its own memory) and the list. |
| half_grads_accumulate | params[i].grad += grads[i] * invScale (allocating the gradient if needed). Returns 1 if every resulting gradient value is finite -- the LossScaler::update argument -- and 0 on an overflow or a size mismatch. |
| tensor_matmul_lowp | cast(cast(a) @ cast(b)): operands and result rounded to 'dtype', products accumulated in float32. Gradients flow to a and b in float32. |
Macros
| TENSOR_FP16 | |
| TENSOR_BF16 | |
| HALF_CHUNK |