Struct std::LossScaler

#include <std/ml/tensor_half.h> source

struct LossScaler {
    float scale;
    float growthFactor;
    float backoffFactor;
    int growthInterval;
    int goodSteps;
}

Fields

§float scalesource

multiply the loss by this before Tensor::backward()

§float growthFactorsource

scale *= growthFactor after growthInterval clean steps

§float backoffFactorsource

scale *= backoffFactor after an overflow

§int growthIntervalsource

§int goodStepssource

clean steps since the last change

Methods

§int unscale(&FloatTensor p)source · definition

Call after Tensor::backward() on (loss * scale): divides p.grad by the scale in place and returns 1 if every gradient is finite, or 0 if any gradient overflowed (inf/NaN) — skip the optimizer step in that case.

§int unscale_all(&Vec<&FloatTensor> params)source · definition

LossScaler::unscale over every tensor in a list; 1 if all are finite.

§void update(int allFinite)source · definition

Call once per step with the combined result of LossScaler::unscale over all parameters (1 = all finite). Grows or backs off the scale.

§static struct LossScaler new()source · definition

scale 65536, growth 2x every 2000 clean steps, backoff 0.5x — the defaults PyTorch's GradScaler uses.