std1.0.0-beta-7 SafeC docs

std :: ml::convnet

Function std::tensor_spectral_norm

#include <std/ml/convnet.h> source ยท definition

&FloatTensor tensor_spectral_norm(
    const &FloatTensor weight,
    &FloatTensor u,
    unsigned long iters,
    int training,
)

Spectral normalization (Miyato et al. 2018): weight / sigma, sigma its largest singular value (the weight seen as a matrix [rows, size/rows]), estimated by power iteration from u ([rows], no gradient). In training 'iters' iterations update u; in inference u is kept. The gradient treats u and v as constants, as PyTorch does.