Struct std::LayerNorm

#include <std/ml/norm.h> source

struct LayerNorm {
    &FloatTensor gamma;
    &FloatTensor beta;
    unsigned long dim;
    float eps;
}

Fields

§&FloatTensor gammasource

[dim], init 1

§&FloatTensor betasource

[dim], init 0

§unsigned long dimsource

§float epssource

1e-5

§&FloatTensor forward(const &FloatTensor x) constsource · definition

§void collect_params(&Vec<&FloatTensor> out) constsource · definition

Parameterized (module.h): the trained tensors, and what a copy or a file carries.

§void collect_state(&Vec<&FloatTensor> out) constsource · definition

§void free() consumesource · definition

= drop()

§static struct LayerNorm new(unsigned long dim)source · definition