std1.0.0-beta-7 SafeC docs

std :: ml::lora

Struct std::LoraMlp

#include <std/ml/lora.h> source

struct LoraMlp {
    &Mlp base;
    struct Vec<&FloatTensor> A;
    struct Vec<&FloatTensor> B;
    struct LoraConfig cfg;
    float scale;
    struct Rng rng;
}

Borrows the base Mlp and freezes its parameters (requiresGrad = 0) until LoraMlp::detach; the base must outlive the adapter.

Fields

§&Mlp basesource
§struct Vec<&FloatTensor> Asource

per layer [in, r]

§struct Vec<&FloatTensor> Bsource

per layer [r, out]

§struct LoraConfig cfgsource
§float scalesource

alpha / r

§struct Rng rngsource

Methods

§&FloatTensor forward(const &FloatTensor x)source · definition
§void collect_params(&Vec<&FloatTensor> out) constsource · definition

Parameterized (module.h): the adapters (A_0, B_0, A_1, B_1, ...), which are all that trains and all a LoRA file holds.

§void collect_state(&Vec<&FloatTensor> out) constsource · definition
§void merge()source · definition

W += scale * A B in the base for every layer, then B = 0: the base alone now computes what the adapted network did.

§void detach()source · definition

Frees the adapters and unfreezes the base.

§static struct LoraMlp attach(&Mlp base, const &LoraConfig cfg)source · definition