Struct std::VqVaeConfig

#include <std/ml/autoencoder.h> source

struct VqVaeConfig {
    unsigned long hidden;
    unsigned long hiddenLayers;
    unsigned long latent;
    unsigned long codes;
    float commitment;
    unsigned long resetDeadAfter;
    int act;
    int recon;
    unsigned long seed;
}

van den Oord et al. 2017. The encoder output z_e is replaced by its nearest codebook vector z_q (a discrete code per row); gradients pass from z_q back to z_e unchanged (straight-through). Loss:

recon + ||sg(z_e) - z_q||^2 + commitment * ||z_e - sg(z_q)||^2

The codebook (codes x latent) is a trained parameter. Two standard fixes against codebook collapse: the first loss call initializes the codebook from that batch's encoder outputs, and a code no batch has picked for 'resetDeadAfter' calls is moved onto a random encoder output.

Fields

§unsigned long hiddensource

§unsigned long hiddenLayerssource

§unsigned long latentsource

code vector size

§unsigned long codessource

codebook size

§float commitmentsource

0.25 in the paper

§unsigned long resetDeadAftersource

0: never restart unused codes

§int actsource

§int reconsource

§unsigned long seedsource

Methods

§static struct VqVaeConfig default(unsigned long latent)source · definition

hidden 128, 1 hidden layer, 32 codes, commitment 0.25, dead codes restarted after 20 batches, ReLU, MSE.