Module ml::module

#include <std/ml/module.h> source

models as composable traits.

Every std::ml model is a struct with methods. The traits below name what a model can do, one capability each, and compose into the roles generic code asks for (trait composition: 'trait Module : Parameterized + Forward {}' requires both). The generic functions after them are written once and apply to every model with those capabilities; each call is resolved at compile time.

Parameterized  collect_params / collect_state: its tensors
Forward        forward(x)
Objective      loss(x, y): its own training loss (autoencoders, VAEs)
Module         Parameterized + Forward: Mlp, ConvNet, LoraMlp,
               QLoraMlp, LayerNorm, MultiHeadAttention, Mla, EgMla,
               JiTBlock, Mamba
LossModel      Parameterized + Objective: Autoencoder, Vae, VqVae,
               ConvAutoencoder, ConvVae
CondModule     Parameterized + forward(x, c): DiTBlock, JiT
Recurrent      Parameterized + forward(x_t, h): RNNCell, GRUCell
Generative     sample(n, labels): Gan, Dcgan, Vae, ConvVae

struct Mlp net = Mlp::new(2UL, 32UL, 2UL, 2UL, ACT_RELU, ACT_NONE, &rng);
struct Optimizer opt = Optimizer::new(OPT_ADAM, (float)0.01);
model_add_params(&net, &opt);
struct TrainReport rep = module_fit(&net, TRAIN_LOSS_CE, &opt, &data, &cfg);
float acc = module_accuracy(&net, &test, 64UL);
model_save_file(&net, "net.bin");

The same calls work on a ConvNet, a LoraMlp, a Mamba layer, ... and on a model of your own once it has the methods.

Traits

Parameterized
Forward
Objective
Module
LossModel
CondModule
Recurrent
Generative

Functions

model_add_params	Every trained tensor into o.
model_param_count	Trained values.
model_to	Places the whole state (parameters and buffers) on 'device' (device.h): the model then trains and runs there through the same calls. 1 when every tensor moved; an optimizer's moments follow its parameters on their next step. DEVICE_AUTO picks device_default().
model_device	Where the model's first tensor lives (DEVICE_CPU for a model without any).
model_sync	Makes every host copy of the model's state current (after device steps): before reading parameters directly or saving. 1 on success.
model_save_file	The state (collect_state) to a file in tensors_save's format, and back into a model built with the same shapes. 1 on success.
model_load_file
model_soft_update	dst = tau * src + (1 - tau) * dst over the whole state (same shapes): target networks, averaged generators. Returns 0 on a shape mismatch.
model_copy
module_loss	A built-in loss (TRAIN_LOSS_*) of m's output against y.
module_predict	forward(x) without recording gradients.
module_fit	trainer_fit with a built-in loss: the same batches, steps, lr decay and report. opt holds m's parameters (model_add_params).
module_evaluate	Mean built-in loss over a dataset, without gradients.
module_accuracy	Classification accuracy of m's logits over a dataset.
model_fit	trainer_fit on the model's own loss (an autoencoder's reconstruction, a VAE's ELBO, ...).
model_evaluate	Mean loss over a dataset, without gradients.
recurrent_run	Runs a cell over a sequence xs [T, inputSize] from h0 [1, hiddenSize] and returns the last hidden state [1, hiddenSize] (differentiable through every step).

Macros

TRAIN_LOSS_CE	Built-in losses for module_fit / module_loss.
TRAIN_LOSS_MSE	mean squared error (tensor_mse_loss)
TRAIN_LOSS_BCE	binary cross-entropy on logits (tensor_bce_logits_loss)