Struct std::XLSTMCell

struct XLSTMCell {
    &FloatTensor Wz;
    &FloatTensor Rz;
    &FloatTensor bz;
    &FloatTensor Wi;
    &FloatTensor Ri;
    &FloatTensor bi;
    &FloatTensor Wf;
    &FloatTensor Rf;
    &FloatTensor bf;
    &FloatTensor Wo;
    &FloatTensor Ro;
    &FloatTensor bo;
}

The scalar-memory sLSTM cell — xLSTM's headline idea is exponential gating (replacing LSTM's sigmoid input/forget gates with unbounded exp(), which can represent much larger gate values) made numerically stable with a running log-domain stabilizer state 'm_t', plus a normalizer state 'n_t' that keeps the exponential gates' magnitude in check without clipping. This is the scalar-memory (sLSTM) member of the xLSTM family specifically — mLSTM (matrix-memory, fully parallelizable) is a separate, larger cell not implemented here. z_t = tanh(x_t@Wz + h@Rz + bz) cell input i_raw = x_t@Wi + h@Ri + bi; f_raw = x_t@Wf + h@Rf + bf (pre-activation logits) o_t = sigmoid(x_t@Wo + h@Ro + bo) m_t = max(f_raw + m_{t-1}, i_raw) stabilizer (log-domain) i_t = exp(i_raw - m_t); f_t = exp(f_raw + m_{t-1} - m_t) c_t = f_t * c_{t-1} + i_t * z_t n_t = f_t * n_{t-1} + i_t h_t = o_t * (c_t / n_t)