std1.0.0-beta-7 SafeC docs

std :: ml::gpu_mps

Function std::mps_linear_attn_bf16_chained

#include <std/ml/gpu_mps.h> source · definition

int mps_linear_attn_bf16_chained(
    struct MpsBuf bufQ,
    struct MpsBuf bufK,
    struct MpsBuf bufV,
    struct MpsBuf bufState,
    &unsigned short out,
    unsigned long seqLen,
    unsigned long numHeads,
    unsigned long numKvHeads,
    unsigned long headDim,
)

Recurrent (causal) linear attention with the state kept on the GPU: 'bufState' holds S and z for every head, starts from the sequence so far and is advanced in place by these 'seqLen' tokens (k/v as [seqLen, ...]). Feed a prompt in one call, then each decode step with seqLen 1 -- O(headDim^2) per token, independent of how long the sequence already is. Must run inside an open batch; output as for mps_flash_attn_bf16_chained. Calls on the same state must stay in order, which a single batch (or batches ended in sequence) guarantees.