std :: ml::attention
Function std::mha_forward
&FloatTensor mha_forward(
const &FloatTensor Q,
const &FloatTensor K,
const &FloatTensor V,
unsigned long numHeads,
)Multi-head attention: splits Q/K/V's dModel columns into 'numHeads' equal headDim-wide slices (dModel must be divisible by numHeads), runs attention_forward() per head, and concatenates the per-head outputs back into one [seqLen, dModel] result — the standard MHA shape from "Attention Is All You Need", without a separate output-projection matrix (apply your own W_O via tensor_matmul on the result if needed).