std :: ml::gpu_mps
Struct std::MpsBuf
struct MpsBuf {
?&MtlObject handle;
unsigned long bytes;
int dtype;
}Fields
Methods
handle != null
back to the buffer pool; handle := null
Copies the first dst.len (<= this.bytes) bytes into 'dst'. Valid once every batch writing the buffer has ended (shared memory: no transfer). 1 = done.
Copies src.len bytes (<= this.bytes) from 'src' into the buffer: valid while no batch that reads or writes it is in flight. 1 = done.
The buffer's memory as the CPU sees it (shared storage: the same bytes the GPU reads and writes), null for an invalid buffer or without Metal. Read or write it only while no batch using the buffer is in flight.
n uninitialized elements
n fp32 values rounded on the CPU (float16.h's f32_to_fp16_bulk / f32_to_bf16_bulk) and stored as fp16 / bf16.
n raw fp16 or bf16 values (dtype MPS_FP16 / MPS_BF16), copied as is.
static struct MpsBuf linear_attn_state(unsigned long numHeads, unsigned long headDim)source · definitionA zeroed recurrent linear-attention state (mps_linear_attn_bf16_chained): mps_linear_attn_state_bytes(numHeads, headDim) bytes of fp32.
The output of the op most recently encoded into the open batch (its bytes and dtype), invalid if no batch is active or nothing has been encoded yet. Call right after a batched op returns to chain its still-pending result into a following op without waiting for mps_batch_end(). The batch owns it: don't free() it; it returns to the pool when the batch ends.