Function std::mps_batch_begin

Every mps_*_f32 call above independently creates its own command buffer, dispatches, and blocks on waitUntilCompleted before returning -- correct, but for a chain of several ops (e.g. matmul -> relu -> matmul in a forward pass), paying one full command-buffer-submission + host/device sync round trip PER OP is the dominant cost once shader compilation is already cached (see mps_matmul_f32's implementation comment), especially for small workloads where dispatch latency swamps actual compute time.

mps_batch_begin()/mps_batch_end() let a caller wrap several mps_*_f32 calls so they share ONE command buffer and ONE sync point:

mps_batch_begin();
mps_matmul_f32(x, w1, h, ...);    // encoded onto the shared buffer,
                                   // NOT dispatched yet -- 'h' is not
                                   // valid at this point
mps_relu_f32(h, r, ...);          // reads 'h' — safe: this dispatch
                                   // is encoded AFTER matmul's on the
                                   // same buffer, so Metal orders it
                                   // after even though the CPU
                                   // hasn't waited yet
mps_matmul_f32(r, w2, y, ...);
mps_batch_end();                   // commits once, waits once, THEN
                                    // every op's 'out' pointer above
                                    // becomes valid

Every batched op's 'out' buffer is invalid to read until mps_batch_end() returns — the same contract as reading a cudaMemcpyAsync destination before synchronizing.

IMPORTANT: that example is only safe because none of mps_matmul_f32's or mps_relu_f32's inputs above are themselves another batched op's still-pending output — 'x'/'w1'/'w2' are all already-populated, ordinary CPU float arrays. mps_matmul_f32/mps_relu_f32 always upload their input from a CPU float* into a fresh GPU buffer, so feeding one batched op's 'out' pointer into a later op's 'a'/'b' within the same unflushed batch silently reads whatever was in that CPU memory before the batch started (found and fixed via direct testing, not something caught by inspection: a naive matmul -> relu chain like this computed 0 instead of the real value). Use mps_matmul_f32_chained/ mps_relu_f32_chained (below) instead for that case — they take the device buffer a previous batched op already wrote to (MpsBuf::batch_last_output()) directly, skipping the CPU round trip entirely rather than reading stale CPU data.