std1.0.0-beta-7 SafeC docs

std :: ml::gpu_cuda_attention

Function std::cuda_attention_use_cudnn

#include <std/ml/gpu_cuda_attention.h> source · definition

int cuda_attention_use_cudnn(int on)

fp16/bf16 flash attention uses cuDNN's fused attention engine when libcudnn.so.9 can be loaded (dlopen; no link dependency) and the GPU is sm_80 or newer, and this library's own kernels otherwise. on = 0 forces the own kernels. Returns 1 if cuDNN is in use after the call.