Module ml::device

devices: where a tensor's values live.

Every tensor has a device: DEVICE_CPU, or a GPU backend (DEVICE_ROCM for AMD, DEVICE_CUDA for NVIDIA, DEVICE_MPS for Apple Silicon, and the cross-vendor DEVICE_VULKAN and DEVICE_WEBGPU). An op runs on the device of its operands -- if any operand lives on a GPU, the op runs there, uploading the others -- and its result lives there too, so a model whose parameters are on a GPU trains on that GPU through the same code that trains it on the CPU. The backward pass runs each node where its forward pass ran. An op without a kernel on the device computes on the CPU (the operands are copied back first), so every op works on every device; the GPU only changes the speed.

Choosing a device:

explicitly  t.to(DEVICE_ROCM) (a copy, a graph node), p.place(DEVICE_ROCM)
            (moves a parameter in place), model_to(&net, DEVICE_ROCM),
            TrainConfig.device;
by default  device_set_default(d) for everything that asks for
            device_default();
automatically device_auto(): the first usable GPU among the backends the
            program was built with (ROCm, CUDA, Metal, then Vulkan and
            WebGPU), else the CPU. SAFEC_DEVICE=cpu|rocm|cuda|mps|vulkan|
            webgpu in the environment overrides it.

Backends are enabled at build time by the program's features (safeguard build --features rocm / cuda / mps / vulkan / webgpu, reaching the sources as SAFEC_FEATURE_ROCM, ..._CUDA, ..._MPS, ..._VULKAN, ..._WEBGPU); a program built without any runs everything on the CPU and links no GPU library. Link:

rocm  -lamdhip64 -lhiprtc -lrocblas (-lMIOpen for MIOpen convolutions)
cuda  -lcuda -lcublas -lnvrtc (-lcudnn for cuDNN convolutions)
mps   -framework Metal -framework Foundation -lobjc
vulkan -lvulkan (the Vulkan loader; on macOS Homebrew's vulkan-loader and
       molten-vk, -L/opt/homebrew/lib)

Each backend is a struct of static functions (the interface below), resolved at compile time; the runtime dispatch in device.sc calls them once the program's tensors say which device an op runs on.