METAL

C++ inference engine for LLMs. GGUF format, quantization, CPU and Metal/CUDA support. The foundation most local tools build on.

Apple's array framework for machine learning on Apple Silicon. Native Metal support, unified memory, first-class LLM inference.