Introducing nexaSDK beta release (July 22). More updates coming soon.
Nexa SDK is an on-device inference framework that runs any model on any device, across any backend. It runs on CPUs, GPUs, NPUs with backend support for CUDA, Metal, Vulkan, and Qualcomm NPU. It handles multiple input modalities including text 📝, image 🖼️, and audio 🎧. The SDK includes an OpenAI-compatible API server with support for JSON schema-based function calling and streaming. It supports model formats such as GGUF, MLX, Nexa AI’s own .nexa format, enabling efficient quantized inference across diverse platforms.

​
Features

  • Device Support: CPU, GPU (CUDA, Metal, Vulkan), NPU (Qualcomm)
  • Input Type Support: Text, Image, Audio
  • Server: OpenAI-compatible API, JSON schema for function calling and streaming support
  • Model Format Support: GGUF, MLX

​
Get Started

Install nexaSDK on your operating system and run your first models on device.

Windows

Get started with NexaSDK on Windows

macOS

Optimized for Apple Silicon with MLX