Documentations
Join DiscordModel HubGitHub
  • Nexa On-Device AI Hub Overview
  • Getting Started
    • 🔗Installation
    • ▶️Use Model
    • ⬆️Upload Model
  • NEXA SDK
    • 📋CLI Reference
    • Python Interface
      • GGUF
      • ONNX
    • 🚀Inference
      • GGUF
      • ONNX
    • ⚙️Local server
  • Resources
    • ‼️Troubleshoot
Powered by GitBook
On this page
  • Prerequisites
  • Installation
  • CPU
  • GPU (Metal)
  • GPU (CUDA)
  • Choosing Between CPU and GPU Versions
  • Verification
  • Uninstallation

Was this helpful?

  1. Getting Started

Installation

Install Nexa SDK as a Python Package with CLI

PreviousNexa On-Device AI Hub OverviewNextUse Model

Last updated 7 months ago

Was this helpful?

Nexa SDK is a versatile toolkit that runs on multiple platforms (Linux, macOS, Windows) and supports various acceleration backends (CUDA, Metal, onnxruntime). This guide provides comprehensive instructions for installing Nexa SDK on your device.

Prerequisites

Before installation, ensure you have the following:

  • Python 3.7 or higher (Recommended)

  • pip/pip3 (Python package installer)

Installation

We have released pre-built wheels for various , platforms, and backends for convenient installation on our .

  1. If you want to use ONNX model, just replace pip install nexaai with pip install "nexaai[onnx]" in provided commands.

  2. If you want to upgrade to the latest version of Nexa SDK, just add -U/--upgrade after pip install in provided commands.

  3. For Chinese developers, we recommend you to use Tsinghua Open Source Mirror as extra index url, just replace --extra-index-url https://pypi.org/simple with --extra-index-url https://pypi.tuna.tsinghua.edu.cn/simple in provided commands.

CPU

pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/cpu --extra-index-url https://pypi.org/simple --no-cache-dir

GPU (Metal)

For Metal (macOS):

CMAKE_ARGS="-DGGML_METAL=ON -DSD_METAL=ON" pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/metal --extra-index-url https://pypi.org/simple --no-cache-dir

GPU (CUDA)

For Linux:

CMAKE_ARGS="-DGGML_CUDA=ON -DSD_CUBLAS=ON" pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/cu124 --extra-index-url https://pypi.org/simple --no-cache-dir

For Windows PowerShell:

$env:CMAKE_ARGS="-DGGML_CUDA=ON -DSD_CUBLAS=ON"; pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/cu124 --extra-index-url https://pypi.org/simple --no-cache-dir

For Windows Command Prompt:

set CMAKE_ARGS="-DGGML_CUDA=ON -DSD_CUBLAS=ON" & pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/cu124 --extra-index-url https://pypi.org/simple --no-cache-dir

For Windows Git Bash:

CMAKE_ARGS="-DGGML_CUDA=ON -DSD_CUBLAS=ON" pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/cu124 --extra-index-url https://pypi.org/simple --no-cache-dir

Choosing Between CPU and GPU Versions

it's important to choose the right version of Nexa SDK for your system. Nexa SDK is compatible with all operating systems.

  • CPU Version Choose this version if You don't have a compatible GPU.

  • GPU Version Choose this version if:

    • You have a compatible NVIDIA GPU (for CUDA acceleration on Linux/Windows)

    • You're using a Mac with Apple Silicon (M1, M2, or later chips) for Metal acceleration

    • You're planning to run computationally intensive tasks (image, vlm) that can benefit from GPU acceleration

nvcc --version

You can also install the required CUDA runtime libraries from PyPI distribution:

pip install nvidia-cuda-runtime-cu12 nvidia-cublas-cu12

Verification

After installation, you can verify the installation by running:

pip show nexaai

This should print the basic information of the installed Nexa SDK.

Uninstallation

To remove Nexa SDK and its dependencies, run the following command:

pip uninstall nexaai

The GPU version can significantly speed up certain operations, but requires additional setup and compatible hardware. For NVIDIA GPUs (Linux/Windows), ensure you have a CUDA-capable NVIDIA GPU and install . Check your CUDA version by running:

🔗
index page
CUDA Toolkit 12.0 or later