Skip to main content

​
What is NexaAI Python SDK?

The NexaAI Python SDK provides a comprehensive API for on-device AI inference across multiple modalities. It supports:
  • Large Language Models (LLM): Text generation and conversation
  • Vision-Language Models (VLM): Multimodal understanding and generation
  • Embedder: Text vectorization and similarity computation
  • Reranker: Document reranking
  • ASR (Automatic Speech Recognition): Speech-to-text transcription
  • CV (Computer Vision): OCR/text recognition

​
Choose Your Platform

The SDK supports multiple platforms with optimized backends. Select your platform for detailed setup instructions:

macOS

Apple Silicon optimized
Python 3.10 • MLX backend • Metal acceleration

Windows x64

CPU/GPU acceleration
Python 3.10 • GGUF models • CUDA support

Windows ARM64

NPU acceleration
Python 3.11-3.13 ARM64 • Snapdragon X Elite

​
Quick Overview

​
Installation

Each platform has specific installation requirements. Follow your platform guide for detailed instructions:
  • macOS: pip install 'nexaai[mlx]'
  • Windows x64: pip install nexaai
  • Windows ARM64: pip install nexaai

​
Authentication

Set up your NexaAI token from https://sdk.nexa.ai/: 
# Linux/macOS
export NEXA_TOKEN="key/your_token_here"

# Windows
$env:NEXA_TOKEN="key/your_token_here"

​
Basic Usage

Here’s a simple example to get you started: 
from nexaai.llm import LLM, GenerationConfig
from nexaai.common import ModelConfig, ChatMessage

# Initialize model (platform-specific)
model_name = "NexaAI/Qwen3-1.7B-4bit-MLX"  # Example for macOS
m_cfg = ModelConfig()
llm = LLM.from_(name_or_path=model_name, m_cfg=m_cfg)

# Create conversation
conversation = [
    ChatMessage(role="system", content="You are a helpful assistant."),
    ChatMessage(role="user", content="Hello, how are you?")
]

# Generate response
prompt = llm.apply_chat_template(conversation)
for token in llm.generate_stream(prompt, g_cfg=GenerationConfig(max_tokens=100)):
    print(token, end="", flush=True)
This is a simplified example. For complete setup instructions, model recommendations, and platform-specific optimizations, please refer to your platform guide above.

​
Next Steps

  1. Choose your platform and follow the detailed setup guide
  2. Explore the API Reference for comprehensive documentation
  3. Check out platform-specific examples in your chosen guide