​

NexaAI Python SDK API Reference

• macOS Guide - Apple Silicon optimization
• Windows x64 Guide - CPU/GPU acceleration
• Windows ARM64 Guide - NPU acceleration

​

Core Modules

​

LLM (Large Language Model)

​

Initialization

from nexaai import LLM, ModelConfig

# Initialize LLM
config = ModelConfig()
llm = LLM.from_(model="model_name_or_path", config=config, plugin_id="cpu_gpu", device_id="cpu")

​

Core Methods

• Generates text tokens in streaming fashion
• Parameters:
  • prompt (str): Input prompt
  • config (GenerationConfig, optional): Generation configuration
• Returns: Generator yielding text tokens, returns GenerateResult when exhausted

• Generates text with optional streaming token callback
• Parameters:
  • prompt (str): Input prompt
  • config (GenerationConfig, optional): Generation configuration
  • on_token (Callable[[str], bool], optional): Callback function for streaming tokens
• Returns: GenerateResult containing the generated text and profile data

• Applies chat template to conversation
• Parameters:
  • messages (List[LlmChatMessage]): Conversation history
  • tools (str, optional): Optional tool JSON string
  • enable_thinking (bool): Enable thinking mode
• Returns: Formatted prompt string

• Resets conversation state and clears KV cache

• Saves key-value cache to file
• Parameters:
  • path (str): File path to save cache

• Loads key-value cache from file
• Parameters:
  • path (str): File path to load cache

​

VLM (Vision Language Model)

​

Initialization

from nexaai import VLM, ModelConfig

# Initialize VLM
config = ModelConfig()
vlm = VLM.from_(model="model_name_or_path", config=config, plugin_id="cpu_gpu", device_id="")

​

Core Methods

• Generates text tokens in streaming fashion
• Parameters:
  • prompt (str): Input prompt
  • config (GenerationConfig, optional): Generation configuration with image_paths/audio_paths
• Returns: Generator yielding text tokens, returns GenerateResult when exhausted

• Generates text with optional streaming token callback
• Parameters:
  • prompt (str): Input prompt
  • config (GenerationConfig, optional): Generation configuration
  • on_token (Callable[[str], bool], optional): Callback function for streaming tokens
• Returns: GenerateResult containing the generated text and profile data

• Applies chat template to multimodal conversation
• Parameters:
  • messages (List[VlmChatMessage]): Multimodal conversation history
  • tools (str, optional): Optional tool JSON string
  • enable_thinking (bool): Enable thinking mode
• Returns: Formatted prompt string

• Resets conversation state and clears KV cache

​

ASR (Automatic Speech Recognition)

​

Initialization

from nexaai import ASR

# Initialize ASR
asr = ASR.from_(model="model_name_or_path", plugin_id="cpu_gpu", device_id="cpu")

​

Core Methods

• Transcribes audio file to text
• Parameters:
  • audio_path (str): Path to audio file
  • language (str, optional): Language code (“en”, “zh”, or None for auto-detect)
  • timestamps (str, optional): Timestamp format (“none”, “segment”, “word”)
  • beam_size (int): Beam size for decoding (default: 5)
• Returns: TranscribeResult containing transcript, confidence scores, timestamps, and profile data

• Lists supported languages
• Returns: List of language codes

• Begins streaming ASR transcription
• Parameters:
  • language (str, optional): Language code
  • on_transcription (Callable[[str], None], optional): Callback function for transcription results
  • chunk_duration (float): Audio chunk duration in seconds
  • overlap_duration (float): Overlap duration in seconds
  • sample_rate (int): Audio sample rate
  • max_queue_size (int): Maximum queue size
  • buffer_size (int): Buffer size
  • timestamps (str, optional): Timestamp format
  • beam_size (int): Beam size for decoding

• Pushes audio data for streaming transcription
• Parameters:
  • audio_data (List[float]): List of audio samples (float values)

• Stops streaming transcription
• Parameters:
  • graceful (bool): If True, wait for current processing to complete

​

Embedder

​

Initialization

from nexaai import Embedder

# Initialize Embedder
embedder = Embedder.from_(model="model_name_or_path", plugin_id="cpu_gpu")

​

Core Methods

• Generates embeddings for texts, tokens, or images
• Parameters:
  • texts (List[str], optional): List of text strings to embed
  • input_ids (List[List[int]], optional): List of token ID sequences (alternative to texts)
  • image_paths (List[str], optional): List of image file paths to embed
  • task_type (str, optional): Task type for embedding (e.g., “classification”, “retrieval”)
  • batch_size (int): Batch size for processing (default: 32)
  • normalize (bool): Whether to normalize embeddings (default: False)
  • normalize_method (str, optional): Normalization method (e.g., “l2”)
• Returns: EmbedResult containing embeddings and profile data

• Returns the dimension of embeddings
• Returns: Integer dimension size

​

Reranker

​

Initialization

from nexaai import Reranker

# Initialize Reranker
reranker = Reranker.from_(model="model_name_or_path", plugin_id="cpu_gpu")

​

Core Methods

• Reranks documents based on query relevance
• Parameters:
  • query (str): Search query
  • documents (List[str]): List of documents to rank
  • batch_size (int): Batch size for processing (default: 32)
  • normalize (bool): Whether to normalize scores (default: False)
  • normalize_method (str, optional): Normalization method
• Returns: RerankResult containing scores and profile data

​

CV (Computer Vision)

​

Initialization

from nexaai import CV

# Initialize CV Model
cv = CV.from_(model="model_name_or_path", capabilities=0, plugin_id="cpu_gpu")

​

Core Methods

• Performs inference on image
• Parameters:
  • input_image_path (str): Path to input image
• Returns: CVResult containing detection/classification results

​

Diarize

​

Initialization

from nexaai import Diarize

# Initialize Diarize
diarize = Diarize.from_(model="model_name_or_path", plugin_id="cpu_gpu", device_id="cpu")

​

Core Methods

• Performs speaker diarization on audio file
• Parameters:
  • audio_path (str): Path to the audio file
  • min_speakers (int, optional): Minimum number of speakers
  • max_speakers (int, optional): Maximum number of speakers
• Returns: DiarizeResult containing speech segments and metadata

​

TTS (Text-to-Speech)

​

Initialization

from nexaai import TTS

# Initialize TTS
tts = TTS.from_(model="model_name_or_path", plugin_id="cpu_gpu", device_id="cpu")

​

Core Methods

• Synthesizes text to speech
• Parameters:
  • text (str): Text to synthesize
  • output_path (str): Path to save the audio file
  • voice (str, optional): Voice identifier. If None, uses default
  • speed (float): Speech speed multiplier (default: 1.0)
  • seed (int): Random seed. -1 for random (default: -1)
  • sample_rate (int): Audio sample rate (default: 22050)
• Returns: SynthesizeResult containing audio file path and metadata

• Lists available voices
• Returns: List of voice identifiers

​

ImageGen

​

Initialization

from nexaai import ImageGen, ModelConfig

# Initialize ImageGen
config = ModelConfig()
imagegen = ImageGen.from_(model="model_name_or_path", config=config, plugin_id="cpu_gpu", device_id="cpu")

​

Core Methods

• Generates image from text prompt
• Parameters:
  • prompt (str): Text prompt for image generation
  • output_path (str): Path to save the generated image
  • negative_prompts (List[str], optional): List of negative prompts
  • height (int): Image height in pixels (default: 512)
  • width (int): Image width in pixels (default: 512)
  • method (str): Sampling method (e.g., ‘ddpm’, ‘ddim’) (default: ‘ddpm’)
  • steps (int): Number of diffusion steps (default: 50)
  • guidance_scale (float): Guidance scale for classifier-free guidance (default: 7.5)
  • eta (float): Eta parameter for DDIM (default: 0.0)
  • seed (int): Random seed. -1 for random (default: -1)
  • strength (float): Strength parameter (default: 1.0)
• Returns: ImageGenResult containing output image path

• Generates image from existing image and text prompt
• Parameters:
  • init_image_path (str): Path to the initial image
  • prompt (str): Text prompt for image generation
  • output_path (str): Path to save the generated image
  • negative_prompts (List[str], optional): List of negative prompts
  • height (int): Image height in pixels (default: 512)
  • width (int): Image width in pixels (default: 512)
  • method (str): Sampling method (e.g., ‘ddpm’, ‘ddim’) (default: ‘ddpm’)
  • steps (int): Number of diffusion steps (default: 50)
  • guidance_scale (float): Guidance scale for classifier-free guidance (default: 7.5)
  • eta (float): Eta parameter for DDIM (default: 0.0)
  • seed (int): Random seed. -1 for random (default: -1)
  • strength (float): Strength parameter (0.0-1.0) (default: 0.8)
• Returns: ImageGenResult containing output image path

​

Configuration Classes

​

ModelConfig

from nexaai import ModelConfig

config = ModelConfig()
# Default configuration is usually sufficient for most use cases

​

GenerationConfig

from nexaai import GenerationConfig

config = GenerationConfig(
    max_tokens=100,           # Maximum tokens to generate
    image_paths=["path/to/image.jpg"],  # Image paths for VLM (optional)
    audio_paths=["path/to/audio.wav"],  # Audio paths for VLM (optional)
)

​

Message Classes

​

LlmChatMessage

from nexaai import LlmChatMessage

message = LlmChatMessage(role="user", content="Hello, how are you?")

​

VlmChatMessage

from nexaai import VlmChatMessage, VlmContent

contents = [
    VlmContent(type="text", text="Describe this image"),
    VlmContent(type="image", text="path/to/image.jpg")
]
message = VlmChatMessage(role="user", contents=contents)

​

VlmContent

from nexaai import VlmContent

# Text content
text_content = VlmContent(type="text", text="Hello")

# Image content
image_content = VlmContent(type="image", text="path/to/image.jpg")

# Audio content
audio_content = VlmContent(type="audio", text="path/to/audio.wav")

​

Result Classes

​

GenerateResult

result = llm.generate(prompt, GenerationConfig(max_tokens=100))
print(result.full_text)  # Generated text
print(result.profile_data)  # Performance profiling data

​

TranscribeResult

result = asr.transcribe(audio_path="audio.wav")
print(result.transcript)  # Transcription text
print(result.confidence_scores)  # Confidence scores
print(result.timestamps)  # Timestamps
print(result.profile_data)  # Performance profiling data

​

EmbedResult

result = embedder.embed(texts=["Hello", "World"])
print(result.embeddings)  # List of embedding vectors
print(result.profile_data)  # Performance profiling data

​

RerankResult

result = reranker.rerank(query="query", documents=["doc1", "doc2"])
print(result.scores)  # List of relevance scores
print(result.profile_data)  # Performance profiling data

​

CVResult

result = cv.infer(image_path="image.jpg")
print(result.results)  # List of CVResultItem objects
for item in result.results:
    print(item.text)  # Detected text
    print(item.confidence)  # Confidence score
    print(item.bbox)  # Bounding box (if available)

​

DiarizeResult

result = diarize.infer(audio_path="audio.wav")
print(result.num_speakers)  # Number of speakers
print(result.duration)  # Audio duration
print(result.segments)  # List of SpeechSegment objects
for segment in result.segments:
    print(segment.start_time)  # Start time
    print(segment.end_time)  # End time
    print(segment.speaker_label)  # Speaker label

​

SynthesizeResult

result = tts.synthesize(text="Hello", output_path="output.wav")
print(result.audio_path)  # Path to generated audio file
print(result.duration_seconds)  # Audio duration
print(result.sample_rate)  # Sample rate
print(result.channels)  # Number of channels
print(result.num_samples)  # Number of samples

​

ImageGenResult

result = imagegen.txt2img(prompt="A beautiful sunset", output_path="output.png")
print(result.output_image_path)  # Path to generated image

​

Plugin ID Options

• cpu_gpu: Default, supports both CPU and GPU
• metal: Apple Silicon optimized (macOS)
• mlx: MLX backend (macOS)
• npu: NPU acceleration (Windows ARM64, Snapdragon X Elite)
• qnn: QNN backend
• nexaml: NexaML optimized backend
• llama_cpp: For GGUF format models
• onnx: ONNX runtime backend

​

Error Handling

• Invalid model paths
• Unsupported plugin/device combinations
• Authentication token issues
• Memory allocation failures
• Model loading errors

​

Performance Tips

1. Model Selection: Choose models optimized for your platform and use case
2. Batch Processing: Use appropriate batch sizes for embedding and reranking tasks
3. Memory Management: Monitor memory usage, especially for large models
4. Caching: Use KV cache for conversational applications
5. Profiling: Access profile_data from result objects to monitor performance

​

Next Steps

• Explore platform-specific examples in the macOS Guide
• Check out the Windows x64 Guide for CPU/GPU optimization
• Visit the Windows ARM64 Guide for NPU acceleration

Was this page helpful?

YesNo