Documentations
Join DiscordModel HubGitHub
  • Nexa On-Device AI Hub Overview
  • Getting Started
    • 🔗Installation
    • ▶️Use Model
    • ⬆️Upload Model
  • NEXA SDK
    • 📋CLI Reference
    • Python Interface
      • GGUF
      • ONNX
    • 🚀Inference
      • GGUF
      • ONNX
    • ⚙️Local server
  • Resources
    • ‼️Troubleshoot
Powered by GitBook
On this page

Was this helpful?

  1. NEXA SDK

Inference

run model locally

This section provides detailed information about the nexaai commands used to run various models locally. These commands support text generation, image generation, and vision-language models, automatic speech recognition model, text-to-speech model etc.

$ nexaai -h 
usage: nexaai [-h] {onnx,gen-text,gen-image,vlm} ...

Nexa CLI tool for handling various model operations.

positional arguments:
  {onnx,gen-text,gen-image,vlm,pull,remove,list,login,whoami,logout}
                        sub-command help
    onnx                Run ONNX models for inference.
    gen-text            Run a GGUF model locally for text generation.
    gen-image           Run a GGUF model locally for image generation.
    vlm                 Run a GGUF model locally for VLM inference.
    asr                 Run a GGUF model locally for voice inference.

options:
  -h, --help            show this help message and exit

By default nexaai will run gguf models.

To run onnx models, simply add onnx afternexaai:

$ nexaai onnx -h 
usage: nexaai onnx [-h] {gen-text,gen-image,asr,tts} ...

positional arguments:
  {gen-text,gen-image,asr,tts}
                        ONNX sub-command help
    gen-text            Run ONNX model for text generation.
    gen-image           Run ONNX model for image generation.
    asr                 Run ONNX model for auto-speech-recognition.
    tts                 Run ONNX model for text-to-speech generation.

options:
  -h, --help            show this help message and exit
PreviousONNXNextGGUF

Last updated 8 months ago

Was this helpful?

Please check the page for more details.

🚀
ONNX