βοΈ Prerequisites
- If you havenβt already, install the nexa-SDK.
- Below are the GGUF-compatible model types you can experiment with right away.
LLM - Language Models
π Language models in GGUF format. Try out this quick example: Try it out:bash
β¨οΈ Once model loads, type or paste multi-line text directly into the CLI to chat with the model.
LMM - Multimodal Models
πΌοΈ Language models that also accept vision and/or audio inputs. LMM in GGUF formats. Try out this quick example:bash
β¨οΈ Drag photos or audio clips directly into the CLI β you can even drop multiple images at once!
Supported Model List
We curated a list of top, high quality models in GGUF format.To try other GGUF models, visit Hugging Face, copy the path of any compatible GGUF model (e.g.,
unsloth/Qwen2.5-VL-3B-Instruct-GGUF
), and replace the model path in the command above.- Create an account at sdk.nexa.ai
- Generate a token: Go to Deployment β Create Token
- Activate your SDK: Run the following command in your terminal (replace with your token):
bash
π Request New Models
Want a specific model? Submit an issue on the nexa-sdk GitHub or request in our Discord/Slack community!Was this page helpful?