Model serving (NVIDIA NIM & vLLM)

AlchemiStudio consumes an OpenAI-compatible model endpoint. On SUSE AI you have two validated ways to provide one, both keeping models inside your cluster.

Option 1 — NVIDIA NIM (GPU)

Use the SUSE AI Factory NVIDIA integration to serve a NIM model on a GPU node.

Validated: Llama 3.2 served on an NVIDIA T4 GPU node.

Outline:

1. Add a GPU node to the RKE2 cluster and enable the NVIDIA stack via SUSE AI Factory (AI Factory › Settings › NVIDIA).
2. Deploy the NIM model (through the AI Factory NVIDIA library / operator).
3. Point AlchemiStudio’s model configuration at the NIM endpoint.

GPU node notes:

• Size the OS disk for the model image (NIM images are large — 128 GB OS disk was needed for the T4 validation; the SLES default ~30 GB is too small).
• Ensure the NVIDIA container runtime / CDI is available after node reboots.

Option 2 — vLLM open model (no external registry)

Serve an open-weights model with vLLM — no NGC / Hugging Face / LiteLLM dependency at runtime.

Validated: Qwen2.5 served via a custom vLLM open-model Blueprint; AlchemiStudio’s AI service returned HTTP 200 against it.

Outline:

1. Deploy the vLLM open-model Blueprint (Fleet/AI Factory) onto a suitable node.
2. Point AlchemiStudio’s model configuration at the vLLM OpenAI-compatible endpoint.

Use this when you want a fully self-contained, open-model deployment.

Choosing

See Requirements for GPU prerequisites, or return to AlchemiStudio on SUSE AI Factory.