Noctaya documentation
Noctaya is a minimal, composable LLM serving control plane for private Kubernetes clusters. It turns an LLMService and a reusable InferenceRuntime into a cold-start-aware gateway, a model
backend that can scale to zero, optional model caching and prewarming, and KEDA autoscaling resources.
Start here
- Getting started covers installation, a first profile deployment, prewarming, request-time activation, and the return to zero.
- Examples and model configuration covers NVIDIA and Ascend profiles and explains how to change the served model.
- Architecture explains the component boundary, reconciliation output, and scale-to-zero lifecycle.
- CRD reference documents the
LLMServiceandInferenceRuntimeAPI.
Operate and validate
- Troubleshooting explains service conditions, activation failures, and recovery.
- Testing without an accelerator exercises the control plane and scale-to-zero path on Kind.
- Hardware validation explains the evidence required for physical accelerator claims.
Project
Read the roadmap for current limitations and v0.4.0 priorities. Contributions are welcome; the contributing guide explains the development workflow and project boundary.