NVIDIA A10 validation
This report records Noctaya's physical NVIDIA A10 result. Use the hardware validation requirements for shared evidence and claim rules.
Result
| Device | Level | Observed lifecycle | Date |
|---|---|---|---|
| Two NVIDIA A10 GPUs, 24 GB each | Scale-to-zero and coexistence verified | 0 → 1 → 2 → 0 | 2026-07-19 |
The run covered whole-GPU allocation, KEDA external-push activation, Volcano placement, model prewarming and caching, bounded admission, draining, recovery, and coexistence with a Kthena-managed hot model.
Validated stack
| Item | Recorded value |
|---|---|
| Host | x86-64 Ubuntu 22.04; 32 vCPU; 125 GiB RAM |
| Accelerators | Two NVIDIA A10 GPUs; 23,028 MiB each; PHB topology; no NVLink |
| NVIDIA stack | Driver 550.90.07; Container Toolkit 1.19.1; device plugin 0.19.3 |
| Allocation | Whole GPU through nvidia.com/gpu; no MIG or time-slicing |
| Kubernetes | K3s v1.36.2+k3s1; Helm 4.2.3; KEDA 2.20.1 |
| Scheduling | Volcano 1.15.0; queue noctaya-longtail |
| Coexistence | Kthena 1.0.0 in a separate Volcano queue |
| Runtime | vllm/vllm-openai:v0.25.1; internal registry mirror used during the run |
| Noctaya | Operator and gateway builds from v0.3.0-rc.1; external-push scaler |
| Model | Qwen/Qwen2.5-7B-Instruct; --max-model-len=4096; GPU utilization 0.9 |
| Cache | 30 GiB NodeLocalPVC on a dedicated 120 GB ext4 data disk; prewarming enabled |
| Scaling | Queue target temporarily set to 1; maximum temporarily raised from 1 to 2; drain timeout 120s |
The checked-in profile is scheduler-neutral and defaults to one replica. Volcano, the lower queue target, and the second replica were validation-only settings.
Observed evidence
| Check | Result |
|---|---|
| Device allocation | The plugin advertised two GPUs; a CUDA matrix operation passed in the vLLM image. |
| Prewarm and cache | Prewarm completed without a GPU in 6m43s; about 15 GB across 18 files remained unchanged after reboot. |
| Cold start | A cold streaming request returned heartbeats, real tokens, [DONE], and HTTP 200 in 91.876s. |
| Scale-out | Six pending requests produced two Ready Pods on distinct GPU UUIDs, then 2 → 1 → 0; both Pods served HTTP 200. |
| Coexistence | Noctaya and Kthena each generated 512 tokens on separate GPUs in about 16.6s; both returned HTTP 200. |
| Admission | A 105-client burst admitted 100 requests and returned five HTTP 429 responses; reject mode returned HTTP 503 with Retry-After: 10. |
| Drain | A stream completed with HTTP 200 and [DONE] after its backend Pod was deleted. |
| Metrics | vLLM exposed KV-cache, waiting-request, running-request, and time-to-first-token metrics. |
| Recovery | Helm upgrade, component replacement, and a host reboot converged without losing cache data or GPU capacity. |
| Sharing boundary | A fractional GPU request was rejected before workload creation, as required by the whole-GPU profile. |
Reproduce the core path
Install the NVIDIA runtime and device plugin using the Container Toolkit, K3s, and device-plugin documentation. Install KEDA independently; Noctaya configures External Push as described in Getting started.
Capture the host and cluster baseline:
uname -a
cat /etc/os-release
nvidia-smi
nvidia-container-cli --version
kubectl version
kubectl get crd scaledobjects.keda.sh
Confirm whole-GPU capacity and the exact product label:
kubectl get nodes \
-o custom-columns='NAME:.metadata.name,GPUS:.status.allocatable.nvidia\.com/gpu'
kubectl label node <a10-node> nvidia.com/gpu.product=NVIDIA-A10 --overwrite
kubectl get node <a10-node> -L nvidia.com/gpu.product
From the repository root, deploy the profile:
export NAMESPACE=noctaya-a10-validation
export SERVICE=qwen2-5-7b-a10
kubectl create namespace "$NAMESPACE" --dry-run=client -o yaml | kubectl apply -f -
kubectl apply -k examples/nvidia/a10 -n "$NAMESPACE"
kubectl get llmservice,pvc,job,deploy,pod,scaledobject -n "$NAMESPACE" -w
On a dedicated two-A10 host, reproduce scale-out with the recorded validation settings:
kubectl patch llmservice "$SERVICE" -n "$NAMESPACE" --type merge \
-p '{"spec":{"scaling":{"max":2,"target":1}}}'
Expose the stable gateway and send a cold streaming request:
kubectl port-forward -n "$NAMESPACE" "service/$SERVICE" 8080:80
curl -N http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d "{\"model\":\"$SERVICE\",\"stream\":true,\"messages\":[{\"role\":\"user\",\"content\":\"Reply with: Noctaya A10 validation passed\"}]}"
Retain the transition to two Ready Pods, their distinct GPU UUIDs, the completed response, and the eventual return to zero. To reproduce the Volcano result, set the runtime scheduler to volcano and queue to noctaya-longtail before creating the LLMService; the prewarm Job is create-once.
Claim boundary
- The result applies to two whole NVIDIA A10 GPUs on the recorded stack.
- It does not validate MIG, time-slicing, HAMi sharing, or another NVIDIA model.
- The Kthena and Volcano results validate coexistence and placement, not ownership of those platforms' lifecycles.