Kubernetes & OpenShift Installation¶
This guide covers deploying the Virtual Data Assistant (VDA) in highly available, production-grade Kubernetes or Red Hat OpenShift clusters.
Installation roadmap¶
Follow these pages in order:
- Install & Launch (this page): prerequisites, configuration, and starting the platform.
- Verify the Installation: pod and endpoint health checks with expected output.
- First-Time Setup: configure Keycloak SSO and the first administrator.
- Connect a Datasource: add your first source in Kubernetes.
- LLM & Model Setup: deploy a local engine or use the cloud tier.
- Air-Gapped Installation: install on a disconnected cluster.
- Backup, Restore & Upgrades: cluster backup, restore, and rolling upgrades.
Production Architecture & Namespaces¶
VDA services are grouped under a dedicated namespace (vda), isolated with strict Kubernetes NetworkPolicies and resource quotas:
flowchart TD
subgraph IngressLayer["Ingress & Edge Routing"]
Ing["TLS Ingress / OpenShift Route<br/>(vda.company.internal)"]
end
subgraph StatelessApps["Core Application Deployments"]
UI["vda-ui (2 Replicas)"]
Server["vda-server (2 Replicas)"]
MetaAPI["metadata-api (2 Replicas)"]
Worker["metadata-worker (2+ Replicas)"]
end
subgraph Identity["Enterprise Identity"]
KC["Keycloak SSO (ClusterIP)"]
end
subgraph StatefulEngines["Stateful Data Engines (StatefulSets)"]
PG[("PostgreSQL (PVC 50Gi)")]
Surreal[("SurrealDB (PVC 50Gi)")]
Qdrant[("Qdrant Vector DB (PVC 50Gi)")]
Temp["Temporal Server (ClusterIP)"]
end
Ing -->|/| UI
Ing -->|/api| Server
Ing -->|/api/v2| MetaAPI
Ing -->|/auth| KC
UI --> Server & MetaAPI
Server --> PG & KC & Temp
MetaAPI --> Surreal & Qdrant & PG & Temp
Worker --> Surreal & Qdrant & PG & TempPod Sizing, Resource Limits & Concurrency Matrix¶
Configure pod resource requests and limits to guarantee predictable performance and prevent Linux Completely Fair Scheduler (CFS) throttling during peak profiling or vector search jobs:
| Deployment / Pod | Service Type | Port | Protocol | Network Exposure | CPU (Req / Limit) | Memory (Req / Limit) | Concurrency & Thread Variables |
|---|---|---|---|---|---|---|---|
vda-ui |
ClusterIP | 80 |
HTTP | Exposed via Ingress | 250m / 500m |
256Mi / 512Mi |
Nginx reverse proxy |
vda-server |
ClusterIP | 4000 |
HTTP | Exposed via Ingress (/api) |
1000m / 2000m |
2Gi / 4Gi |
NODE_OPTIONS="--max-old-space-size=4096" |
metadata-api |
ClusterIP | 8200 |
HTTP/SSE | Exposed via Ingress (/api/v2) |
2000m / 4000m |
4Gi / 8Gi |
OMP_NUM_THREADS=4, TORCH_NUM_THREADS=4 |
metadata-worker |
None (Worker) | None | Internal | Strictly Private | 2000m / 4000m |
4Gi / 8Gi |
GLINER_NUM_THREADS=4, TEMPORAL_QUEUE=all |
keycloak |
ClusterIP | 8080 |
HTTP | Exposed via Ingress (/auth) |
1000m / 2000m |
1Gi / 2Gi |
Quarkus runtime |
postgresql |
ClusterIP | 5432 |
TCP (PG) | Strictly Private (ClusterIP) | 1000m / 2000m |
2Gi / 4Gi |
Relational storage |
surrealdb |
ClusterIP | 8000 |
HTTP/WS | Strictly Private (ClusterIP) | 1000m / 2000m |
2Gi / 4Gi |
RocksDB storage engine |
qdrant |
ClusterIP | 6333 |
HTTP/gRPC | Strictly Private (ClusterIP) | 1000m / 2000m |
2Gi / 4Gi |
In-memory HNSW index |
temporal |
ClusterIP | 7233 |
gRPC | Strictly Private (ClusterIP) | 1000m / 2000m |
1Gi / 2Gi |
Distributed workflow engine |
temporal-ui |
ClusterIP | 8080 |
HTTP | Admin Port-Forward / VPN | 250m / 500m |
256Mi / 512Mi |
Workflow visualizer |
Network Exposure Rules
Databases (postgresql, surrealdb, qdrant), workflow gRPC (temporal), and background workers must never have external Ingress rules or NodePort exposure. Only vda-ui, /api (vda-server), /api/v2 (metadata-api), and the Keycloak authentication endpoint should be routed through the Ingress controller.
Kubernetes Manifest Structure (k8s/)¶
The repository includes declarative Kubernetes manifests organized into modular layers:
k8s/
├── 00-namespace.yaml # Creates the 'vda' namespace
├── config/
│ ├── configmap.yaml # Application topology, URLs, and thread bounds
│ └── secret.example.yaml # Secret template for passwords and tokens
├── storage/
│ ├── postgresql.yaml # PostgreSQL StatefulSet, Service & 50Gi PVC
│ ├── surrealdb.yaml # SurrealDB StatefulSet, Service & 50Gi PVC
│ ├── qdrant.yaml # Qdrant Vector DB StatefulSet, Service & 50Gi PVC
│ └── temporal.yaml # Temporal Server & Temporal UI Deployments
├── apps/
│ ├── metadata-api.yaml # Metadata Core Engine Deployment (2 replicas) & Service
│ ├── metadata-worker.yaml # Background Temporal Worker (2 replicas)
│ ├── vda-server.yaml # GraphQL Application Server (2 replicas) & Service
│ └── vda-ui.yaml # Web UI React SPA & Nginx (2 replicas) & Service
└── ingress.yaml # TLS Ingress with routing rules for UI and APIs
Step-by-Step Deployment Execution¶
Follow these steps to deploy VDA into your Kubernetes or OpenShift cluster using kubectl.
Step 1: Create Namespace¶
Step 2: Apply Configuration & Secrets¶
Customize credentials in k8s/config/secret.example.yaml and save as k8s/config/secret.yaml:
# Apply ConfigMap
kubectl apply -f k8s/config/configmap.yaml -n vda
# Apply Secrets (ensure passwords are changed from defaults)
kubectl apply -f k8s/config/secret.yaml -n vda
Step 3: Deploy Storage StatefulSets & Wait for Readiness¶
Deploy PostgreSQL, SurrealDB, Qdrant, and Temporal:
kubectl apply -f k8s/storage/ -n vda
# Wait for datastore pods to report Ready
kubectl wait --for=condition=ready pod -l app=postgresql -n vda --timeout=120s
kubectl wait --for=condition=ready pod -l app=surrealdb -n vda --timeout=120s
kubectl wait --for=condition=ready pod -l app=qdrant -n vda --timeout=120s
kubectl wait --for=condition=ready pod -l app=temporal -n vda --timeout=120s
Step 4: Deploy Core Applications & Workers¶
kubectl apply -f k8s/apps/ -n vda
# Monitor rollout status
kubectl rollout status deployment/metadata-api -n vda
kubectl rollout status deployment/metadata-worker -n vda
kubectl rollout status deployment/vda-server -n vda
kubectl rollout status deployment/vda-ui -n vda
Step 5: Apply Ingress & TLS Termination¶
Update the domain name in k8s/ingress.yaml to match your enterprise DNS:
Verify all pods across the namespace: