Docker Compose Installation¶

This guide covers deploying the complete Virtual Data Assistant (VDA) platform onto a single enterprise Linux host or virtual machine using Docker Compose.

Installation roadmap¶

Follow these pages in order:

Install & Launch (this page): prerequisites, configuration, and starting the platform.
Verify the Installation: health checks and the output you should see.
First-Time Setup: create the Keycloak realm, admin user, and roles, then sign in.
Connect a Datasource: add your first source and watch ingestion run.
LLM & Model Setup: choose the local or cloud model tier.
Air-Gapped Installation: install on a host with no internet access.
Backup, Restore & Upgrades: protect your data and move to new versions.

System Preparation & Prerequisites¶

Hardware Requirements¶

CPU: Minimum 8 Cores (x86_64 with AVX-512 recommended for GLiNER NER and vector embeddings)
Memory: Minimum 32 GB RAM (64 GB recommended for production workloads)
Disk: 250 GB NVMe or SSD storage
Operating System: Ubuntu 22.04/24.04 LTS, Red Hat Enterprise Linux 8/9, or SUSE Linux Enterprise Server 15

Software Prerequisites¶

Ensure Docker Engine 24.0+ and Docker Compose v2.20+ are installed:

# Verify Docker version
docker --version
docker compose version

Configure virtual memory and file descriptor limits required by search engines and vector stores:

# Set virtual memory limit (required for Qdrant and search indices)
sudo sysctl -w vm.max_map_count=262144
echo "vm.max_map_count=262144" | sudo tee -a /etc/sysctl.conf

# Set file descriptor limits
ulimit -n 65536

Port Allocation, Sizing & Network Exposure Matrix¶

The table below details every service in the VDA platform, listening ports, protocol, resource sizing, and strict network exposure rules.

Service	Container Name	Host Port	Container Port	Protocol	Network Exposure	Minimum Sizing	Production Sizing	Purpose
VDA Web UI	`vda-ui`	`8080`	`80`	HTTP	Public / Ingress	0.5 CPU, 512 MiB	2 CPU, 2 GiB	React SPA & Nginx reverse proxy
Keycloak SSO	`vda-keycloak`	`8081`	`8080`	HTTP	Public / Ingress	1 CPU, 1 GiB	2 CPU, 2 GiB	User authentication & OIDC identity provider
VDA GraphQL API	`vda-server`	`127.0.0.1:4000`	`4000`	HTTP	Internal Only	1 CPU, 2 GiB	2 CPU, 4 GiB	Node.js / Prisma core catalog API
Metadata Core Engine	`vda-metadata-api`	`127.0.0.1:8200`	`8200`	HTTP / SSE	Internal Only	2 CPU, 4 GiB	4 CPU, 8 GiB	FastAPI, Text-to-SQL, MCP SSE Server, PBAC
Metadata Worker	`vda-metadata-worker`	None	None	Worker	Internal Only	2 CPU, 4 GiB	4 CPU, 8 GiB	Temporal worker for GLiNER PII, profiling, embeddings
PostgreSQL	`vda-postgres`	`127.0.0.1:5432`	`5432`	TCP (PG Wire)	Strictly Private	1 CPU, 2 GiB	4 CPU, 8 GiB	Relational database for metadata, users, API keys
SurrealDB	`vda-surrealdb`	`127.0.0.1:8000`	`8000`	HTTP / WS	Strictly Private	1 CPU, 2 GiB	2 CPU, 4 GiB	Graph database for column and table lineage DAGs
Qdrant Vector DB	`vda-qdrant`	`127.0.0.1:6333`	`6333`	HTTP / gRPC	Strictly Private	1 CPU, 2 GiB	2 CPU, 4 GiB	Vector index for semantic catalog search
Temporal Server	`vda-temporal`	`127.0.0.1:7233`	`7233`	gRPC	Strictly Private	1 CPU, 1 GiB	2 CPU, 2 GiB	Durable distributed workflow orchestration
Temporal Web UI	`vda-temporal-ui`	`8480`	`8080`	HTTP	Admin VPN Only	0.25 CPU, 256 MiB	0.5 CPU, 512 MiB	Workflow execution monitoring interface
Ollama (Local LLM)	`vda-ollama`	`127.0.0.1:11434`	`11434`	HTTP	Strictly Private	4 CPU (or 1 GPU), 8 GiB	8 CPU (or 1 GPU), 16 GiB	Local air-gapped text-to-SQL LLM engine

Strict Network Security Rules

Never bind backend datastores to 0.0.0.0. In the provided docker-compose.yml, PostgreSQL, SurrealDB, Qdrant, Temporal gRPC, and Ollama are explicitly bound to 127.0.0.1 so they cannot be accessed outside the server.
Only the VDA Web UI (8080) and Keycloak SSO (8081) should receive external network traffic or be placed behind your enterprise TLS reverse proxy.
Access to the Temporal Web UI (8480) should be restricted to administrators via VPN or SSH tunneling.

Environment Configuration (`.env`)¶

Create an environment configuration file named .env in the repository root directory:

# =============================================================================
# Virtual Data Assistant (VDA) Deployment Configuration
# =============================================================================
GIT_SHA=latest
DOMAIN_NAME=vda.company.internal

# -----------------------------------------------------------------------------
# Database & Datastore Credentials (CHANGE ALL PASSWORDS FOR PRODUCTION)
# -----------------------------------------------------------------------------
POSTGRES_USER=vda
POSTGRES_PASSWORD=replace_with_secure_postgres_password
POSTGRES_DB=vda
SURREAL_USER=root
SURREAL_PASS=replace_with_secure_surreal_password
SURREAL_NS=vda
SURREAL_DB=vda

# -----------------------------------------------------------------------------
# Keycloak Identity & SSO Settings
# -----------------------------------------------------------------------------
KEYCLOAK_ADMIN=admin
KEYCLOAK_ADMIN_PASSWORD=replace_with_secure_keycloak_password
KEYCLOAK_REALM=vda
KEYCLOAK_CLIENTID=vda-client
# Public Keycloak URL accessed by user browsers
PUBLIC_KEYCLOAK_URL=http://localhost:8081

# -----------------------------------------------------------------------------
# AI Engine & LLM Configuration (Local Ollama vs. Cloud)
# -----------------------------------------------------------------------------
# For Local Air-Gapped Mode:
LLM_BASE_URL=http://ollama:11434/v1
LLM_MODEL=qwen2.5:7b
LLM_TIER=auto
EMBEDDING_MODEL=Alibaba-NLP/gte-multilingual-base

# For Cloud LLM Mode (Uncomment and populate if using external providers):
# LLM_BASE_URL=https://api.openai.com/v1
# LLM_MODEL=gpt-4o
# OPENAI_API_KEY=sk-proj-...

# -----------------------------------------------------------------------------
# PII Detection (GLiNER) & CPU Thread Bounds
# -----------------------------------------------------------------------------
ENABLE_GLINER=true
GLINER_MODEL_PATH=/models/gliner_final
OMP_NUM_THREADS=4
TORCH_NUM_THREADS=4
GLINER_NUM_THREADS=4

Launching the Platform¶

Start all platform services using Docker Compose:

# Start all containers in detached mode
docker compose --env-file .env up -d

Monitor service startup and dependency health checks:

# Check container statuses and healthchecks
docker compose ps

# Follow logs from the core metadata API
docker compose logs -f metadata-api

# Follow logs from the background worker
docker compose logs -f metadata-worker

All containers should report Up (or Up (healthy) for PostgreSQL).