Skip to navigation

Host Postgres on EKS for PrivateLink

Run Postgres with pgvector on EKS and expose it to Astropods over PrivateLink
View as Markdown

This page is a working sample for connecting a private database. It runs Postgres with pgvector on Amazon EKS, puts an internal NLB in front of it, and creates the VPC Endpoint Service that Astropods connects to. Adapt the names, sizes, and CIDRs to your environment.

Prerequisites

  • An EKS cluster with the AWS Load Balancer Controller installed.
  • The Amazon EBS CSI driver add-on, with an IAM role for its service account. A cluster with no stateful workloads yet often lacks it.
  • Private subnets in at least two Availability Zones.
  • kubectl access to the cluster and the AWS CLI for the cluster’s account.

The sample uses 10.0.0.0/16 as the VPC CIDR and playbook as the name. Replace both.

1

Create the namespace, storage, and password

postgres-storage.yaml
apiVersion: v1
kind: Namespace
metadata:
name: playbook
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: gp3
provisioner: ebs.csi.aws.com
parameters:
type: gp3
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: postgres-data
namespace: playbook
spec:
accessModes: [ReadWriteOnce]
storageClassName: gp3
resources:
requests:
storage: 50Gi
kubectl apply -f postgres-storage.yaml
kubectl -n playbook create secret generic postgres-auth \
--from-literal=POSTGRES_USER=agent \
--from-literal=POSTGRES_PASSWORD="$(openssl rand -base64 24)"

Skip the StorageClass if the cluster already has a gp3 class.

2

Deploy Postgres

postgres-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: postgres
namespace: playbook
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels: {app: postgres}
template:
metadata:
labels: {app: postgres}
spec:
containers:
- name: postgres
image: pgvector/pgvector:pg17
ports:
- containerPort: 5432
envFrom:
- secretRef: {name: postgres-auth}
env:
- {name: POSTGRES_DB, value: playbook}
- {name: PGDATA, value: /var/lib/postgresql/data/pgdata}
securityContext:
capabilities:
drop: [ALL]
add: [CHOWN, DAC_OVERRIDE, FOWNER, SETUID, SETGID]
readinessProbe:
exec:
command: [pg_isready, -U, agent, -d, playbook]
periodSeconds: 10
resources:
requests: {cpu: 500m, memory: 1Gi}
volumeMounts:
- {name: data, mountPath: /var/lib/postgresql/data}
volumes:
- name: data
persistentVolumeClaim:
claimName: postgres-data
kubectl apply -f postgres-deployment.yaml
kubectl -n playbook rollout status deploy/postgres

Three settings in this manifest matter:

  • strategy: Recreate. An EBS volume attaches to one node at a time. A rolling update starts the new pod before the old one releases the volume, so the new pod stays Pending.
  • The capabilities. The image’s entrypoint starts as root, changes the volume’s owner to the postgres user, then switches to that user. Dropping all capabilities breaks that step and the pod crash-loops. Don’t set runAsNonRoot or runAsUser for the same reason.
  • PGDATA points at a subdirectory. A new EBS volume contains lost+found, and Postgres refuses to initialize a non-empty directory.
3

Expose Postgres on an internal NLB

postgres-service.yaml
apiVersion: v1
kind: Service
metadata:
name: postgres
namespace: playbook
annotations:
service.beta.kubernetes.io/aws-load-balancer-type: external
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
service.beta.kubernetes.io/aws-load-balancer-scheme: internal
service.beta.kubernetes.io/load-balancer-source-ranges: 10.0.0.0/16,100.64.0.0/10
spec:
type: LoadBalancer
selector: {app: postgres}
ports:
- {name: postgres, port: 5432, targetPort: 5432}
kubectl apply -f postgres-service.yaml
kubectl -n playbook get svc postgres \
-o jsonpath='{.status.loadBalancer.ingress[0].hostname}'

The source ranges must include 100.64.0.0/10, because PrivateLink traffic reaches the NLB from that range. Keep the VPC CIDR for callers inside the VPC. The annotation key has no aws- prefix: the controller ignores aws-load-balancer-source-ranges and allows 0.0.0.0/0.

4

Create the VPC Endpoint Service

Look up the NLB’s ARN from the hostname the previous step printed, then create the endpoint service:

NLB_ARN=$(aws elbv2 describe-load-balancers \
--query "LoadBalancers[?DNSName=='<nlb-hostname>'].LoadBalancerArn" \
--output text)
aws ec2 create-vpc-endpoint-service-configuration \
--network-load-balancer-arns "$NLB_ARN" \
--acceptance-required

Note the ServiceId (vpce-svc-...) and the ServiceName (com.amazonaws.vpce.<region>.vpce-svc-...) in the output. With --acceptance-required, every connection request waits for you to approve it.

5

Allow Astropods to connect

In the dashboard, go to Knowledge > Add store and turn PrivateLink on. The form shows the principal to allow and the supported region to add. Set both on the endpoint service:

aws ec2 modify-vpc-endpoint-service-permissions \
--service-id <service-id> \
--add-allowed-principals <principal-from-dashboard>
aws ec2 modify-vpc-endpoint-service-configuration \
--service-id <service-id> \
--add-supported-regions <region-from-dashboard>
6

Connect the store and approve the request

Finish the Add store form with the ServiceName, port 5432, database playbook, and the credentials from step 1. You can use ast knowledge connect instead.

Astropods then requests a connection. Approve it:

aws ec2 describe-vpc-endpoint-connections \
--filters Name=service-id,Values=<service-id>
aws ec2 accept-vpc-endpoint-connections \
--service-id <service-id> \
--vpc-endpoint-ids <vpc-endpoint-id>

The store turns Ready once the connection’s VpcEndpointState is available.

7

Confirm the agent can query it

Deploy an agent that declares a postgres store and choose Shared for it on the deploy form. Ask the agent a question that only your data can answer. Ready confirms the endpoint exists, not that queries reach Postgres.

If queries hang and time out, the NLB is dropping PrivateLink traffic. Check the source ranges from step 3, then see Troubleshooting.

Next steps