Kubernetes Fundamentals for GenAI Deployment

Kubernetes is a container orchestration platform. It manages deployment, scaling, networking, self-healing, and rolling updates for containers. In production, it helps run GenAI APIs, agent services, retrieval services, caches, and supporting tools.

GenAI workloads may receive unpredictable traffic. Kubernetes can scale pods, restart failed services, route traffic, manage configuration, and support zero-downtime deployments.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: genai-api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: genai-api
  template:
    metadata:
      labels:
        app: genai-api
    spec:
      containers:
        - name: genai-api
          image: healthcare-genai-api:1.0
          ports:
            - containerPort: 8000
---
apiVersion: v1
kind: Service
metadata:
  name: genai-api-service
spec:
  selector:
    app: genai-api
  ports:
    - port: 80
      targetPort: 8000