1  ---
  2  title: "Kubernetes"
  3  id: kubernetes
  4  slug: "/kubernetes"
  5  description: "Learn how to deploy your Haystack pipelines through Kubernetes."
  6  ---
  7  
  8  import ClickableImage from "@site/src/components/ClickableImage";
  9  
 10  # Kubernetes
 11  
 12  Learn how to deploy your Haystack pipelines through Kubernetes.
 13  
 14  The best way to get Haystack running as a workload in a container orchestrator like Kubernetes is to create a service to expose one or more [Hayhooks](../hayhooks.mdx) instances.
 15  
 16  ## Create a Haystack Kubernetes Service using Hayhooks
 17  
 18  As a first step, we recommend to create a local [KinD](https://github.com/kubernetes-sigs/kind) or [Minikube](https://github.com/kubernetes/minikube) Kubernetes cluster. You can manage your cluster from CLI, but tools like [k9s](https://k9scli.io/) or [Lens](https://k8slens.dev/) can ease the process.
 19  
 20  When done, start with a very simple Kubernetes Service running a single Hayhooks Pod:
 21  
 22  ```yaml
 23  kind: Pod
 24  apiVersion: v1
 25  metadata:
 26    name: hayhooks
 27    labels:
 28      app: haystack
 29  spec:
 30    containers:
 31      - image: deepset/hayhooks:v0.6.0
 32        name: hayhooks
 33        imagePullPolicy: IfNotPresent
 34        resources:
 35          limits:
 36            memory: "512Mi"
 37            cpu: "500m"
 38          requests:
 39            memory: "256Mi"
 40            cpu: "250m"
 41  
 42  ---
 43  
 44  kind: Service
 45  apiVersion: v1
 46  metadata:
 47    name: haystack-service
 48  spec:
 49    selector:
 50      app: haystack
 51    type: ClusterIP
 52    ports:
 53      # Default port used by the Hayhooks Docker image
 54      - port: 1416
 55  
 56  ```
 57  
 58  After applying the above to an existing Kubernetes cluster, a `hayhooks` Pod will show up as a Service called `haystack-service`.
 59  <ClickableImage src="/img/6eb9fb0c7b00367bfbe8182ffc7c3746f3f3d03b720e963df045e28160362d7f-Screenshot_2025-04-15_at_16.15.28.png" alt="Kubernetes Lens interface showing the hayhooks Pod running in the default namespace with status Running" />
 60  
 61  Note that the `Service` defined above is of type `ClusterIP`. That means it's exposed only _inside_ the Kubernetes cluster. To expose the Hayhooks API to the _outside_ world as well, you need a `NodePort` or `Ingress` resource. As an alternative, it's also possible to use [Port Forwarding](https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/) to access the `Service` locally.
 62  
 63  To do that, add port `30080` to Host-To-Node Mapping of our KinD cluster. In other words, make sure that the cluster is created with a node configuration similar to the following:
 64  
 65  ```yaml
 66  kind: Cluster
 67  apiVersion: kind.x-k8s.io/v1alpha4
 68  nodes:
 69    - role: control-plane
 70      # ...
 71      extraPortMappings:
 72        - containerPort: 30080
 73          hostPort: 30080
 74          protocol: TCP
 75  ```
 76  
 77  Then, create a simple `NodePort`  to test if Hayhooks Pod is running correctly:
 78  
 79  ```yaml
 80  apiVersion: v1
 81  kind: Service
 82  metadata:
 83    name: haystack-nodeport
 84  spec:
 85    selector:
 86      app: haystack
 87    type: NodePort
 88    ports:
 89    - port: 1416
 90      targetPort: 1416
 91      nodePort: 30080
 92      name: http
 93  ```
 94  
 95  After applying this, `hayhooks` Pod will be accessible on `localhost:30080`.
 96  
 97  From here, you should be able to manage pipelines. Remember that it's possible to deploy multiple different pipelines on a single Hayhooks instance. Check the [Hayhooks docs](../hayhooks.mdx) for more details.
 98  
 99  ## Auto-Run Pipelines at Pod Start
100  
101  Hayhooks can load Haystack pipelines at startup, making them readily available when the server starts. You can leverage this mechanism to have your pods immediately serve one or more pipelines when they start.
102  
103  At startup, it will look for deployed pipelines on the path specified at `HAYHOOKS_PIPELINES_DIR`, then load them.
104  
105  A [deployed pipeline](https://github.com/deepset-ai/hayhooks?tab=readme-ov-file#deploy-a-pipeline) is essentially a directory which must contain a `pipeline_wrapper.py` file and possibly other files. To preload an [example pipeline](https://github.com/deepset-ai/hayhooks/tree/main/examples/pipeline_wrappers/chat_with_website), you need to mount a local folder inside the cluster node, then make it available on Hayhooks Pod as well.
106  
107  First, ensure that a local folder is mounted correctly on the KinD cluster node at `/data`:
108  
109  ```yaml
110  kind: Cluster
111  apiVersion: kind.x-k8s.io/v1alpha4
112  nodes:
113    - role: control-plane
114      # ...
115      extraMounts:
116        - hostPath: /path/to/local/pipelines/folder
117          containerPath: /data
118  ```
119  
120  Next, make `/data` available as a volume and mount it on Hayhooks Pod. To do that, update your previous Pod configuration to the following:
121  
122  ```yaml
123  kind: Pod
124  apiVersion: v1
125  metadata:
126    name: hayhooks
127    labels:
128      app: haystack
129  spec:
130    containers:
131      - image: deepset/hayhooks:v0.6.0
132        name: hayhooks
133        imagePullPolicy: IfNotPresent
134        command: ["/bin/sh", "-c"]
135        args:
136          - |
137            pip install trafilatura && \
138            hayhooks run --host 0.0.0.0
139        volumeMounts:
140          - name: local-data
141            mountPath: /mnt/data
142        env:
143          - name: HAYHOOKS_PIPELINES_DIR
144            value: /mnt/data
145          - name: OPENAI_API_KEY
146            valueFrom:
147              secretKeyRef:
148                name: openai-secret
149                key: api-key
150        resources:
151          limits:
152            memory: "512Mi"
153            cpu: "500m"
154          requests:
155            memory: "256Mi"
156            cpu: "250m"
157    volumes:
158      - name: local-data
159        hostPath:
160          path: /data
161          type: Directory
162  
163  ```
164  
165  Note that:
166  
167  - We changed the Hayhooks container `command` to install `trafilaura` dependency before startup, since it's needed for our [chat_with_website](https://github.com/deepset-ai/hayhooks/tree/main/examples/pipeline_wrappers/chat_with_website) example pipeline. For a real production environment, we recommend creating a custom Hayhooks image as described [here](docker.mdx#customizing-the-haystack-docker-image).
168  - We make Hayhooks container read `OPENAI_API_KEY` from a Kubernetes Secret.
169  
170  Before applying this new configuration, create the `openai-secret`:
171  
172  ```yaml
173  apiVersion: v1
174  kind: Secret
175  metadata:
176    name: openai-secret
177  type: Opaque
178  data:
179    # Replace the placeholder below with the base64 encoded value of your API key
180    # Generate it using: echo -n $OPENAI_API_KEY | base64
181    api-key: YOUR_BASE64_ENCODED_API_KEY_HERE
182  ```
183  
184  After applying this, check your Hayhooks Pod logs, and you'll see that the `chat_with_website` pipelines have already been deployed.
185  <ClickableImage src="/img/2dbf42dd2db1cb355ee7222d7f8e96c45b611200d83ca289be3456264a854c38-Screenshot_2025-04-16_at_09.19.14.png" alt="Kubernetes Lens interface displaying pod logs with application startup messages and deployed pipeline confirmation" />
186  
187  ## Roll Out Multiple Pods
188  
189  Haystack pipelines are usually stateless, which is a perfect use case for distributing the requests to multiple pods running the same set of pipelines. Let's convert the single-Pod configuration to an actual Kubernetes `Deployment`:
190  
191  ```yaml
192  apiVersion: apps/v1
193  kind: Deployment
194  metadata:
195    name: haystack-deployment
196  spec:
197    replicas: 3
198    selector:
199      matchLabels:
200        app: haystack
201    template:
202      metadata:
203        labels:
204          app: haystack
205      spec:
206        initContainers:
207          - name: install-dependencies
208            image: python:3.12-slim
209            workingDir: /mnt/data
210            command: ["/bin/bash", "-c"]
211            args:
212              - |
213                echo "Installing dependencies..."
214                pip install trafilatura
215                echo "Dependencies installed successfully!"
216                touch /mnt/data/init-complete
217            volumeMounts:
218              - name: local-data
219                mountPath: /mnt/data
220            resources:
221              requests:
222                memory: "64Mi"
223                cpu: "100m"
224              limits:
225                memory: "128Mi"
226                cpu: "250m"
227        containers:
228          - image: deepset/hayhooks:v0.6.0
229            name: hayhooks
230            imagePullPolicy: IfNotPresent
231            command: ["/bin/sh", "-c"]
232            args:
233              - |
234                pip install trafilatura && \
235                hayhooks run --host 0.0.0.0
236            ports:
237              - containerPort: 1416
238                name: http
239            volumeMounts:
240              - name: local-data
241                mountPath: /mnt/data
242            env:
243              - name: HAYHOOKS_PIPELINES_DIR
244                value: /mnt/data
245              - name: OPENAI_API_KEY
246                valueFrom:
247                  secretKeyRef:
248                    name: openai-secret
249                    key: api-key
250            resources:
251              requests:
252                memory: "256Mi"
253                cpu: "250m"
254              limits:
255                memory: "512Mi"
256                cpu: "500m"
257        volumes:
258          - name: local-data
259            hostPath:
260              path: /data
261              type: Directory
262  
263  ```
264  
265  Implementing the above configuration will create three pods. Each pod will run a different instance of Hayhooks, all serving the same example pipeline provided by the mounted volume in the previous example.
266  
267  <ClickableImage src="/img/f3f0ac4b22a37039f0837c22b0cb8b640937bbb0db4acfcbdf7bd016b545d84a-Screenshot_2025-04-16_at_09.32.07.png" alt="Kubernetes Lens interface showing three haystack-deployment pods in Running status with their resource configurations" />
268  
269  Note that the `NodePort` you created before will now act as a load balancer and will distribute incoming requests to the three Hayhooks Pods.