1  ---
  2  title: "Kubernetes"
  3  id: kubernetes
  4  slug: "/kubernetes"
  5  description: "Learn how to deploy your Haystack pipelines through Kubernetes."
  6  ---
  7  
  8  import ClickableImage from "@site/src/components/ClickableImage";
  9  
 10  # Kubernetes
 11  
 12  Learn how to deploy your Haystack pipelines through Kubernetes.
 13  
 14  The best way to get Haystack running as a workload in a container orchestrator like Kubernetes is to create a service to expose one or more [Hayhooks](../hayhooks.mdx) instances.
 15  
 16  ## Create a Haystack Kubernetes Service using Hayhooks
 17  
 18  As a first step, we recommend to create a local [KinD](https://github.com/kubernetes-sigs/kind) or [Minikube](https://github.com/kubernetes/minikube) Kubernetes cluster. You can manage your cluster from CLI, but tools like [k9s](https://k9scli.io/) or [Lens](https://k8slens.dev/) can ease the process.
 19  
 20  When done, start with a very simple Kubernetes Service running a single Hayhooks Pod:
 21  
 22  ```yaml
 23  kind: Pod
 24  apiVersion: v1
 25  metadata:
 26    name: hayhooks
 27    labels:
 28      app: haystack
 29  spec:
 30    containers:
 31      - image: deepset/hayhooks:v0.6.0
 32        name: hayhooks
 33        imagePullPolicy: IfNotPresent
 34        resources:
 35          limits:
 36            memory: "512Mi"
 37            cpu: "500m"
 38          requests:
 39            memory: "256Mi"
 40            cpu: "250m"
 41  
 42  ---
 43  
 44  kind: Service
 45  apiVersion: v1
 46  metadata:
 47    name: haystack-service
 48  spec:
 49    selector:
 50      app: haystack
 51    type: ClusterIP
 52    ports:
 53      # Default port used by the Hayhooks Docker image
 54      - port: 1416
 55  
 56  ```
 57  
 58  After applying the above to an existing Kubernetes cluster, a `hayhooks` Pod will show up as a Service called `haystack-service`.
 59  
 60  <ClickableImage src="/img/6eb9fb0c7b00367bfbe8182ffc7c3746f3f3d03b720e963df045e28160362d7f-Screenshot_2025-04-15_at_16.15.28.png" alt="None" />
 61  
 62  Note that the `Service` defined above is of type `ClusterIP`. That means it's exposed only _inside_ the Kubernetes cluster. To expose the Hayhooks API to the _outside_ world as well, you need a `NodePort` or `Ingress` resource. As an alternative, it's also possible to use [Port Forwarding](https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/) to access the `Service` locally.
 63  
 64  To do that, add port `30080` to Host-To-Node Mapping of our KinD cluster. In other words, make sure that the cluster is created with a node configuration similar to the following:
 65  
 66  ```yaml
 67  kind: Cluster
 68  apiVersion: kind.x-k8s.io/v1alpha4
 69  nodes:
 70    - role: control-plane
 71      # ...
 72      extraPortMappings:
 73        - containerPort: 30080
 74          hostPort: 30080
 75          protocol: TCP
 76  ```
 77  
 78  Then, create a simple `NodePort`  to test if Hayhooks Pod is running correctly:
 79  
 80  ```yaml
 81  apiVersion: v1
 82  kind: Service
 83  metadata:
 84    name: haystack-nodeport
 85  spec:
 86    selector:
 87      app: haystack
 88    type: NodePort
 89    ports:
 90    - port: 1416
 91      targetPort: 1416
 92      nodePort: 30080
 93      name: http
 94  ```
 95  
 96  After applying this, `hayhooks` Pod will be accessible on `localhost:30080`.
 97  
 98  From here, you should be able to manage pipelines. Remember that it's possible to deploy multiple different pipelines on a single Hayhooks instance. Check the [Hayhooks docs](../hayhooks.mdx) for more details.
 99  
100  ## Auto-Run Pipelines at Pod Start
101  
102  Hayhooks can load Haystack pipelines at startup, making them readily available when the server starts. You can leverage this mechanism to have your pods immediately serve one or more pipelines when they start.
103  
104  At startup, it will look for deployed pipelines on the path specified at `HAYHOOKS_PIPELINES_DIR`, then load them.
105  
106  A [deployed pipeline](https://github.com/deepset-ai/hayhooks?tab=readme-ov-file#deploy-a-pipeline) is essentially a directory which must contain a `pipeline_wrapper.py` file and possibly other files. To preload an [example pipeline](https://github.com/deepset-ai/hayhooks/tree/main/examples/pipeline_wrappers/chat_with_website), you need to mount a local folder inside the cluster node, then make it available on Hayhooks Pod as well.
107  
108  First, ensure that a local folder is mounted correctly on the KinD cluster node at `/data`:
109  
110  ```yaml
111  kind: Cluster
112  apiVersion: kind.x-k8s.io/v1alpha4
113  nodes:
114    - role: control-plane
115      # ...
116      extraMounts:
117        - hostPath: /path/to/local/pipelines/folder
118          containerPath: /data
119  ```
120  
121  Next, make `/data` available as a volume and mount it on Hayhooks Pod. To do that, update your previous Pod configuration to the following:
122  
123  ```yaml
124  kind: Pod
125  apiVersion: v1
126  metadata:
127    name: hayhooks
128    labels:
129      app: haystack
130  spec:
131    containers:
132      - image: deepset/hayhooks:v0.6.0
133        name: hayhooks
134        imagePullPolicy: IfNotPresent
135        command: ["/bin/sh", "-c"]
136        args:
137          - |
138            pip install trafilatura && \
139            hayhooks run --host 0.0.0.0
140        volumeMounts:
141          - name: local-data
142            mountPath: /mnt/data
143        env:
144          - name: HAYHOOKS_PIPELINES_DIR
145            value: /mnt/data
146          - name: OPENAI_API_KEY
147            valueFrom:
148              secretKeyRef:
149                name: openai-secret
150                key: api-key
151        resources:
152          limits:
153            memory: "512Mi"
154            cpu: "500m"
155          requests:
156            memory: "256Mi"
157            cpu: "250m"
158    volumes:
159      - name: local-data
160        hostPath:
161          path: /data
162          type: Directory
163  
164  ```
165  
166  Note that:
167  
168  - We changed the Hayhooks container `command` to install `trafilaura` dependency before startup, since it's needed for our [chat_with_website](https://github.com/deepset-ai/hayhooks/tree/main/examples/pipeline_wrappers/chat_with_website) example pipeline. For a real production environment, we recommend creating a custom Hayhooks image as described [here](docker.mdx#customizing-the-haystack-docker-image).
169  - We make Hayhooks container read `OPENAI_API_KEY` from a Kubernetes Secret.
170  
171  Before applying this new configuration, create the `openai-secret`:
172  
173  ```yaml
174  apiVersion: v1
175  kind: Secret
176  metadata:
177    name: openai-secret
178  type: Opaque
179  data:
180    # Replace the placeholder below with the base64 encoded value of your API key
181    # Generate it using: echo -n $OPENAI_API_KEY | base64
182    api-key: YOUR_BASE64_ENCODED_API_KEY_HERE
183  ```
184  
185  After applying this, check your Hayhooks Pod logs, and you'll see that the `chat_with_website` pipelines have already been deployed.
186  
187  <ClickableImage src="/img/2dbf42dd2db1cb355ee7222d7f8e96c45b611200d83ca289be3456264a854c38-Screenshot_2025-04-16_at_09.19.14.png" alt="None" />
188  
189  ## Roll Out Multiple Pods
190  
191  Haystack pipelines are usually stateless, which is a perfect use case for distributing the requests to multiple pods running the same set of pipelines. Let's convert the single-Pod configuration to an actual Kubernetes `Deployment`:
192  
193  ```yaml
194  apiVersion: apps/v1
195  kind: Deployment
196  metadata:
197    name: haystack-deployment
198  spec:
199    replicas: 3
200    selector:
201      matchLabels:
202        app: haystack
203    template:
204      metadata:
205        labels:
206          app: haystack
207      spec:
208        initContainers:
209          - name: install-dependencies
210            image: python:3.12-slim
211            workingDir: /mnt/data
212            command: ["/bin/bash", "-c"]
213            args:
214              - |
215                echo "Installing dependencies..."
216                pip install trafilatura
217                echo "Dependencies installed successfully!"
218                touch /mnt/data/init-complete
219            volumeMounts:
220              - name: local-data
221                mountPath: /mnt/data
222            resources:
223              requests:
224                memory: "64Mi"
225                cpu: "100m"
226              limits:
227                memory: "128Mi"
228                cpu: "250m"
229        containers:
230          - image: deepset/hayhooks:v0.6.0
231            name: hayhooks
232            imagePullPolicy: IfNotPresent
233            command: ["/bin/sh", "-c"]
234            args:
235              - |
236                pip install trafilatura && \
237                hayhooks run --host 0.0.0.0
238            ports:
239              - containerPort: 1416
240                name: http
241            volumeMounts:
242              - name: local-data
243                mountPath: /mnt/data
244            env:
245              - name: HAYHOOKS_PIPELINES_DIR
246                value: /mnt/data
247              - name: OPENAI_API_KEY
248                valueFrom:
249                  secretKeyRef:
250                    name: openai-secret
251                    key: api-key
252            resources:
253              requests:
254                memory: "256Mi"
255                cpu: "250m"
256              limits:
257                memory: "512Mi"
258                cpu: "500m"
259        volumes:
260          - name: local-data
261            hostPath:
262              path: /data
263              type: Directory
264  
265  ```
266  
267  Implementing the above configuration will create three pods. Each pod will run a different instance of Hayhooks, all serving the same example pipeline provided by the mounted volume in the previous example.
268  
269  <ClickableImage src="/img/f3f0ac4b22a37039f0837c22b0cb8b640937bbb0db4acfcbdf7bd016b545d84a-Screenshot_2025-04-16_at_09.32.07.png" alt="Kubernetes Lens interface showing three haystack-deployment pods in Running status with their resource configurations" />
270  
271  Note that the `NodePort` you created before will now act as a load balancer and will distribute incoming requests to the three Hayhooks Pods.