1  ---
  2  title: "OptimumDocumentEmbedder"
  3  id: optimumdocumentembedder
  4  slug: "/optimumdocumentembedder"
  5  description: "A component to compute documents’ embeddings using models loaded with the Hugging Face Optimum library."
  6  ---
  7  
  8  # OptimumDocumentEmbedder
  9  
 10  A component to compute documents’ embeddings using models loaded with the Hugging Face Optimum library.
 11  
 12  <div className="key-value-table">
 13  
 14  |  |  |
 15  | --- | --- |
 16  | **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx)  in an indexing pipeline                |
 17  | **Mandatory run variables**            | `documents`: A list of documents                                                          |
 18  | **Output variables**                   | `documents`: A list of documents enriched with embeddings                                 |
 19  | **API reference**                      | [Optimum](/reference/integrations-optimum)                                                       |
 20  | **GitHub link**                        | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/optimum |
 21  
 22  </div>
 23  
 24  ## Overview
 25  
 26  `OptimumDocumentEmbedder` embeds text strings using models loaded with the [HuggingFace Optimum](https://huggingface.co/docs/optimum/index) library. It uses the [ONNX runtime](https://onnxruntime.ai/) for high-speed inference.
 27  
 28  The default model is `sentence-transformers/all-mpnet-base-v2`.
 29  
 30  Similarly to other Embedders, this component allows adding prefixes (and suffixes) to include instructions. For more details, refer to the component’s API reference.
 31  
 32  There are three useful parameters specific to the Optimum Embedder that you can control with various modes:
 33  
 34  - [Pooling](/reference/integrations-optimum#optimumembedderpooling): generate a fixed-sized sentence embedding from a variable-sized sentence embedding
 35  - [Optimization](https://huggingface.co/docs/optimum/onnxruntime/usage_guides/optimization): apply graph optimization to the model and improve inference speed
 36  - [Quantization](https://huggingface.co/docs/optimum/onnxruntime/usage_guides/quantization): reduce the computational and memory costs
 37  
 38  Find all the available mode details in our Optimum [API Reference](/reference/integrations-optimum).
 39  
 40  ### Authentication
 41  
 42  Authentication with a Hugging Face API Token is only required to access private or gated models through Serverless Inference API or the Inference Endpoints.
 43  
 44  The component uses an `HF_API_TOKEN` or `HF_TOKEN` environment variable, or you can pass a Hugging Face API token at initialization. See our [Secret Management](../../concepts/secret-management.mdx) page for more information.
 45  
 46  ## Usage
 47  
 48  To start using this integration with Haystack, install it with:
 49  
 50  ```shell
 51  pip install optimum-haystack
 52  ```
 53  
 54  ### On its own
 55  
 56  ```python
 57  from haystack.dataclasses import Document
 58  from haystack_integrations.components.embedders.optimum import OptimumDocumentEmbedder
 59  
 60  doc = Document(content="I love pizza!")
 61  
 62  document_embedder = OptimumDocumentEmbedder(
 63      model="sentence-transformers/all-mpnet-base-v2",
 64  )
 65  document_embedder.warm_up()
 66  
 67  result = document_embedder.run([doc])
 68  print(result["documents"][0].embedding)
 69  
 70  ## [0.017020374536514282, -0.023255806416273117, ...]
 71  ```
 72  
 73  ### In a pipeline
 74  
 75  ```python
 76  from haystack import Pipeline
 77  from haystack import Document
 78  from haystack_integrations.components.embedders.optimum import (
 79      OptimumDocumentEmbedder,
 80      OptimumEmbedderPooling,
 81      OptimumEmbedderOptimizationConfig,
 82      OptimumEmbedderOptimizationMode,
 83  )
 84  
 85  documents = [
 86      Document(content="My name is Wolfgang and I live in Berlin"),
 87      Document(content="I saw a black horse running"),
 88      Document(content="Germany has many big cities"),
 89  ]
 90  
 91  embedder = OptimumDocumentEmbedder(
 92      model="intfloat/e5-base-v2",
 93      normalize_embeddings=True,
 94      onnx_execution_provider="CUDAExecutionProvider",
 95      optimizer_settings=OptimumEmbedderOptimizationConfig(
 96          mode=OptimumEmbedderOptimizationMode.O4,
 97          for_gpu=True,
 98      ),
 99      working_dir="/tmp/optimum",
100      pooling_mode=OptimumEmbedderPooling.MEAN,
101  )
102  
103  pipeline = Pipeline()
104  pipeline.add_component("embedder", embedder)
105  
106  pipeline.run({"embedder": {"documents": documents}})
107  
108  print(results["embedder"]["embedding"])
109  ```