1  ---
  2  title: "AzureOpenAIGenerator"
  3  id: azureopenaigenerator
  4  slug: "/azureopenaigenerator"
  5  description: "This component enables text generation using OpenAI's large language models (LLMs) through Azure services."
  6  ---
  7  
  8  # AzureOpenAIGenerator
  9  
 10  This component enables text generation using OpenAI's large language models (LLMs) through Azure services.
 11  
 12  <div className="key-value-table">
 13  
 14  |  |  |
 15  | --- | --- |
 16  | **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx) |
 17  | **Mandatory init variables** | `api_key`: The Azure OpenAI API key. Can be set with `AZURE_OPENAI_API_KEY` env var.  <br /> <br />`azure_ad_token`: Microsoft Entra ID token. Can be set with `AZURE_OPENAI_AD_TOKEN` env var. |
 18  | **Mandatory run variables** | `prompt`: A string containing the prompt for the LLM |
 19  | **Output variables** | `replies`: A list of strings with all the replies generated by the LLM  <br /> <br />`meta`: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on |
 20  | **API reference** | [Generators](/reference/generators-api) |
 21  | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/azure.py |
 22  
 23  </div>
 24  
 25  ## Overview
 26  
 27  `AzureOpenAIGenerator` supports OpenAI models deployed through Azure services. To see the list of supported models, head over to Azure [documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?source=recommendations). The default model used with the component is `gpt-4o-mini`.
 28  
 29  To work with Azure components, you will need an Azure OpenAI API key, as well as an Azure OpenAI Endpoint. You can learn more about them in Azure [documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference).
 30  
 31  The component uses `AZURE_OPENAI_API_KEY` and `AZURE_OPENAI_AD_TOKEN` environment variables by default. Otherwise, you can pass `api_key` and `azure_ad_token` at initialization:
 32  
 33  ```python
 34  client = AzureOpenAIGenerator(
 35      azure_endpoint="<Your Azure endpoint e.g. `https://your-company.azure.openai.com/>",
 36      api_key=Secret.from_token("<your-api-key>"),
 37      azure_deployment="<a model name>",
 38  )
 39  ```
 40  
 41  :::info
 42  We recommend using environment variables instead of initialization parameters.
 43  :::
 44  
 45  Then, the component needs a prompt to operate, but you can pass any text generation parameters valid for the `openai.ChatCompletion.create` method directly to this component using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the supported parameters, refer to the [Azure documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference).
 46  
 47  You can also specify a model for this component through the `azure_deployment` init parameter.
 48  
 49  ### Streaming
 50  
 51  `AzureOpenAIGenerator` supports streaming the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter. Note that streaming the tokens is only compatible with generating a single response, so `n` must be set to 1 for streaming to work.
 52  
 53  :::info
 54  This component is designed for text generation, not for chat. If you want to use LLMs for chat, use [`AzureOpenAIChatGenerator`](azureopenaichatgenerator.mdx) instead.
 55  :::
 56  
 57  ## Usage
 58  
 59  ### On its own
 60  
 61  Basic usage:
 62  
 63  ```python
 64  from haystack.components.generators import AzureOpenAIGenerator
 65  client = AzureOpenAIGenerator()
 66  response = client.run("What's Natural Language Processing? Be brief.")
 67  print(response)
 68  
 69  >> {'replies': ['Natural Language Processing (NLP) is a branch of artificial intelligence that focuses on
 70  >> the interaction between computers and human language. It involves enabling computers to understand, interpret,
 71  >> and respond to natural human language in a way that is both meaningful and useful.'], 'meta': [{'model':
 72  >> 'gpt-4o-mini', 'index': 0, 'finish_reason': 'stop', 'usage': {'prompt_tokens': 16,
 73  >> 'completion_tokens': 49, 'total_tokens': 65}}]}
 74  
 75  ```
 76  
 77  With streaming:
 78  
 79  ```python
 80  from haystack.components.generators import AzureOpenAIGenerator
 81  
 82  client = AzureOpenAIGenerator(streaming_callback=lambda chunk: print(chunk.content, end="", flush=True))
 83  response = client.run("What's Natural Language Processing? Be brief.")
 84  print(response)
 85  
 86  >>> Natural Language Processing (NLP) is a branch of artificial
 87  	intelligence that focuses on the interaction between computers and human
 88    language. It involves enabling computers to understand, interpret,and respond
 89    to natural human language in a way that is both meaningful and useful.
 90  >>> {'replies': ['Natural Language Processing (NLP) is a branch of artificial
 91  	intelligence that focuses on the interaction between computers and human
 92    language. It involves enabling computers to understand, interpret,and respond
 93    to natural human language in a way that is both meaningful and useful.'],
 94    'meta': [{'model': 'gpt-4o-mini', 'index': 0, 'finish_reason':
 95    'stop', 'usage': {'prompt_tokens': 16, 'completion_tokens': 49,
 96    'total_tokens': 65}}]}
 97  
 98  ```
 99  
100  ### In a Pipeline
101  
102  ```python
103  from haystack import Pipeline
104  from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
105  from haystack.components.builders.prompt_builder import PromptBuilder
106  from haystack.components.generators import AzureOpenAIGenerator
107  from haystack.document_stores.in_memory import InMemoryDocumentStore
108  from haystack import Document
109  
110  docstore = InMemoryDocumentStore()
111  docstore.write_documents(
112      [
113          Document(content="Rome is the capital of Italy"),
114          Document(content="Paris is the capital of France"),
115      ],
116  )
117  
118  query = "What is the capital of France?"
119  
120  template = """
121  Given the following information, answer the question.
122  
123  Context:
124  {% for document in documents %}
125      {{ document.content }}
126  {% endfor %}
127  
128  Question: {{ query }}?
129  """
130  pipe = Pipeline()
131  
132  pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
133  pipe.add_component("prompt_builder", PromptBuilder(template=template))
134  pipe.add_component("llm", AzureOpenAIGenerator())
135  pipe.connect("retriever", "prompt_builder.documents")
136  pipe.connect("prompt_builder", "llm")
137  
138  res = pipe.run({"prompt_builder": {"query": query}, "retriever": {"query": query}})
139  
140  print(res)
141  ```