1  ---
  2  title: "OpenAIGenerator"
  3  id: openaigenerator
  4  slug: "/openaigenerator"
  5  description: "`OpenAIGenerator` enables text generation using OpenAI's large language models (LLMs)."
  6  ---
  7  
  8  # OpenAIGenerator
  9  
 10  `OpenAIGenerator` enables text generation using OpenAI's large language models (LLMs).
 11  
 12  <div className="key-value-table">
 13  
 14  |  |  |
 15  | --- | --- |
 16  | **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx) |
 17  | **Mandatory init variables** | `api_key`: An OpenAI API key. Can be set with `OPENAI_API_KEY` env var. |
 18  | **Mandatory run variables** | `prompt`: A string containing the prompt for the LLM |
 19  | **Output variables** | `replies`: A list of strings with all the replies generated by the LLM  <br /> <br />`meta`: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on |
 20  | **API reference** | [Generators](/reference/generators-api) |
 21  | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/openai.py |
 22  
 23  </div>
 24  
 25  ## Overview
 26  
 27  `OpenAIGenerator` supports OpenAI models starting from gpt-3.5-turbo and later (gpt-4, gpt-4-turbo, and so on).
 28  
 29  `OpenAIGenerator` needs an OpenAI key to work. It uses an `OPENAI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
 30  
 31  ```
 32  generator = OpenAIGenerator(api_key=Secret.from_token("<your-api-key>"), model="gpt-4o-mini")
 33  ```
 34  
 35  Then, the component needs a prompt to operate, but you can pass any text generation parameters valid for the `openai.ChatCompletion.create` method directly to this component using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the parameters supported by the OpenAI API, refer to the [OpenAI documentation](https://platform.openai.com/docs/api-reference/chat).
 36  
 37  `OpenAIGenerator` supports custom deployments of your OpenAI models through the `api_base_url` init parameter.
 38  
 39  ### Streaming
 40  
 41  `OpenAIGenerator` supports streaming the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter. Note that streaming the tokens is only compatible with generating a single response, so `n` must be set to 1 for streaming to work.
 42  
 43  :::info
 44  This component is designed for text generation, not for chat. If you want to use OpenAI LLMs for chat, use [`OpenAIChatGenerator`](openaichatgenerator.mdx) instead.
 45  :::
 46  
 47  ## Usage
 48  
 49  ### On its own
 50  
 51  Basic usage:
 52  
 53  ```python
 54  from haystack.components.generators import OpenAIGenerator
 55  from haystack.utils import Secret
 56  
 57  client = OpenAIGenerator(model="gpt-4", api_key=Secret.from_token("<your-api-key>"))
 58  response = client.run("What's Natural Language Processing? Be brief.")
 59  print(response)
 60  
 61  >>> {'replies': ['Natural Language Processing, often abbreviated as NLP, is a field
 62      of artificial intelligence that focuses on the interaction between computers
 63      and humans through natural language. The primary aim of NLP is to enable
 64      computers to understand, interpret, and generate human language in a valuable way.'],
 65      'meta': [{'model': 'gpt-4-0613', 'index': 0, 'finish_reason':
 66      'stop', 'usage': {'prompt_tokens': 16, 'completion_tokens': 53,
 67      'total_tokens': 69}}]}
 68  ```
 69  
 70  With streaming:
 71  
 72  ```python
 73  from haystack.components.generators import OpenAIGenerator
 74  from haystack.utils import Secret
 75  
 76  client = OpenAIGenerator(streaming_callback=lambda chunk: print(chunk.content, end="", flush=True))
 77  response = client.run("What's Natural Language Processing? Be brief.")
 78  print(response)
 79  
 80  >>> Natural Language Processing (NLP) is a branch of artificial
 81  	intelligence that focuses on the interaction between computers and human
 82    language. It involves enabling computers to understand, interpret,and respond
 83    to natural human language in a way that is both meaningful and useful.
 84  >>> {'replies': ['Natural Language Processing (NLP) is a branch of artificial
 85  	intelligence that focuses on the interaction between computers and human
 86    language. It involves enabling computers to understand, interpret,and respond
 87    to natural human language in a way that is both meaningful and useful.'],
 88    'meta': [{'model': 'gpt-4o-mini', 'index': 0, 'finish_reason':
 89    'stop', 'usage': {'prompt_tokens': 16, 'completion_tokens': 49,
 90    'total_tokens': 65}}]}
 91  ```
 92  
 93  ### In a Pipeline
 94  
 95  Here's an example of RAG Pipeline:
 96  
 97  ```python
 98  from haystack import Pipeline
 99  from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
100  from haystack.components.builders.prompt_builder import PromptBuilder
101  from haystack.components.generators import OpenAIGenerator
102  from haystack.document_stores.in_memory import InMemoryDocumentStore
103  from haystack import Document
104  from haystack.utils import Secret
105  
106  docstore = InMemoryDocumentStore()
107  docstore.write_documents([Document(content="Rome is the capital of Italy"), Document(content="Paris is the capital of France")])
108  
109  query = "What is the capital of France?"
110  
111  template = """
112  Given the following information, answer the question.
113  
114  Context:
115  {% for document in documents %}
116      {{ document.content }}
117  {% endfor %}
118  
119  Question: {{ query }}?
120  """
121  pipe = Pipeline()
122  
123  pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
124  pipe.add_component("prompt_builder", PromptBuilder(template=template))
125  pipe.add_component("llm", OpenAIGenerator(api_key=Secret.from_token("<your-api-key>"))
126  pipe.connect("retriever", "prompt_builder.documents")
127  pipe.connect("prompt_builder", "llm")
128  
129  res=pipe.run({
130      "prompt_builder": {
131          "query": query
132      },
133      "retriever": {
134          "query": query
135      }
136  })
137  
138  print(res)
139  ```