1  ---
  2  title: "OpenAIGenerator"
  3  id: openaigenerator
  4  slug: "/openaigenerator"
  5  description: "`OpenAIGenerator` enables text generation using OpenAI's large language models (LLMs)."
  6  ---
  7  
  8  # OpenAIGenerator
  9  
 10  `OpenAIGenerator` enables text generation using OpenAI's large language models (LLMs).
 11  
 12  |  |  |
 13  | --- | --- |
 14  | **Most common position in a pipeline** | After a [`PromptBuilder`](../builders/promptbuilder.mdx)  |
 15  | **Mandatory init variables** | "api_key": An OpenAI API key. Can be set with `OPENAI_API_KEY` env var. |
 16  | **Mandatory run variables** | “prompt”: A string containing the prompt for the LLM |
 17  | **Output variables** | “replies”: A list of strings with all the replies generated by the LLM  <br /> <br />”meta”: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on |
 18  | **API reference** | [Generators](/reference/generators-api) |
 19  | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/openai.py |
 20  
 21  ## Overview
 22  
 23  `OpenAIGenerator` supports OpenAI models starting from gpt-3.5-turbo and later (gpt-4, gpt-4-turbo, and so on).
 24  
 25  `OpenAIGenerator` needs an OpenAI key to work. It uses an `OPENAI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
 26  
 27  ```
 28  generator = OpenAIGenerator(api_key=Secret.from_token("<your-api-key>"), model="gpt-4o-mini")
 29  ```
 30  
 31  Then, the component needs a prompt to operate, but you can pass any text generation parameters valid for the `openai.ChatCompletion.create` method directly to this component using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the parameters supported by the OpenAI API, refer to the [OpenAI documentation](https://platform.openai.com/docs/api-reference/chat).
 32  
 33  `OpenAIGenerator` supports custom deployments of your OpenAI models through the `api_base_url` init parameter.
 34  
 35  ### Streaming
 36  
 37  `OpenAIGenerator` supports streaming the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter. Note that streaming the tokens is only compatible with generating a single response, so `n` must be set to 1 for streaming to work.
 38  
 39  :::note
 40  This component is designed for text generation, not for chat. If you want to use OpenAI LLMs for chat, use [`OpenAIChatGenerator`](openaichatgenerator.mdx) instead.
 41  
 42  :::
 43  
 44  ## Usage
 45  
 46  ### On its own
 47  
 48  Basic usage:
 49  
 50  ```python
 51  from haystack.components.generators import OpenAIGenerator
 52  from haystack.utils import Secret
 53  
 54  client = OpenAIGenerator(model="gpt-4", api_key=Secret.from_token("<your-api-key>"))
 55  response = client.run("What's Natural Language Processing? Be brief.")
 56  print(response)
 57  
 58      of artificial intelligence that focuses on the interaction between computers
 59      and humans through natural language. The primary aim of NLP is to enable
 60      computers to understand, interpret, and generate human language in a valuable way.'],
 61      'meta': [{'model': 'gpt-4-0613', 'index': 0, 'finish_reason':
 62      'stop', 'usage': {'prompt_tokens': 16, 'completion_tokens': 53,
 63      'total_tokens': 69}}]}
 64  ```
 65  
 66  With streaming:
 67  
 68  ```python
 69  from haystack.components.generators import OpenAIGenerator
 70  from haystack.utils import Secret
 71  
 72  client = OpenAIGenerator(streaming_callback=lambda chunk: print(chunk.content, end="", flush=True))
 73  response = client.run("What's Natural Language Processing? Be brief.")
 74  print(response)
 75  
 76  	intelligence that focuses on the interaction between computers and human
 77    language. It involves enabling computers to understand, interpret,and respond
 78    to natural human language in a way that is both meaningful and useful.
 79  	intelligence that focuses on the interaction between computers and human
 80    language. It involves enabling computers to understand, interpret,and respond
 81    to natural human language in a way that is both meaningful and useful.'],
 82    'meta': [{'model': 'gpt-4o-mini', 'index': 0, 'finish_reason':
 83    'stop', 'usage': {'prompt_tokens': 16, 'completion_tokens': 49,
 84    'total_tokens': 65}}]}
 85  ```
 86  
 87  ### In a Pipeline
 88  
 89  Here's an example of RAG Pipeline:
 90  
 91  ```python
 92  from haystack import Pipeline
 93  from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
 94  from haystack.components.builders.prompt_builder import PromptBuilder
 95  from haystack.components.generators import OpenAIGenerator
 96  from haystack.document_stores.in_memory import InMemoryDocumentStore
 97  from haystack import Document
 98  from haystack.utils import Secret
 99  
100  docstore = InMemoryDocumentStore()
101  docstore.write_documents([Document(content="Rome is the capital of Italy"), Document(content="Paris is the capital of France")])
102  
103  query = "What is the capital of France?"
104  
105  template = """
106  Given the following information, answer the question.
107  
108  Context:
109  {% for document in documents %}
110      {{ document.content }}
111  {% endfor %}
112  
113  Question: {{ query }}?
114  """
115  pipe = Pipeline()
116  
117  pipe.add_component("retriever", InMemoryBM25Retriever(document_store=docstore))
118  pipe.add_component("prompt_builder", PromptBuilder(template=template))
119  pipe.add_component("llm", OpenAIGenerator(api_key=Secret.from_token("<your-api-key>"))
120  pipe.connect("retriever", "prompt_builder.documents")
121  pipe.connect("prompt_builder", "llm")
122  
123  res=pipe.run({
124      "prompt_builder": {
125          "query": query
126      },
127      "retriever": {
128          "query": query
129      }
130  })
131  
132  print(res)
133  ```