1  ---
  2  title: "Generators"
  3  id: experimental-generators-api
  4  description: "Enables text generation using LLMs."
  5  slug: "/experimental-generators-api"
  6  ---
  7  
  8  <a id="haystack_experimental.components.generators.chat.openai"></a>
  9  
 10  ## Module haystack\_experimental.components.generators.chat.openai
 11  
 12  <a id="haystack_experimental.components.generators.chat.openai.OpenAIChatGenerator"></a>
 13  
 14  ### OpenAIChatGenerator
 15  
 16  An OpenAI chat-based text generator component that supports hallucination risk scoring.
 17  
 18  This is based on the paper
 19  [LLMs are Bayesian, in Expectation, not in Realization](https://arxiv.org/abs/2507.11768).
 20  
 21  ## Usage Example:
 22  
 23      ```python
 24      from haystack.dataclasses import ChatMessage
 25  
 26      from haystack_experimental.utils.hallucination_risk_calculator.dataclasses import HallucinationScoreConfig
 27      from haystack_experimental.components.generators.chat.openai import OpenAIChatGenerator
 28  
 29      # Evidence-based Example
 30      llm = OpenAIChatGenerator(model="gpt-4o")
 31      rag_result = llm.run(
 32          messages=[
 33              ChatMessage.from_user(
 34                  text="Task: Answer strictly based on the evidence provided below.
 35  "
 36                  "Question: Who won the Nobel Prize in Physics in 2019?
 37  "
 38                  "Evidence:
 39  "
 40                  "- Nobel Prize press release (2019): James Peebles (1/2); Michel Mayor & Didier Queloz (1/2).
 41  "
 42                  "Constraints: If evidence is insufficient or conflicting, refuse."
 43              )
 44          ],
 45          hallucination_score_config=HallucinationScoreConfig(skeleton_policy="evidence_erase"),
 46      )
 47      print(f"Decision: {rag_result['replies'][0].meta['hallucination_decision']}")
 48      print(f"Risk bound: {rag_result['replies'][0].meta['hallucination_risk']:.3f}")
 49      print(f"Rationale: {rag_result['replies'][0].meta['hallucination_rationale']}")
 50      print(f"Answer:
 51  {rag_result['replies'][0].text}")
 52      print("---")
 53      ```
 54  
 55  <a id="haystack_experimental.components.generators.chat.openai.OpenAIChatGenerator.run"></a>
 56  
 57  #### OpenAIChatGenerator.run
 58  
 59  ```python
 60  @component.output_types(replies=list[ChatMessage])
 61  def run(
 62      messages: list[ChatMessage],
 63      streaming_callback: StreamingCallbackT | None = None,
 64      generation_kwargs: dict[str, Any] | None = None,
 65      *,
 66      tools: ToolsType | None = None,
 67      tools_strict: bool | None = None,
 68      hallucination_score_config: HallucinationScoreConfig | None = None
 69  ) -> dict[str, list[ChatMessage]]
 70  ```
 71  
 72  Invokes chat completion based on the provided messages and generation parameters.
 73  
 74  **Arguments**:
 75  
 76  - `messages`: A list of ChatMessage instances representing the input messages.
 77  - `streaming_callback`: A callback function that is called when a new token is received from the stream.
 78  - `generation_kwargs`: Additional keyword arguments for text generation. These parameters will
 79  override the parameters passed during component initialization.
 80  For details on OpenAI API parameters, see [OpenAI documentation](https://platform.openai.com/docs/api-reference/chat/create).
 81  - `tools`: A list of Tool and/or Toolset objects, or a single Toolset for which the model can prepare calls.
 82  If set, it will override the `tools` parameter provided during initialization.
 83  - `tools_strict`: Whether to enable strict schema adherence for tool calls. If set to `True`, the model will follow exactly
 84  the schema provided in the `parameters` field of the tool definition, but this may increase latency.
 85  If set, it will override the `tools_strict` parameter set during component initialization.
 86  - `hallucination_score_config`: If provided, the generator will evaluate the hallucination risk of its responses using
 87  the OpenAIPlanner and annotate each response with hallucination metrics.
 88  This involves generating multiple samples and analyzing their consistency, which may increase
 89  latency and cost. Use this option when you need to assess the reliability of the generated content
 90  in scenarios where accuracy is critical.
 91  For details, see the [research paper](https://arxiv.org/abs/2507.11768)
 92  
 93  **Returns**:
 94  
 95  A dictionary with the following key:
 96  - `replies`: A list containing the generated responses as ChatMessage instances. If hallucination
 97  scoring is enabled, each message will include additional metadata:
 98    - `hallucination_decision`: "ANSWER" if the model decided to answer, "REFUSE" if it abstained.
 99    - `hallucination_risk`: The EDFL hallucination risk bound.
100    - `hallucination_rationale`: The rationale behind the hallucination decision.
101  
102  <a id="haystack_experimental.components.generators.chat.openai.OpenAIChatGenerator.run_async"></a>
103  
104  #### OpenAIChatGenerator.run\_async
105  
106  ```python
107  @component.output_types(replies=list[ChatMessage])
108  async def run_async(
109      messages: list[ChatMessage],
110      streaming_callback: StreamingCallbackT | None = None,
111      generation_kwargs: dict[str, Any] | None = None,
112      *,
113      tools: ToolsType | None = None,
114      tools_strict: bool | None = None,
115      hallucination_score_config: HallucinationScoreConfig | None = None
116  ) -> dict[str, list[ChatMessage]]
117  ```
118  
119  Asynchronously invokes chat completion based on the provided messages and generation parameters.
120  
121  This is the asynchronous version of the `run` method. It has the same parameters and return values
122  but can be used with `await` in async code.
123  
124  **Arguments**:
125  
126  - `messages`: A list of ChatMessage instances representing the input messages.
127  - `streaming_callback`: A callback function that is called when a new token is received from the stream.
128  Must be a coroutine.
129  - `generation_kwargs`: Additional keyword arguments for text generation. These parameters will
130  override the parameters passed during component initialization.
131  For details on OpenAI API parameters, see [OpenAI documentation](https://platform.openai.com/docs/api-reference/chat/create).
132  - `tools`: A list of Tool and/or Toolset objects, or a single Toolset for which the model can prepare calls.
133  If set, it will override the `tools` parameter provided during initialization.
134  - `tools_strict`: Whether to enable strict schema adherence for tool calls. If set to `True`, the model will follow exactly
135  the schema provided in the `parameters` field of the tool definition, but this may increase latency.
136  If set, it will override the `tools_strict` parameter set during component initialization.
137  - `hallucination_score_config`: If provided, the generator will evaluate the hallucination risk of its responses using
138  the OpenAIPlanner and annotate each response with hallucination metrics.
139  This involves generating multiple samples and analyzing their consistency, which may increase
140  latency and cost. Use this option when you need to assess the reliability of the generated content
141  in scenarios where accuracy is critical.
142  For details, see the [research paper](https://arxiv.org/abs/2507.11768)
143  
144  **Returns**:
145  
146  A dictionary with the following key:
147  - `replies`: A list containing the generated responses as ChatMessage instances. If hallucination
148  scoring is enabled, each message will include additional metadata:
149    - `hallucination_decision`: "ANSWER" if the model decided to answer, "REFUSE" if it abstained.
150    - `hallucination_risk`: The EDFL hallucination risk bound.
151    - `hallucination_rationale`: The rationale behind the hallucination decision.
152