1  ---
  2  title: "MetaLlamaChatGenerator"
  3  id: metallamachatgenerator
  4  slug: "/metallamachatgenerator"
  5  description: "This component enables chat completion with any model hosted available with Meta Llama API."
  6  ---
  7  
  8  # MetaLlamaChatGenerator
  9  
 10  This component enables chat completion with any model hosted available with Meta Llama API.
 11  
 12  <div className="key-value-table">
 13  
 14  |  |  |
 15  | --- | --- |
 16  | **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx)                                                          |
 17  | **Mandatory init variables**           | `api_key`: A Meta Llama API key. Can be set with `LLAMA_API_KEY` env variable or passed to `init()` method. |
 18  | **Mandatory run variables**            | `messages`: A list of [ChatMessage](../../concepts/data-classes/chatmessage.mdx) objects                                                |
 19  | **Output variables**                   | `replies`: A list of [ChatMessage](../../concepts/data-classes/chatmessage.mdx) objects                                                 |
 20  | **API reference**                      | [Meta Llama API](/reference/integrations-meta-llama)                                                               |
 21  | **GitHub link**                        | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/meta_llama                |
 22  
 23  </div>
 24  
 25  ## Overview
 26  
 27  The `MetaLlamaChatGenerator` enables you to use multiple Meta Llama models by making chat completion calls to the Meta [Llama API](https://llama.developer.meta.com/?utm_source=partner-haystack&utm_medium=website). The default model is `Llama-4-Scout-17B-16E-Instruct-FP8`.
 28  
 29  Currently available models are:
 30  
 31  <div className="key-value-table">
 32  
 33  |  |  |  |  |  |
 34  | --- | --- | --- | --- | --- |
 35  | Model ID                                 | Input context length | Output context length | Input Modalities | Output Modalities |
 36  | `Llama-4-Scout-17B-16E-Instruct-FP8`     | 128k                 | 4028                  | Text, Image      | Text              |
 37  | `Llama-4-Maverick-17B-128E-Instruct-FP8` | 128k                 | 4028                  | Text, Image      | Text              |
 38  | `Llama-3.3-70B-Instruct`                 | 128k                 | 4028                  | Text             | Text              |
 39  | `Llama-3.3-8B-Instruct`                  | 128k                 | 4028                  | Text             | Text              |
 40  
 41  </div>
 42  This component uses the same `ChatMessage` format as other Haystack Chat Generators for structured input and output. For more information, see the [ChatMessage documentation](../../concepts/data-classes/chatmessage.mdx).
 43  
 44  ### Tool Support
 45  
 46  `MetaLlamaChatGenerator` supports function calling through the `tools` parameter, which accepts flexible tool configurations:
 47  
 48  - **A list of Tool objects**: Pass individual tools as a list
 49  - **A single Toolset**: Pass an entire Toolset directly
 50  - **Mixed Tools and Toolsets**: Combine multiple Toolsets with standalone tools in a single list
 51  
 52  This allows you to organize related tools into logical groups while also including standalone tools as needed.
 53  
 54  ```python
 55  from haystack.tools import Tool, Toolset
 56  from haystack_integrations.components.generators.meta_llama import MetaLlamaChatGenerator
 57  
 58  # Create individual tools
 59  weather_tool = Tool(name="weather", description="Get weather info", ...)
 60  news_tool = Tool(name="news", description="Get latest news", ...)
 61  
 62  # Group related tools into a toolset
 63  math_toolset = Toolset([add_tool, subtract_tool, multiply_tool])
 64  
 65  # Pass mixed tools and toolsets to the generator
 66  generator = MetaLlamaChatGenerator(
 67      tools=[math_toolset, weather_tool, news_tool]  # Mix of Toolset and Tool objects
 68  )
 69  ```
 70  
 71  For more details on working with tools, see the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
 72  
 73  ### Initialization
 74  
 75  To use this integration, you must have a Meta Llama API key. You can provide it with the `LLAMA_API_KEY` environment variable or by using a [Secret](../../concepts/secret-management.mdx).
 76  
 77  Then, install the `meta-llama-haystack` integration:
 78  
 79  ```shell
 80  pip install meta-llama-haystack
 81  ```
 82  
 83  ### Streaming
 84  
 85  `MetaLlamaChatGenerator` supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) responses from the LLM, allowing tokens to be emitted as they are generated. To enable streaming, pass a callable to the `streaming_callback` parameter during initialization.
 86  
 87  ## Usage
 88  
 89  ### On its own
 90  
 91  ```python
 92  from haystack.dataclasses import ChatMessage
 93  from haystack_integrations.components.generators.meta_llama import (
 94      MetaLlamaChatGenerator,
 95  )
 96  
 97  llm = MetaLlamaChatGenerator()
 98  response = llm.run([ChatMessage.from_user("What are Agentic Pipelines? Be brief.")])
 99  print(response["replies"][0].text)
100  ```
101  
102  With streaming and model routing:
103  
104  ```python
105  from haystack.dataclasses import ChatMessage
106  from haystack_integrations.components.generators.meta_llama import (
107      MetaLlamaChatGenerator,
108  )
109  
110  llm = MetaLlamaChatGenerator(
111      model="Llama-3.3-8B-Instruct",
112      streaming_callback=lambda chunk: print(chunk.content, end="", flush=True),
113  )
114  
115  response = llm.run([ChatMessage.from_user("What are Agentic Pipelines? Be brief.")])
116  
117  ## check the model used for the response
118  print("\n\n Model used: ", response["replies"][0].meta["model"])
119  ```
120  
121  With multimodal inputs:
122  
123  ```python
124  from haystack.dataclasses import ChatMessage, ImageContent
125  from haystack_integrations.components.generators.meta_llama import (
126      MetaLlamaChatGenerator,
127  )
128  
129  llm = MetaLlamaChatGenerator(model="Llama-4-Scout-17B-16E-Instruct-FP8")
130  
131  image = ImageContent.from_file_path("apple.jpg")
132  user_message = ChatMessage.from_user(
133      content_parts=["What does the image show? Max 5 words.", image],
134  )
135  
136  response = llm.run([user_message])["replies"][0].text
137  print(response)
138  
139  # Red apple on straw.
140  ```
141  
142  ### In a pipeline
143  
144  ```python
145  ## To run this example, you will need to set a `LLAMA_API_KEY` environment variable.
146  
147  from haystack import Document, Pipeline
148  from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
149  from haystack.components.generators.utils import print_streaming_chunk
150  from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
151  from haystack.dataclasses import ChatMessage
152  from haystack.document_stores.in_memory import InMemoryDocumentStore
153  from haystack.utils import Secret
154  
155  from haystack_integrations.components.generators.meta_llama import (
156      MetaLlamaChatGenerator,
157  )
158  
159  ## Write documents to InMemoryDocumentStore
160  document_store = InMemoryDocumentStore()
161  document_store.write_documents(
162      [
163          Document(content="My name is Jean and I live in Paris."),
164          Document(content="My name is Mark and I live in Berlin."),
165          Document(content="My name is Giorgio and I live in Rome."),
166      ],
167  )
168  
169  ## Build a RAG pipeline
170  prompt_template = [
171      ChatMessage.from_user(
172          "Given these documents, answer the question.\n"
173          "Documents:\n{% for doc in documents %}{{ doc.content }}{% endfor %}\n"
174          "Question: {{question}}\n"
175          "Answer:",
176      ),
177  ]
178  
179  ## Define required variables explicitly
180  prompt_builder = ChatPromptBuilder(
181      template=prompt_template,
182      required_variables={"question", "documents"},
183  )
184  
185  retriever = InMemoryBM25Retriever(document_store=document_store)
186  llm = MetaLlamaChatGenerator(
187      api_key=Secret.from_env_var("LLAMA_API_KEY"),
188      streaming_callback=print_streaming_chunk,
189  )
190  
191  rag_pipeline = Pipeline()
192  rag_pipeline.add_component("retriever", retriever)
193  rag_pipeline.add_component("prompt_builder", prompt_builder)
194  rag_pipeline.add_component("llm", llm)
195  rag_pipeline.connect("retriever", "prompt_builder.documents")
196  rag_pipeline.connect("prompt_builder", "llm.messages")
197  
198  ## Ask a question
199  question = "Who lives in Paris?"
200  rag_pipeline.run(
201      {
202          "retriever": {"query": question},
203          "prompt_builder": {"question": question},
204      },
205  )
206  ```