1  ---
  2  title: "MetaLlamaChatGenerator"
  3  id: metallamachatgenerator
  4  slug: "/metallamachatgenerator"
  5  description: "This component enables chat completion with any model hosted available with Meta Llama API."
  6  ---
  7  
  8  # MetaLlamaChatGenerator
  9  
 10  This component enables chat completion with any model hosted available with Meta Llama API.
 11  
 12  <div className="key-value-table">
 13  
 14  |                                        |                                                                                                             |
 15  | -------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
 16  | **Most common position in a pipeline** | After a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx)                                                          |
 17  | **Mandatory init variables**           | `api_key`: A Meta Llama API key. Can be set with `LLAMA_API_KEY` env variable or passed to `init()` method. |
 18  | **Mandatory run variables**            | `messages`: A list of [ChatMessage](../../concepts/data-classes/chatmessage.mdx) objects                                                |
 19  | **Output variables**                   | `replies`: A list of [ChatMessage](../../concepts/data-classes/chatmessage.mdx) objects                                                 |
 20  | **API reference**                      | [Meta Llama API](/reference/integrations-meta-llama)                                                               |
 21  | **GitHub link**                        | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/meta_llama                |
 22  
 23  </div>
 24  
 25  ## Overview
 26  
 27  The `MetaLlamaChatGenerator` enables you to use multiple Meta Llama models by making chat completion calls to the Meta [Llama API](https://llama.developer.meta.com/?utm_source=partner-haystack&utm_medium=website). The default model is `Llama-4-Scout-17B-16E-Instruct-FP8`.
 28  
 29  Currently available models are:
 30  
 31  | Model ID                                 | Input context length | Output context length | Input Modalities | Output Modalities |
 32  | ---------------------------------------- | -------------------- | --------------------- | ---------------- | ----------------- |
 33  | `Llama-4-Scout-17B-16E-Instruct-FP8`     | 128k                 | 4028                  | Text, Image      | Text              |
 34  | `Llama-4-Maverick-17B-128E-Instruct-FP8` | 128k                 | 4028                  | Text, Image      | Text              |
 35  | `Llama-3.3-70B-Instruct`                 | 128k                 | 4028                  | Text             | Text              |
 36  | `Llama-3.3-8B-Instruct`                  | 128k                 | 4028                  | Text             | Text              |
 37  
 38  This component uses the same `ChatMessage` format as other Haystack Chat Generators for structured input and output. For more information, see the [ChatMessage documentation](../../concepts/data-classes/chatmessage.mdx).
 39  
 40  ### Tool Support
 41  
 42  `MetaLlamaChatGenerator` supports function calling through the `tools` parameter, which accepts flexible tool configurations:
 43  
 44  - **A list of Tool objects**: Pass individual tools as a list
 45  - **A single Toolset**: Pass an entire Toolset directly
 46  - **Mixed Tools and Toolsets**: Combine multiple Toolsets with standalone tools in a single list
 47  
 48  This allows you to organize related tools into logical groups while also including standalone tools as needed.
 49  
 50  ```python
 51  from haystack.tools import Tool, Toolset
 52  from haystack_integrations.components.generators.meta_llama import MetaLlamaChatGenerator
 53  
 54  # Create individual tools
 55  weather_tool = Tool(name="weather", description="Get weather info", ...)
 56  news_tool = Tool(name="news", description="Get latest news", ...)
 57  
 58  # Group related tools into a toolset
 59  math_toolset = Toolset([add_tool, subtract_tool, multiply_tool])
 60  
 61  # Pass mixed tools and toolsets to the generator
 62  generator = MetaLlamaChatGenerator(
 63      tools=[math_toolset, weather_tool, news_tool]  # Mix of Toolset and Tool objects
 64  )
 65  ```
 66  
 67  For more details on working with tools, see the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
 68  
 69  ### Initialization
 70  
 71  To use this integration, you must have a Meta Llama API key. You can provide it with the `LLAMA_API_KEY` environment variable or by using a [Secret](../../concepts/secret-management.mdx).
 72  
 73  Then, install the `meta-llama-haystack` integration:
 74  
 75  ```shell
 76  pip install meta-llama-haystack
 77  ```
 78  
 79  ### Streaming
 80  
 81  `MetaLlamaChatGenerator` supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) responses from the LLM, allowing tokens to be emitted as they are generated. To enable streaming, pass a callable to the `streaming_callback` parameter during initialization.
 82  
 83  ## Usage
 84  
 85  ### On its own
 86  
 87  ```python
 88  from haystack.dataclasses import ChatMessage
 89  from haystack_integrations.components.generators.meta_llama import (
 90      MetaLlamaChatGenerator,
 91  )
 92  
 93  llm = MetaLlamaChatGenerator()
 94  response = llm.run([ChatMessage.from_user("What are Agentic Pipelines? Be brief.")])
 95  print(response["replies"][0].text)
 96  ```
 97  
 98  With streaming and model routing:
 99  
100  ```python
101  from haystack.dataclasses import ChatMessage
102  from haystack_integrations.components.generators.meta_llama import (
103      MetaLlamaChatGenerator,
104  )
105  
106  llm = MetaLlamaChatGenerator(
107      model="Llama-3.3-8B-Instruct",
108      streaming_callback=lambda chunk: print(chunk.content, end="", flush=True),
109  )
110  
111  response = llm.run([ChatMessage.from_user("What are Agentic Pipelines? Be brief.")])
112  
113  ## check the model used for the response
114  print("\n\n Model used: ", response["replies"][0].meta["model"])
115  ```
116  
117  ### In a pipeline
118  
119  ```python
120  ## To run this example, you will need to set a `LLAMA_API_KEY` environment variable.
121  
122  from haystack import Document, Pipeline
123  from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
124  from haystack.components.generators.utils import print_streaming_chunk
125  from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
126  from haystack.dataclasses import ChatMessage
127  from haystack.document_stores.in_memory import InMemoryDocumentStore
128  from haystack.utils import Secret
129  
130  from haystack_integrations.components.generators.meta_llama import (
131      MetaLlamaChatGenerator,
132  )
133  
134  ## Write documents to InMemoryDocumentStore
135  document_store = InMemoryDocumentStore()
136  document_store.write_documents(
137      [
138          Document(content="My name is Jean and I live in Paris."),
139          Document(content="My name is Mark and I live in Berlin."),
140          Document(content="My name is Giorgio and I live in Rome."),
141      ],
142  )
143  
144  ## Build a RAG pipeline
145  prompt_template = [
146      ChatMessage.from_user(
147          "Given these documents, answer the question.\n"
148          "Documents:\n{% for doc in documents %}{{ doc.content }}{% endfor %}\n"
149          "Question: {{question}}\n"
150          "Answer:",
151      ),
152  ]
153  
154  ## Define required variables explicitly
155  prompt_builder = ChatPromptBuilder(
156      template=prompt_template,
157      required_variables={"question", "documents"},
158  )
159  
160  retriever = InMemoryBM25Retriever(document_store=document_store)
161  llm = MetaLlamaChatGenerator(
162      api_key=Secret.from_env_var("LLAMA_API_KEY"),
163      streaming_callback=print_streaming_chunk,
164  )
165  
166  rag_pipeline = Pipeline()
167  rag_pipeline.add_component("retriever", retriever)
168  rag_pipeline.add_component("prompt_builder", prompt_builder)
169  rag_pipeline.add_component("llm", llm)
170  rag_pipeline.connect("retriever", "prompt_builder.documents")
171  rag_pipeline.connect("prompt_builder", "llm.messages")
172  
173  ## Ask a question
174  question = "Who lives in Paris?"
175  rag_pipeline.run(
176      {
177          "retriever": {"query": question},
178          "prompt_builder": {"question": question},
179      },
180  )
181  ```