cohere.md
1 --- 2 title: "Cohere" 3 id: integrations-cohere 4 description: "Cohere integration for Haystack" 5 slug: "/integrations-cohere" 6 --- 7 8 9 ## haystack_integrations.components.embedders.cohere.document_embedder 10 11 ### CohereDocumentEmbedder 12 13 A component for computing Document embeddings using Cohere models. 14 15 The embedding of each Document is stored in the `embedding` field of the Document. 16 17 Usage example: 18 19 ```python 20 from haystack import Document 21 from haystack_integrations.components.embedders.cohere import CohereDocumentEmbedder 22 23 doc = Document(content="I love pizza!") 24 25 document_embedder = CohereDocumentEmbedder() 26 27 result = document_embedder.run([doc]) 28 print(result['documents'][0].embedding) 29 30 # [-0.453125, 1.2236328, 2.0058594, ...] 31 ``` 32 33 #### SUPPORTED_MODELS 34 35 ```python 36 SUPPORTED_MODELS: list[str] = [ 37 "embed-v4.0", 38 "embed-english-v3.0", 39 "embed-english-light-v3.0", 40 "embed-multilingual-v3.0", 41 "embed-multilingual-light-v3.0", 42 ] 43 44 ``` 45 46 A non-exhaustive list of embed models supported by this component. 47 See https://docs.cohere.com/docs/models#embed for the full list. 48 49 #### __init__ 50 51 ```python 52 __init__( 53 api_key: Secret = Secret.from_env_var(["COHERE_API_KEY", "CO_API_KEY"]), 54 model: str = "embed-v4.0", 55 input_type: str = "search_document", 56 api_base_url: str = "https://api.cohere.com", 57 truncate: str = "END", 58 timeout: float = 120.0, 59 batch_size: int = 32, 60 progress_bar: bool = True, 61 meta_fields_to_embed: list[str] | None = None, 62 embedding_separator: str = "\n", 63 embedding_type: EmbeddingTypes | None = None, 64 ) -> None 65 ``` 66 67 Initialize the CohereDocumentEmbedder. 68 69 **Parameters:** 70 71 - **api_key** (<code>Secret</code>) – the Cohere API key. 72 - **model** (<code>str</code>) – the name of the model to use. 73 Read [Cohere documentation](https://docs.cohere.com/docs/models#embed) for a list of all supported models. 74 - **input_type** (<code>str</code>) – specifies the type of input you're giving to the model. Supported values are 75 "search_document", "search_query", "classification" and "clustering". 76 - **api_base_url** (<code>str</code>) – the Cohere API Base url. 77 - **truncate** (<code>str</code>) – truncate embeddings that are too long from start or end, ("NONE"|"START"|"END"). 78 Passing "START" will discard the start of the input. "END" will discard the end of the input. In both 79 cases, input is discarded until the remaining input is exactly the maximum input token length for the model. 80 If "NONE" is selected, when the input exceeds the maximum input token length an error will be returned. 81 - **timeout** (<code>float</code>) – request timeout in seconds. 82 - **batch_size** (<code>int</code>) – number of Documents to encode at once. 83 - **progress_bar** (<code>bool</code>) – whether to show a progress bar or not. Can be helpful to disable in production deployments 84 to keep the logs clean. 85 - **meta_fields_to_embed** (<code>list\[str\] | None</code>) – list of meta fields that should be embedded along with the Document text. 86 - **embedding_separator** (<code>str</code>) – separator used to concatenate the meta fields to the Document text. 87 - **embedding_type** (<code>EmbeddingTypes | None</code>) – the type of embeddings to return. Defaults to float embeddings. 88 Note that int8, uint8, binary, and ubinary are only valid for v3 models. 89 90 #### to_dict 91 92 ```python 93 to_dict() -> dict[str, Any] 94 ``` 95 96 Serializes the component to a dictionary. 97 98 **Returns:** 99 100 - <code>dict\[str, Any\]</code> – Dictionary with serialized data. 101 102 #### from_dict 103 104 ```python 105 from_dict(data: dict[str, Any]) -> CohereDocumentEmbedder 106 ``` 107 108 Deserializes the component from a dictionary. 109 110 **Parameters:** 111 112 - **data** (<code>dict\[str, Any\]</code>) – Dictionary to deserialize from. 113 114 **Returns:** 115 116 - <code>CohereDocumentEmbedder</code> – Deserialized component. 117 118 #### run 119 120 ```python 121 run(documents: list[Document]) -> dict[str, list[Document] | dict[str, Any]] 122 ``` 123 124 Embed a list of `Documents`. 125 126 **Parameters:** 127 128 - **documents** (<code>list\[Document\]</code>) – documents to embed. 129 130 **Returns:** 131 132 - <code>dict\[str, list\[Document\] | dict\[str, Any\]\]</code> – A dictionary with the following keys: 133 - `documents`: documents with the `embedding` field set. 134 - `meta`: metadata about the embedding process. 135 136 **Raises:** 137 138 - <code>TypeError</code> – if the input is not a list of `Documents`. 139 140 #### run_async 141 142 ```python 143 run_async( 144 documents: list[Document], 145 ) -> dict[str, list[Document] | dict[str, Any]] 146 ``` 147 148 Embed a list of `Documents` asynchronously. 149 150 **Parameters:** 151 152 - **documents** (<code>list\[Document\]</code>) – documents to embed. 153 154 **Returns:** 155 156 - <code>dict\[str, list\[Document\] | dict\[str, Any\]\]</code> – A dictionary with the following keys: 157 - `documents`: documents with the `embedding` field set. 158 - `meta`: metadata about the embedding process. 159 160 **Raises:** 161 162 - <code>TypeError</code> – if the input is not a list of `Documents`. 163 164 ## haystack_integrations.components.embedders.cohere.document_image_embedder 165 166 ### CohereDocumentImageEmbedder 167 168 A component for computing Document embeddings based on images using Cohere models. 169 170 The embedding of each Document is stored in the `embedding` field of the Document. 171 172 ### Usage example 173 174 ```python 175 from haystack import Document 176 from haystack_integrations.components.embedders.cohere import CohereDocumentImageEmbedder 177 178 embedder = CohereDocumentImageEmbedder(model="embed-v4.0") 179 180 documents = [ 181 Document(content="A photo of a cat", meta={"file_path": "cat.jpg"}), 182 Document(content="A photo of a dog", meta={"file_path": "dog.jpg"}), 183 ] 184 185 result = embedder.run(documents=documents) 186 documents_with_embeddings = result["documents"] 187 print(documents_with_embeddings) 188 189 # [Document(id=..., 190 # content='A photo of a cat', 191 # meta={'file_path': 'cat.jpg', 192 # 'embedding_source': {'type': 'image', 'file_path_meta_field': 'file_path'}}, 193 # embedding=vector of size 1536), 194 # ...] 195 ``` 196 197 #### SUPPORTED_MODELS 198 199 ```python 200 SUPPORTED_MODELS: list[str] = [ 201 "embed-v4.0", 202 "embed-english-v3.0", 203 "embed-english-light-v3.0", 204 "embed-multilingual-v3.0", 205 "embed-multilingual-light-v3.0", 206 ] 207 208 ``` 209 210 A non-exhaustive list of embed models supported by this component. 211 See https://docs.cohere.com/docs/models#embed for the full list. 212 213 #### __init__ 214 215 ```python 216 __init__( 217 *, 218 file_path_meta_field: str = "file_path", 219 root_path: str | None = None, 220 image_size: tuple[int, int] | None = None, 221 api_key: Secret = Secret.from_env_var(["COHERE_API_KEY", "CO_API_KEY"]), 222 model: str = "embed-v4.0", 223 api_base_url: str = "https://api.cohere.com", 224 timeout: float = 120.0, 225 embedding_dimension: int | None = None, 226 embedding_type: EmbeddingTypes = EmbeddingTypes.FLOAT, 227 progress_bar: bool = True 228 ) -> None 229 ``` 230 231 Creates a CohereDocumentImageEmbedder component. 232 233 **Parameters:** 234 235 - **file_path_meta_field** (<code>str</code>) – The metadata field in the Document that contains the file path to the image or PDF. 236 - **root_path** (<code>str | None</code>) – The root directory path where document files are located. If provided, file paths in 237 document metadata will be resolved relative to this path. If None, file paths are treated as absolute paths. 238 - **image_size** (<code>tuple\[int, int\] | None</code>) – If provided, resizes the image to fit within the specified dimensions (width, height) while 239 maintaining aspect ratio. This reduces file size, memory usage, and processing time, which is beneficial 240 when working with models that have resolution constraints or when transmitting images to remote services. 241 - **api_key** (<code>Secret</code>) – The Cohere API key. 242 - **model** (<code>str</code>) – The Cohere model to use for calculating embeddings. 243 Read [Cohere documentation](https://docs.cohere.com/docs/models#embed) for a list of all supported models. 244 - **api_base_url** (<code>str</code>) – The Cohere API base URL. 245 - **timeout** (<code>float</code>) – Request timeout in seconds. 246 - **embedding_dimension** (<code>int | None</code>) – The dimension of the embeddings to return. Only valid for v4 and newer models. 247 Read [Cohere API reference](https://docs.cohere.com/reference/embed) for a list possible values and 248 supported models. 249 - **embedding_type** (<code>EmbeddingTypes</code>) – The type of embeddings to return. Defaults to float embeddings. 250 Specifying a type different from float is only supported for Embed v3.0 and newer models. 251 - **progress_bar** (<code>bool</code>) – Whether to show a progress bar or not. Can be helpful to disable in production deployments 252 to keep the logs clean. 253 254 #### to_dict 255 256 ```python 257 to_dict() -> dict[str, Any] 258 ``` 259 260 Serializes the component to a dictionary. 261 262 **Returns:** 263 264 - <code>dict\[str, Any\]</code> – Dictionary with serialized data. 265 266 #### from_dict 267 268 ```python 269 from_dict(data: dict[str, Any]) -> CohereDocumentImageEmbedder 270 ``` 271 272 Deserializes the component from a dictionary. 273 274 **Parameters:** 275 276 - **data** (<code>dict\[str, Any\]</code>) – Dictionary to deserialize from. 277 278 **Returns:** 279 280 - <code>CohereDocumentImageEmbedder</code> – Deserialized component. 281 282 #### run 283 284 ```python 285 run(documents: list[Document]) -> dict[str, list[Document]] 286 ``` 287 288 Embed a list of image documents. 289 290 **Parameters:** 291 292 - **documents** (<code>list\[Document\]</code>) – Documents to embed. 293 294 **Returns:** 295 296 - <code>dict\[str, list\[Document\]\]</code> – A dictionary with the following keys: 297 - `documents`: Documents with embeddings. 298 299 #### run_async 300 301 ```python 302 run_async(documents: list[Document]) -> dict[str, list[Document]] 303 ``` 304 305 Asynchronously embed a list of image documents. 306 307 **Parameters:** 308 309 - **documents** (<code>list\[Document\]</code>) – Documents to embed. 310 311 **Returns:** 312 313 - <code>dict\[str, list\[Document\]\]</code> – A dictionary with the following keys: 314 - `documents`: Documents with embeddings. 315 316 ## haystack_integrations.components.embedders.cohere.text_embedder 317 318 ### CohereTextEmbedder 319 320 A component for embedding strings using Cohere models. 321 322 Usage example: 323 324 ```python 325 from haystack_integrations.components.embedders.cohere import CohereTextEmbedder 326 327 text_to_embed = "I love pizza!" 328 329 text_embedder = CohereTextEmbedder() 330 331 print(text_embedder.run(text_to_embed)) 332 333 # {'embedding': [-0.453125, 1.2236328, 2.0058594, ...] 334 # 'meta': {'api_version': {'version': '1'}, 'billed_units': {'input_tokens': 4}}} 335 ``` 336 337 #### SUPPORTED_MODELS 338 339 ```python 340 SUPPORTED_MODELS: list[str] = [ 341 "embed-v4.0", 342 "embed-english-v3.0", 343 "embed-english-light-v3.0", 344 "embed-multilingual-v3.0", 345 "embed-multilingual-light-v3.0", 346 ] 347 348 ``` 349 350 A non-exhaustive list of embed models supported by this component. 351 See https://docs.cohere.com/docs/models#embed for the full list. 352 353 #### __init__ 354 355 ```python 356 __init__( 357 api_key: Secret = Secret.from_env_var(["COHERE_API_KEY", "CO_API_KEY"]), 358 model: str = "embed-v4.0", 359 input_type: str = "search_query", 360 api_base_url: str = "https://api.cohere.com", 361 truncate: str = "END", 362 timeout: float = 120.0, 363 embedding_type: EmbeddingTypes | None = None, 364 ) -> None 365 ``` 366 367 Initialize the CohereTextEmbedder. 368 369 **Parameters:** 370 371 - **api_key** (<code>Secret</code>) – the Cohere API key. 372 - **model** (<code>str</code>) – the name of the model to use. 373 Read [Cohere documentation](https://docs.cohere.com/docs/models#embed) for a list of all supported models. 374 - **input_type** (<code>str</code>) – specifies the type of input you're giving to the model. Supported values are 375 "search_document", "search_query", "classification" and "clustering". 376 - **api_base_url** (<code>str</code>) – the Cohere API Base url. 377 - **truncate** (<code>str</code>) – truncate embeddings that are too long from start or end, ("NONE"|"START"|"END"). 378 Passing "START" will discard the start of the input. "END" will discard the end of the input. In both 379 cases, input is discarded until the remaining input is exactly the maximum input token length for the model. 380 If "NONE" is selected, when the input exceeds the maximum input token length an error will be returned. 381 - **timeout** (<code>float</code>) – request timeout in seconds. 382 - **embedding_type** (<code>EmbeddingTypes | None</code>) – the type of embeddings to return. Defaults to float embeddings. 383 Note that int8, uint8, binary, and ubinary are only valid for v3 models. 384 385 #### to_dict 386 387 ```python 388 to_dict() -> dict[str, Any] 389 ``` 390 391 Serializes the component to a dictionary. 392 393 **Returns:** 394 395 - <code>dict\[str, Any\]</code> – Dictionary with serialized data. 396 397 #### from_dict 398 399 ```python 400 from_dict(data: dict[str, Any]) -> CohereTextEmbedder 401 ``` 402 403 Deserializes the component from a dictionary. 404 405 **Parameters:** 406 407 - **data** (<code>dict\[str, Any\]</code>) – Dictionary to deserialize from. 408 409 **Returns:** 410 411 - <code>CohereTextEmbedder</code> – Deserialized component. 412 413 #### run 414 415 ```python 416 run(text: str) -> dict[str, list[float] | dict[str, Any]] 417 ``` 418 419 Embed text. 420 421 **Parameters:** 422 423 - **text** (<code>str</code>) – the text to embed. 424 425 **Returns:** 426 427 - <code>dict\[str, list\[float\] | dict\[str, Any\]\]</code> – A dictionary with the following keys: 428 - `embedding`: the embedding of the text. 429 - `meta`: metadata about the request. 430 431 **Raises:** 432 433 - <code>TypeError</code> – If the input is not a string. 434 435 #### run_async 436 437 ```python 438 run_async(text: str) -> dict[str, list[float] | dict[str, Any]] 439 ``` 440 441 Asynchronously embed text. 442 443 This is the asynchronous version of the `run` method. It has the same parameters and return values 444 but can be used with `await` in async code. 445 446 :param text: 447 Text to embed. 448 449 **Returns:** 450 451 - <code>dict\[str, list\[float\] | dict\[str, Any\]\]</code> – A dictionary with the following keys: 452 - `embedding`: the embedding of the text. 453 - `meta`: metadata about the request. 454 455 **Raises:** 456 457 - <code>TypeError</code> – If the input is not a string. 458 459 ## haystack_integrations.components.embedders.cohere.utils 460 461 ### get_async_response 462 463 ```python 464 get_async_response( 465 cohere_async_client: AsyncClientV2, 466 texts: list[str], 467 model_name: str, 468 input_type: str, 469 truncate: str, 470 embedding_type: EmbeddingTypes | None = None, 471 ) -> tuple[list[list[float]], dict[str, Any]] 472 ``` 473 474 Embeds a list of texts asynchronously using the Cohere API. 475 476 **Parameters:** 477 478 - **cohere_async_client** (<code>AsyncClientV2</code>) – the Cohere `AsyncClient` 479 - **texts** (<code>list\[str\]</code>) – the texts to embed 480 - **model_name** (<code>str</code>) – the name of the model to use 481 - **input_type** (<code>str</code>) – one of "classification", "clustering", "search_document", "search_query". 482 The type of input text provided to embed. 483 - **truncate** (<code>str</code>) – one of "NONE", "START", "END". How the API handles text longer than the maximum token length. 484 - **embedding_type** (<code>EmbeddingTypes | None</code>) – the type of embeddings to return. Defaults to float embeddings. 485 486 **Returns:** 487 488 - <code>tuple\[list\[list\[float\]\], dict\[str, Any\]\]</code> – A tuple of the embeddings and metadata. 489 490 **Raises:** 491 492 - <code>ValueError</code> – If an error occurs while querying the Cohere API. 493 494 ### get_response 495 496 ```python 497 get_response( 498 cohere_client: ClientV2, 499 texts: list[str], 500 model_name: str, 501 input_type: str, 502 truncate: str, 503 batch_size: int = 32, 504 progress_bar: bool = False, 505 embedding_type: EmbeddingTypes | None = None, 506 ) -> tuple[list[list[float]], dict[str, Any]] 507 ``` 508 509 Embeds a list of texts using the Cohere API. 510 511 **Parameters:** 512 513 - **cohere_client** (<code>ClientV2</code>) – the Cohere `Client` 514 - **texts** (<code>list\[str\]</code>) – the texts to embed 515 - **model_name** (<code>str</code>) – the name of the model to use 516 - **input_type** (<code>str</code>) – one of "classification", "clustering", "search_document", "search_query". 517 The type of input text provided to embed. 518 - **truncate** (<code>str</code>) – one of "NONE", "START", "END". How the API handles text longer than the maximum token length. 519 - **batch_size** (<code>int</code>) – the batch size to use 520 - **progress_bar** (<code>bool</code>) – if `True`, show a progress bar 521 - **embedding_type** (<code>EmbeddingTypes | None</code>) – the type of embeddings to return. Defaults to float embeddings. 522 523 **Returns:** 524 525 - <code>tuple\[list\[list\[float\]\], dict\[str, Any\]\]</code> – A tuple of the embeddings and metadata. 526 527 **Raises:** 528 529 - <code>ValueError</code> – If an error occurs while querying the Cohere API. 530 531 ## haystack_integrations.components.generators.cohere.chat.chat_generator 532 533 ### CohereChatGenerator 534 535 Completes chats using Cohere's models using cohere.ClientV2 `chat` endpoint. 536 537 This component supports both text-only and multimodal (text + image) conversations 538 using Cohere's vision models like Command A Vision. 539 540 Supported image formats: PNG, JPEG, WEBP, GIF (non-animated). 541 Maximum 20 images per request with 20MB total limit. 542 543 You can customize how the chat response is generated by passing parameters to the 544 Cohere API through the `**generation_kwargs` parameter. You can do this when 545 initializing or running the component. Any parameter that works with 546 `cohere.ClientV2.chat` will work here too. 547 For details, see [Cohere API](https://docs.cohere.com/reference/chat). 548 549 Below is an example of how to use the component: 550 551 ### Simple example 552 553 ```python 554 from haystack.dataclasses import ChatMessage 555 from haystack.utils import Secret 556 from haystack_integrations.components.generators.cohere import CohereChatGenerator 557 558 client = CohereChatGenerator(api_key=Secret.from_env_var("COHERE_API_KEY")) 559 messages = [ChatMessage.from_user("What's Natural Language Processing?")] 560 client.run(messages) 561 562 # Output: {'replies': [ChatMessage(_role=<ChatRole.ASSISTANT: 'assistant'>, 563 # _content=[TextContent(text='Natural Language Processing (NLP) is an interdisciplinary... 564 ``` 565 566 ### Multimodal example 567 568 ```python 569 from haystack.dataclasses import ChatMessage, ImageContent 570 from haystack.utils import Secret 571 from haystack_integrations.components.generators.cohere import CohereChatGenerator 572 573 # Create an image from file path or base64 574 image_content = ImageContent.from_file_path("path/to/your/image.jpg") 575 576 # Create a multimodal message with both text and image 577 messages = [ChatMessage.from_user(content_parts=["What's in this image?", image_content])] 578 579 # Use a multimodal model like Command A Vision 580 client = CohereChatGenerator(model="command-a-vision-07-2025", api_key=Secret.from_env_var("COHERE_API_KEY")) 581 response = client.run(messages) 582 print(response) 583 ``` 584 585 ### Advanced example 586 587 CohereChatGenerator can be integrated into pipelines and supports Haystack's tooling 588 architecture, enabling tools to be invoked seamlessly across various generators. 589 590 ```python 591 from haystack import Pipeline 592 from haystack.dataclasses import ChatMessage 593 from haystack.components.tools import ToolInvoker 594 from haystack.tools import Tool 595 from haystack_integrations.components.generators.cohere import CohereChatGenerator 596 597 # Create a weather tool 598 def weather(city: str) -> str: 599 return f"The weather in {city} is sunny and 32°C" 600 601 weather_tool = Tool( 602 name="weather", 603 description="useful to determine the weather in a given location", 604 parameters={ 605 "type": "object", 606 "properties": { 607 "city": { 608 "type": "string", 609 "description": "The name of the city to get weather for, e.g. Paris, London", 610 } 611 }, 612 "required": ["city"], 613 }, 614 function=weather, 615 ) 616 617 # Create and set up the pipeline 618 pipeline = Pipeline() 619 pipeline.add_component("generator", CohereChatGenerator(tools=[weather_tool])) 620 pipeline.add_component("tool_invoker", ToolInvoker(tools=[weather_tool])) 621 pipeline.connect("generator", "tool_invoker") 622 623 # Run the pipeline with a weather query 624 results = pipeline.run( 625 data={"generator": {"messages": [ChatMessage.from_user("What's the weather like in Paris?")]}} 626 ) 627 628 # The tool result will be available in the pipeline output 629 print(results["tool_invoker"]["tool_messages"][0].tool_call_result.result) 630 # Output: "The weather in Paris is sunny and 32°C" 631 ``` 632 633 #### SUPPORTED_MODELS 634 635 ```python 636 SUPPORTED_MODELS: list[str] = [ 637 "command-a-03-2025", 638 "command-r7b-12-2024", 639 "command-a-translate-08-2025", 640 "command-a-reasoning-08-2025", 641 "command-a-vision-07-2025", 642 "command-r-08-2024", 643 "command-r-plus-08-2024", 644 "command-r-03-2024", 645 "command-r-plus-04-2024", 646 "command-r-plus", 647 "command-r", 648 "command-light", 649 "command", 650 ] 651 652 ``` 653 654 A non-exhaustive list of chat models supported by this component. 655 See https://docs.cohere.com/docs/models#command for the full list. 656 657 #### __init__ 658 659 ```python 660 __init__( 661 api_key: Secret = Secret.from_env_var(["COHERE_API_KEY", "CO_API_KEY"]), 662 model: str = "command-a-03-2025", 663 streaming_callback: StreamingCallbackT | None = None, 664 api_base_url: str | None = None, 665 generation_kwargs: dict[str, Any] | None = None, 666 tools: ToolsType | None = None, 667 *, 668 timeout: float | None = None, 669 max_retries: int | None = None 670 ) -> None 671 ``` 672 673 Initialize the CohereChatGenerator instance. 674 675 **Parameters:** 676 677 - **api_key** (<code>Secret</code>) – The API key for the Cohere API. 678 - **model** (<code>str</code>) – The name of the model to use. You can use models from the `command` family. 679 - **streaming_callback** (<code>StreamingCallbackT | None</code>) – A callback function that is called when a new token is received from the stream. 680 The callback function accepts [StreamingChunk](https://docs.haystack.deepset.ai/docs/data-classes#streamingchunk) 681 as an argument. 682 - **api_base_url** (<code>str | None</code>) – The base URL of the Cohere API. 683 - **generation_kwargs** (<code>dict\[str, Any\] | None</code>) – Other parameters to use for the model during generation. For a list of parameters, 684 see [Cohere Chat endpoint](https://docs.cohere.com/reference/chat). 685 Some of the parameters are: 686 - 'messages': A list of messages between the user and the model, meant to give the model 687 conversational context for responding to the user's message. 688 - 'system_message': When specified, adds a system message at the beginning of the conversation. 689 - 'citation_quality': Defaults to `accurate`. Dictates the approach taken to generating citations 690 as part of the RAG flow by allowing the user to specify whether they want 691 `accurate` results or `fast` results. 692 - 'temperature': A non-negative float that tunes the degree of randomness in generation. Lower temperatures 693 mean less random generations. 694 - **tools** (<code>ToolsType | None</code>) – A list of Tool and/or Toolset objects, or a single Toolset that the model can use. 695 Each tool should have a unique name. 696 - **timeout** (<code>float | None</code>) – Timeout for Cohere client calls. If not set, it defaults to the default set by the Cohere client. 697 - **max_retries** (<code>int | None</code>) – Maximum number of retries to attempt for failed requests. If not set, it defaults to the default set by 698 the Cohere client. 699 700 #### to_dict 701 702 ```python 703 to_dict() -> dict[str, Any] 704 ``` 705 706 Serializes the component to a dictionary. 707 708 **Returns:** 709 710 - <code>dict\[str, Any\]</code> – Dictionary with serialized data. 711 712 #### from_dict 713 714 ```python 715 from_dict(data: dict[str, Any]) -> CohereChatGenerator 716 ``` 717 718 Deserializes the component from a dictionary. 719 720 **Parameters:** 721 722 - **data** (<code>dict\[str, Any\]</code>) – Dictionary to deserialize from. 723 724 **Returns:** 725 726 - <code>CohereChatGenerator</code> – Deserialized component. 727 728 #### run 729 730 ```python 731 run( 732 messages: list[ChatMessage], 733 generation_kwargs: dict[str, Any] | None = None, 734 tools: ToolsType | None = None, 735 streaming_callback: StreamingCallbackT | None = None, 736 ) -> dict[str, list[ChatMessage]] 737 ``` 738 739 Invoke the chat endpoint based on the provided messages and generation parameters. 740 741 **Parameters:** 742 743 - **messages** (<code>list\[ChatMessage\]</code>) – list of `ChatMessage` instances representing the input messages. 744 - **generation_kwargs** (<code>dict\[str, Any\] | None</code>) – additional keyword arguments for chat generation. These parameters will 745 potentially override the parameters passed in the __init__ method. 746 For more details on the parameters supported by the Cohere API, refer to the 747 Cohere [documentation](https://docs.cohere.com/reference/chat). 748 - **tools** (<code>ToolsType | None</code>) – A list of Tool and/or Toolset objects, or a single Toolset for which the model can prepare calls. 749 If set, it will override the `tools` parameter set during component initialization. 750 - **streaming_callback** (<code>StreamingCallbackT | None</code>) – A callback function that is called when a new token is received from the stream. 751 The callback function accepts StreamingChunk as an argument. 752 753 **Returns:** 754 755 - <code>dict\[str, list\[ChatMessage\]\]</code> – A dictionary with the following keys: 756 - `replies`: a list of `ChatMessage` instances representing the generated responses. 757 758 #### run_async 759 760 ```python 761 run_async( 762 messages: list[ChatMessage], 763 generation_kwargs: dict[str, Any] | None = None, 764 tools: ToolsType | None = None, 765 streaming_callback: StreamingCallbackT | None = None, 766 ) -> dict[str, list[ChatMessage]] 767 ``` 768 769 Asynchronously invoke the chat endpoint based on the provided messages and generation parameters. 770 771 **Parameters:** 772 773 - **messages** (<code>list\[ChatMessage\]</code>) – list of `ChatMessage` instances representing the input messages. 774 - **generation_kwargs** (<code>dict\[str, Any\] | None</code>) – additional keyword arguments for chat generation. These parameters will 775 potentially override the parameters passed in the __init__ method. 776 For more details on the parameters supported by the Cohere API, refer to the 777 Cohere [documentation](https://docs.cohere.com/reference/chat). 778 - **tools** (<code>ToolsType | None</code>) – A list of Tool and/or Toolset objects, or a single Toolset for which the model can prepare calls. 779 If set, it will override the `tools` parameter set during component initialization. 780 - **streaming_callback** (<code>StreamingCallbackT | None</code>) – A callback function that is called when a new token is received from the stream. 781 782 **Returns:** 783 784 - <code>dict\[str, list\[ChatMessage\]\]</code> – A dictionary with the following keys: 785 - `replies`: a list of `ChatMessage` instances representing the generated responses. 786 787 ## haystack_integrations.components.generators.cohere.generator 788 789 ### CohereGenerator 790 791 Bases: <code>CohereChatGenerator</code> 792 793 Generates text using Cohere's models through Cohere's `generate` endpoint. 794 795 NOTE: Cohere discontinued the `generate` API, so this generator is a mere wrapper 796 around `CohereChatGenerator` provided for backward compatibility. 797 798 ### Usage example 799 800 ```python 801 from haystack_integrations.components.generators.cohere import CohereGenerator 802 803 generator = CohereGenerator(api_key="test-api-key") 804 generator.run(prompt="What's the capital of France?") 805 ``` 806 807 #### SUPPORTED_MODELS 808 809 ```python 810 SUPPORTED_MODELS: list[str] = [ 811 "command-a-03-2025", 812 "command-r7b-12-2024", 813 "command-a-translate-08-2025", 814 "command-a-reasoning-08-2025", 815 "command-a-vision-07-2025", 816 "command-r-08-2024", 817 "command-r-plus-08-2024", 818 "command-r-03-2024", 819 "command-r-plus-04-2024", 820 "command-r-plus", 821 "command-r", 822 "command-light", 823 "command", 824 ] 825 826 ``` 827 828 A non-exhaustive list of chat models supported by this component. 829 See https://docs.cohere.com/docs/models#command for the full list. 830 831 #### __init__ 832 833 ```python 834 __init__( 835 api_key: Secret = Secret.from_env_var(["COHERE_API_KEY", "CO_API_KEY"]), 836 model: str = "command-a-03-2025", 837 streaming_callback: Callable | None = None, 838 api_base_url: str | None = None, 839 **kwargs: Any 840 ) -> None 841 ``` 842 843 Instantiates a `CohereGenerator` component. 844 845 **Parameters:** 846 847 - **api_key** (<code>Secret</code>) – Cohere API key. 848 - **model** (<code>str</code>) – Cohere model to use for generation. 849 - **streaming_callback** (<code>Callable | None</code>) – Callback function that is called when a new token is received from the stream. 850 The callback function accepts [StreamingChunk](https://docs.haystack.deepset.ai/docs/data-classes#streamingchunk) 851 as an argument. 852 - **api_base_url** (<code>str | None</code>) – Cohere base URL. 853 - \*\***kwargs** (<code>Any</code>) – Additional arguments passed to the model. These arguments are specific to the model. 854 You can check them in model's documentation. 855 856 #### run 857 858 ```python 859 run(prompt: str) -> dict[str, list[str] | list[dict[str, Any]]] 860 ``` 861 862 Queries the LLM with the prompts to produce replies. 863 864 **Parameters:** 865 866 - **prompt** (<code>str</code>) – the prompt to be sent to the generative model. 867 868 **Returns:** 869 870 - <code>dict\[str, list\[str\] | list\[dict\[str, Any\]\]\]</code> – A dictionary with the following keys: 871 - `replies`: A list of replies generated by the model. 872 - `meta`: Information about the request. 873 874 #### run_async 875 876 ```python 877 run_async(prompt: str) -> dict[str, list[str] | list[dict[str, Any]]] 878 ``` 879 880 Queries the LLM asynchronously with the prompts to produce replies. 881 882 **Parameters:** 883 884 - **prompt** (<code>str</code>) – the prompt to be sent to the generative model. 885 886 **Returns:** 887 888 - <code>dict\[str, list\[str\] | list\[dict\[str, Any\]\]\]</code> – A dictionary with the following keys: 889 - `replies`: A list of replies generated by the model. 890 - `meta`: Information about the request. 891 892 ## haystack_integrations.components.rankers.cohere.ranker 893 894 ### CohereRanker 895 896 Ranks Documents based on their similarity to the query using [Cohere models](https://docs.cohere.com/reference/rerank-1). 897 898 Documents are indexed from most to least semantically relevant to the query. 899 900 Usage example: 901 902 ```python 903 from haystack import Document 904 from haystack_integrations.components.rankers.cohere import CohereRanker 905 906 ranker = CohereRanker(model="rerank-v3.5", top_k=2) 907 908 docs = [Document(content="Paris"), Document(content="Berlin")] 909 query = "What is the capital of germany?" 910 output = ranker.run(query=query, documents=docs) 911 docs = output["documents"] 912 ``` 913 914 #### __init__ 915 916 ```python 917 __init__( 918 model: str = "rerank-v3.5", 919 top_k: int = 10, 920 api_key: Secret = Secret.from_env_var(["COHERE_API_KEY", "CO_API_KEY"]), 921 api_base_url: str = "https://api.cohere.com", 922 meta_fields_to_embed: list[str] | None = None, 923 meta_data_separator: str = "\n", 924 max_tokens_per_doc: int = 4096, 925 ) -> None 926 ``` 927 928 Creates an instance of the 'CohereRanker'. 929 930 **Parameters:** 931 932 - **model** (<code>str</code>) – Cohere model name. Check the list of supported models in the [Cohere documentation](https://docs.cohere.com/docs/models). 933 - **top_k** (<code>int</code>) – The maximum number of documents to return. 934 - **api_key** (<code>Secret</code>) – Cohere API key. 935 - **api_base_url** (<code>str</code>) – the base URL of the Cohere API. 936 - **meta_fields_to_embed** (<code>list\[str\] | None</code>) – List of meta fields that should be concatenated 937 with the document content for reranking. 938 - **meta_data_separator** (<code>str</code>) – Separator used to concatenate the meta fields 939 to the Document content. 940 - **max_tokens_per_doc** (<code>int</code>) – The maximum number of tokens to embed for each document defaults to 4096. 941 942 #### to_dict 943 944 ```python 945 to_dict() -> dict[str, Any] 946 ``` 947 948 Serializes the component to a dictionary. 949 950 **Returns:** 951 952 - <code>dict\[str, Any\]</code> – Dictionary with serialized data. 953 954 #### from_dict 955 956 ```python 957 from_dict(data: dict[str, Any]) -> CohereRanker 958 ``` 959 960 Deserializes the component from a dictionary. 961 962 **Parameters:** 963 964 - **data** (<code>dict\[str, Any\]</code>) – The dictionary to deserialize from. 965 966 **Returns:** 967 968 - <code>CohereRanker</code> – The deserialized component. 969 970 #### run 971 972 ```python 973 run( 974 query: str, documents: list[Document], top_k: int | None = None 975 ) -> dict[str, list[Document]] 976 ``` 977 978 Use the Cohere Reranker to re-rank the list of documents based on the query. 979 980 **Parameters:** 981 982 - **query** (<code>str</code>) – Query string. 983 - **documents** (<code>list\[Document\]</code>) – List of Documents. 984 - **top_k** (<code>int | None</code>) – The maximum number of Documents you want the Ranker to return. 985 986 **Returns:** 987 988 - <code>dict\[str, list\[Document\]\]</code> – A dictionary with the following keys: 989 - `documents`: List of Documents most similar to the given query in descending order of similarity. 990 991 **Raises:** 992 993 - <code>ValueError</code> – If `top_k` is not > 0. 994 995 #### run_async 996 997 ```python 998 run_async( 999 query: str, documents: list[Document], top_k: int | None = None 1000 ) -> dict[str, list[Document]] 1001 ``` 1002 1003 Asynchronously re-rank the list of documents based on the query. 1004 1005 This is the asynchronous version of the `run` method. It has the same parameters and return values 1006 but can be used with `await` in async code. 1007 1008 **Parameters:** 1009 1010 - **query** (<code>str</code>) – Query string. 1011 - **documents** (<code>list\[Document\]</code>) – List of Documents. 1012 - **top_k** (<code>int | None</code>) – The maximum number of Documents you want the Ranker to return. 1013 1014 **Returns:** 1015 1016 - <code>dict\[str, list\[Document\]\]</code> – A dictionary with the following keys: 1017 - `documents`: List of Documents most similar to the given query in descending order of similarity. 1018 1019 **Raises:** 1020 1021 - <code>ValueError</code> – If `top_k` is not > 0.