.venv/transformers/docs/source/en/main_classes/agent.md

Agents & Tools

Transformers Agents is an experimental API which is subject to change at any time. Results returned by the agents can vary as the APIs or underlying models are prone to change.

To learn more about agents and tools make sure to read the introductory guide. This page contains the API docs for the underlying classes.

Agents

We provide two types of agents, based on the main [Agent] class:

• [CodeAgent] acts in one shot, generating code to solve the task, then executes it at once.
• [ReactAgent] acts step by step, each step consisting of one thought, then one tool call and execution. It has two classes:
  • [ReactJsonAgent] writes its tool calls in JSON.
  • [ReactCodeAgent] writes its tool calls in Python code.

Agent

[[autodoc]] Agent

CodeAgent

[[autodoc]] CodeAgent

React agents

[[autodoc]] ReactAgent

[[autodoc]] ReactJsonAgent

[[autodoc]] ReactCodeAgent

ManagedAgent

[[autodoc]] ManagedAgent

Tools

load_tool

[[autodoc]] load_tool

tool

[[autodoc]] tool

Tool

[[autodoc]] Tool

Toolbox

[[autodoc]] Toolbox

PipelineTool

[[autodoc]] PipelineTool

launch_gradio_demo

[[autodoc]] launch_gradio_demo

stream_to_gradio

[[autodoc]] stream_to_gradio

ToolCollection

[[autodoc]] ToolCollection

Engines

You're free to create and use your own engines to be usable by the Agents framework. These engines have the following specification:

1. Follow the messages format for its input (List[Dict[str, str]]) and return a string.
2. Stop generating outputs before the sequences passed in the argument stop_sequences

TransformersEngine

For convenience, we have added a TransformersEngine that implements the points above, taking a pre-initialized Pipeline as input.

>>> from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline, TransformersEngine

>>> model_name = "HuggingFaceTB/SmolLM-135M-Instruct"
>>> tokenizer = AutoTokenizer.from_pretrained(model_name)
>>> model = AutoModelForCausalLM.from_pretrained(model_name)

>>> pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)

>>> engine = TransformersEngine(pipe)
>>> engine([{"role": "user", "content": "Ok!"}], stop_sequences=["great"])

"What a "

[[autodoc]] TransformersEngine

HfApiEngine

The HfApiEngine is an engine that wraps an HF Inference API client for the execution of the LLM.

>>> from transformers import HfApiEngine

>>> messages = [
...   {"role": "user", "content": "Hello, how are you?"},
...   {"role": "assistant", "content": "I'm doing great. How can I help you today?"},
...   {"role": "user", "content": "No need to help, take it easy."},
... ]

>>> HfApiEngine()(messages, stop_sequences=["conversation"])

"That's very kind of you to say! It's always nice to have a relaxed "

[[autodoc]] HfApiEngine

Agent Types

Agents can handle any type of object in-between tools; tools, being completely multimodal, can accept and return text, image, audio, video, among other types. In order to increase compatibility between tools, as well as to correctly render these returns in ipython (jupyter, colab, ipython notebooks, ...), we implement wrapper classes around these types.

The wrapped objects should continue behaving as initially; a text object should still behave as a string, an image object should still behave as a PIL.Image.

These types have three specific purposes:

• Calling to_raw on the type should return the underlying object
• Calling to_string on the type should return the object as a string: that can be the string in case of an AgentText but will be the path of the serialized version of the object in other instances
• Displaying it in an ipython kernel should display the object correctly

AgentText

[[autodoc]] transformers.agents.agent_types.AgentText

AgentImage

[[autodoc]] transformers.agents.agent_types.AgentImage

AgentAudio

[[autodoc]] transformers.agents.agent_types.AgentAudio