License to Call: Introducing Transformers Agents 2.0

TL;DR

We are releasing Transformers Agents 2.0!

⇒ 🎁 On top of our existing agent type, we introduce two new agents that can iterate based on past observations to solve complex tasks.

⇒ 💡 We aim for the code to be clear and modular, and for common attributes like the final prompt and tools to be transparent.

⇒ 🤝 We add sharing options to boost community agents.

⇒ 💪 Extremely performant new agent framework, allowing a Llama-3-70B-Instruct agent to outperform GPT-4 based agents in the GAIA Leaderboard!

🚀 Go try it out and climb ever higher on the GAIA leaderboard!

transformers.agents has now been upgraded to the stand-alone library smolagents! The two libraries have very similar APIs, so switching is easy. Go checkout the smolagents introduction blog here.

Table of Contents

• What is an agent?
• The Transformers Agents approach
  • Main elements
• Example use-cases
  • Self-correcting Retrieval-Augmented-Generation
  • Using a simple multi-agent setup 🤝 for efficient web browsing
• Testing our agents
  • Benchmarking LLM engines
  • Climbing up the GAIA Leaderboard with a multi-modal agent
• Conclusion

What is an agent?

Large Language Models (LLMs) can tackle a wide range of tasks, but they often struggle with specific tasks like logic, calculation, and search. When prompted in these domains in which they do not perform well, they frequently fail to generate a correct answer.

One approach to overcome this weakness is to create an agent, which is just a program driven by an LLM. The agent is empowered by tools to help it perform actions. When the agent needs a specific skill to solve a particular problem, it relies on an appropriate tool from its toolbox.

Thus when during problem-solving the agent needs a specific skill, it can just rely on an appropriate tool from its toolbox.

Experimentally, agent frameworks generally work very well, achieving state-of-the-art performance on several benchmarks. For instance, have a look at the top submissions for HumanEval: they are agent systems.