Open-source LLMs as LangChain Agents

TL;DR

Open-source LLMs have now reached a performance level that makes them suitable reasoning engines for powering agent workflows: Mixtral even surpasses GPT-3.5 on our benchmark, and its performance could easily be further enhanced with fine-tuning.

We've released the simplest agentic library out there: smolagents! Go checkout the smolagents introduction blog here.

Introduction

Large Language Models (LLMs) trained for causal language modeling can tackle a wide range of tasks, but they often struggle with basic tasks like logic, calculation, and search. The worst scenario is when they perform poorly in a domain, such as math, yet still attempt to handle all the calculations themselves.

To overcome this weakness, amongst other approaches, one can integrate the LLM into a system where it can call tools: such a system is called an LLM agent.

In this post, we explain the inner workings of ReAct agents, then show how to build them using the ChatHuggingFace class recently integrated in LangChain. Finally, we benchmark several open-source LLMs against GPT-3.5 and GPT-4.

Table of Contents

• What are agents?
  • Toy example of a ReAct agent's inner working
  • Challenges of agent systems
• Running agents with LangChain
• Agents Showdown: how do different LLMs perform as general purpose reasoning agents?
  • Evaluation
  • Results

What are agents?

The definition of LLM agents is quite broad: LLM agents are all systems that use LLMs as their engine and can perform actions on their environment based on observations. They can use several iterations of the Perception ⇒ Reflexion ⇒ Action cycle to achieve their task and are often augmented with planning or knowledge management systems to enhance their performance. You can find a good review of the Agents landscape in Xi et al., 2023.

Toy example of a ReAct agent's inner working

The graph above seems very high-level, but under the hood it’s quite simple.

Take a look at this notebook: we implement a barebones tool call example with the Transformers library.