Introducing Agents.js: Give tools to your LLMs using JavaScript

We have recently been working on Agents.js at huggingface.js. It's a new library for giving tool access to LLMs from JavaScript in either the browser or the server. It ships with a few multi-modal tools out of the box and can easily be extended with your own tools and language models.

Installation

Getting started is very easy, you can grab the library from npm with the following:

npm install @huggingface/agents

Usage

The library exposes the HfAgent object which is the entry point to the library. You can instantiate it like this:

import { HfAgent } from "@huggingface/agents";

const HF_ACCESS_TOKEN = "hf_..."; // get your token at https://huggingface.co/settings/tokens

const agent = new HfAgent(HF_ACCESS_TOKEN);

Afterward, using the agent is easy. You give it a plain-text command and it will return some messages.

const code = await agent.generateCode(
  "Draw a picture of a rubber duck with a top hat, then caption this picture."
);

which in this case generated the following code

// code generated by the LLM
async function generate() {
  const output = await textToImage("rubber duck with a top hat");
  message("We generate the duck picture", output);
  const caption = await imageToText(output);
  message("Now we caption the image", caption);
  return output;
}

Then the code can be evaluated as such:

const messages = await agent.evaluateCode(code);

The messages returned by the agent are objects with the following shape:

export interface Update {
    message: string;
    data: undefined | string | Blob;

where message is an info text and data can contain either a string or a blob. The blob can be used to display images or audio.

If you trust your environment (see warning), you can also run the code directly from the prompt with run :

const messages = await agent.run(
  "Draw a picture of a rubber duck with a top hat, then caption this picture."
);

Usage warning

Custom LLMs 💬

Custom Tools 🛠️

Now this tool can be added to the list of tools when initiating your agent.

Then you can do: