Same Model, Different Harness: CUGA, Hermes, and Deep Agents — When to Pick CUGA?

There are nearly 200 agent harnesses to choose from. We gave three of them the same open-weight model and 90 AppWorld tasks, and changing the harness moved task completion from 30% to 81%. For enterprise use, CUGA adds what this benchmark does not test: rules that the harness enforces and deployment inside your own boundary.

Community lists such as awesome-agent-harness and best-of-Agent-Harnesses track nearly 200 of them, and they make one point clearly: the harness matters, not only the model. In one analysis that best-of-Agent-Harnesses cites, the same GLM-5.2 model went from 23% to 52% on SWE-bench Pro when only the harness changed.

So the right question is not "which harness is best?" but "which harness fits my workload, with my model?" Hermes from Nous Research is built for always-on personal assistants, and Deep Agents from LangChain for long, multi-step tasks such as research. Both are excellent at those jobs.

Enterprise work is a different job. Agents act on business systems, such as CRM and payments, and they must follow rules that hold whatever the model decides. CUGA (ConfigUrable Generalist Agent) is IBM's open-source harness built for that job, with open-weight or hosted models, inside your own environment, including sovereign clouds and air-gapped deployments. To measure what the harness adds, we kept the model the same and changed only the harness.

A controlled experiment: same model, different harness

Harness	How it ran
CUGA	CUGA's AppWorld configuration: AppWorld instructions, task decomposition, verification before execution, and automatic reflection
Hermes	Harbor's default Hermes setup, AppWorld tools over MCP
Deep Agents	AppWorld tools over MCP, plus LangChain's built-in middleware, configured for the 469 tools: tool selection, context summarization, a to-do list, retries, and call limits. We also added two small fixes so that malformed GLM replies did not end runs

The results

Where the points come from

🧭  Plan the work

🐍  Act in code