Lite-Harness SDK

AI harnesses are the new vendor lock-in. To swap across harnesses easily without rewriting your app, LiteLLM launched the Lite-Harness SDK.

Run your prompt across different harnesses:

from lite_harness import query, AgentOptions

prompt = "Fix the failing test"

# Claude Code harness
async for message in query(
    prompt=prompt,
    options=AgentOptions(harness="claude-code", model="claude-opus-4-8"),
):
    print(message)

# Codex harness
async for message in query(
    prompt=prompt,
    options=AgentOptions(harness="codex", model="gpt-5.5"),
):
    print(message)

To enable cost controls, fallbacks, and logging, point it to your LiteLLM AI Gateway:

export LITELLM_API_BASE=https://litellm.your-company.com/v1
export LITELLM_API_KEY=sk-litellm-...

Engineer's Takeaway:
This SDK unifies how you invoke the agents, not how they run internally. Each harness keeps its native loop and tool-calling semantics. It is perfect for A/B testing agent performance and centralizing costs, but remember it is in public beta, so custom tool injection might require extra work!

The Problem I Had

My team was building an internal bot to fix failing CI/CD tests. We had three engineers advocating for three different harnesses: one wanted Claude Code, another Codex, and another Pi AI. Without an abstraction layer, we would have had to maintain three forks of the same bot, with three different SDKs, three logging systems, and three ways to track costs. It would have been an impossible maintenance burden.

How Lite-Harness Helped

The SDK solved that exact pain point in three concrete dimensions:

1. Unified Invocation (Time Savings)

Instead of maintaining three separate implementations, I had a single query() that routed to whichever harness I wanted. Switching from Claude Code to Codex was literally just changing a string in the options. This allowed us to do real A/B testing in production for two weeks without rewriting any core logic.

2. Cost Observability (The Killer Feature)

By connecting it to the LiteLLM AI Gateway, I could suddenly see on a single dashboard:

• Claude Code resolved 78% of tests, averaging 4 iterations and $0.12 per fix.
• Codex resolved 65% of tests with 6 iterations and $0.08 per fix.
• Pi AI was cheaper but failed on tests involving complex mocks.

Without the gateway, tracking the real cost of an agent (which makes multiple sequential tool calls) is a nightmare of scattered logs.

3. Future Portability

When Anthropic released new capabilities in Claude Opus 4.8, I just updated the model string. I didn't have to touch the bot's underlying code. That's the real promise of LiteLLM: decoupling your application from the provider.

What Hit Me (Lessons Learned)

• It doesn't unify behavior, only invocation. Each harness interprets the prompt and environment differently. We had to normalize our prompts with highly explicit instructions (e.g., "use grep before editing", "do not modify test files").
• Lacks native iteration control. Without a built-in max_iterations, an agent can burn $5 in tokens if it gets stuck in an infinite loop. I had to wrap the query() call in an asyncio.wait_for with a strict timeout to protect our budget.
• Custom tool injection is limited. If your agent needs to call internal APIs (Jira, Slack, internal DBs), the abstraction quickly becomes too restrictive. For those complex use cases, you end up dropping down to the harness's native SDK anyway.

Final Verdict

Lite-Harness probably saved me 3 weeks of integration work and gave me hard data to make an informed architecture decision. We ended up choosing Claude Code as our primary harness and Codex as a fallback for simpler, cost-sensitive tasks.