Agent-Friendly Interactive CLIs

ai/agents cli developertools conference videonotes

Core Idea

How can a command-line interface retain its interactive guidance and guardrails while becoming reliably usable by AI agents?

Workshop answer: Keep the CLI deterministic, expose machine-readable state and actions, give agents task-specific context, and use tmux as a practical bridge for existing terminal UIs.
Three ideas from the closing slide: tmux → IPC side channels → context injection.

Video

## Why interactive CLIs fail with agents

The problem: a human can see the prompt; an agent may not

Agents routinely execute shell commands. But an interactive CLI may render a selection menu, wait for arrow keys, or redraw part of the terminal. A human sees an obvious question; an agent’s execution harness may capture only escape sequences or repeated screen redraws.

Silent failure mode

A tool can appear to run without raising an error, while the agent never receives the actual prompt or loses the interactive UI state. No error does not mean success.

The instinctive workaround is to add more flags. That works when inputs are known beforehand, choices are independent, and the command runs in one pass. It breaks down when later questions depend on previous responses, external API results, or live environmental conditions.

Approach	Works well when…	Main limitation
CLI flags	The operation is single-pass and inputs are predictable	Hard to represent changing decision trees
Interactive terminal UI	A human benefits from guided, contextual questions	Agent harnesses may not understand terminal state
Machine-readable interaction	An agent needs to inspect state and choose its next action	Requires explicit structure and a reliable interface

Key distinction: A conventional script follows predefined decisions; an agent can interpret information, retry, and navigate a decision tree if the CLI exposes that tree clearly.

The accessibility analogy

The speakers compare agent-facing interfaces with accessibility work: keep one underlying capability, but make it perceivable and operable for a different kind of user.

The three proposed techniques

1. Use tmux to preserve terminal state

tmux keeps a program running in a persistent terminal session. An agent can send keypresses and capture the current pane, instead of expecting a single shell-command response to contain the entire interaction.

What it provides: persistent sessions, terminal inspection, multi-pane operation, and a way to resume long-running workflows.

Illustrative tmux workflow

These commands are an example of the technique, not copied from the workshop’s CLI.

tmux new-session -d -s cafe
tmux send-keys -t cafe 'barista brew' Enter
tmux capture-pane -p -t cafe
# Send further keys after inspecting the captured terminal state

Quick to adopt, but the agent still has to infer meaning from a terminal screen. It is a bridge, not a fully structured interface.

2. Add IPC side channels with structured data

A side channel is a machine-readable companion to the human-facing terminal interface. The workshop compares it to a web application offering both a website for people and an API for software.

Instead of scraping the screen, the agent should be able to:

1. Read the current state as structured data.
2. Discover valid choices and required inputs.
3. Submit a choice in a predictable format.
4. Receive the next state or a clear result.

Conceptual JSON shape

Illustrative schema based on the talk’s size-selection example, not an asserted API from the repository.

{
  "step": "choose_size",
  "field": "size",
  "options": ["small", "medium", "large"],
  "default": "large",
  "required": true
}

JSON makes the question, allowed answers, and current context explicit. The agent responds to data instead of reconstructing meaning from arrow-key menus.

3. Inject context that explains the decision

Structured options tell the agent **what it can do**. Context tells it **why a choice matters** and **when to involve the user**.

The workshop’s example: the café has only a small amount of oat milk left. A useful agent-facing interface would provide enough information to ask whether a reduced amount is acceptable or whether a substitute should be chosen.

Useful context is actionable

Include relevant inventory or API state, constraints, defaults, alternatives, and situations that require confirmation. Don’t merely duplicate the text displayed to a human.

A practical design pattern

flowchart TD
    H[Human user] --> T[Interactive terminal UI]
    A[AI agent] --> M[Structured state and action interface]
    T --> C[Shared deterministic CLI logic]
    M --> C
    C --> V[Validation and guardrails]
    V --> S[External services or local state]
    S --> C
    C --> M
    C --> T

This diagram synthesizes the workshop’s recommendations; it was not presented as an exact architecture.

Keep the CLI deterministic

The speakers advocate treating the CLI as a small, sharp tool. Given the same command and relevant state, it should behave predictably. Let the agent handle reasoning; don’t hide a second agent inside the CLI.

Agent-friendly CLI design checklist

• Provide accurate, discoverable --help or equivalent command documentation.
• Keep core operations deterministic and narrowly scoped.
• Return structured output for state, choices, success, and failure.
• Express dependencies between steps explicitly, instead of multiplying flags.
• Expose valid values, sensible defaults, and constraints.
• Supply real-time context when external conditions change the next decision.
• Make confirmation requirements and guardrails visible to the agent.
• Preserve clear error messages and support retries or correction.
• Test through an actual agent harness, not only in a human terminal.
• Use tmux as a compatibility layer for terminal-only workflows.

Checklist = practical synthesis of the ideas in the workshop, not an official specification.

Workshop exercise: Barista 9000

Challenge: Attendees received a barebones café CLI that takes coffee orders but is difficult for agents because it is interactive. They were asked to modify the tool, run it via an agent, register a username, and complete a coffee-ordering workflow. Results went to a leaderboard; a Mac Mini was offered as the prize.

Constraint that matters: The presenters suggested optimizing for moderate-cost models, not merely relying on a highly capable model to brute-force difficult terminal interaction. The engineering objective was to reduce the reasoning and token cost required to use the tool.

What attendees learned from the competition

• Scoring: primarily accuracy, with response time used to break ties.
• Benchmark weakness: one participant noted that a benchmark centered on one order and very low latency could be gamed using a deterministic parser and favorable network placement.
• Winning approach: a participant described using an agent to build a deterministic CLI solver and then run it through tmux; the reported result benefited from low network latency.
• Accessibility of development: another participant, with little prior coding experience, described directing coding agents to iteratively build and speed up a solution.

Don't confuse leaderboard success with robust agent usability

A specialized solver can perform extremely well on a narrow evaluation without demonstrating that the CLI supports general, changing, multi-step workflows. This limitation emerged during the audience discussion.

Mental models worth remembering

Idea	One-line mental model
Agent ≠ script	A script executes a fixed path; an agent chooses the next step from information provided.
Flags ≠ decision tree	Flags are good parameters, but become awkward when questions depend on earlier answers.
tmux = eyes and hands	The agent can inspect a terminal and send keys, albeit imperfectly.
Side channel = API for interaction	Express prompts, options, and actions as data.
Context injection = situational awareness	Share the constraints that determine a safe, useful next step.
Determinism = reliable tool boundary	Keep non-deterministic reasoning in the agent, not hidden in the CLI.

Questions to apply to your own CLI

1. If a prompt appears mid-command, can the agent detect it?
2. Can the agent obtain the current state and legal next actions without rendering a terminal?
3. If an API response changes what is allowed, how does the agent learn that?
4. Are validation and confirmation requirements expressed in machine-readable form?
5. Can an inexpensive model complete the workflow reliably and with few retries?
6. Does your benchmark measure robustness across scenarios, rather than only fastest success on one task?

Final takeaway

Design the workflow for two readers

Humans need a navigable interface. Agents need explicit state, actions, and constraints. Preserve the same reliable underlying tool, but expose it in forms each reader can use.

Start pragmatically with tmux, improve reliability through structured side channels, and improve decision quality with context injection.