llm_cli

Imported from yashs33244/devops-agent (agents/opensre/app/integrations/llm_cli/AGENTS.md). Install upstream with npx skills add yashs33244/devops-agent --skill llm_cli. Copyright stays with the author.

LLM CLI providers (subprocess)

Use this package when adding a new non-interactive LLM that shells out to a vendor CLI (like OpenAI Codex), instead of HTTP APIs.

Layout

Wiring a new provider

Before merging, read Subprocess environment allowlist below: if your CLI reads vendor-specific env vars, you must extend _SAFE_SUBPROCESS_ENV_PREFIXES in subprocess_env.py or the subprocess will not see them (auth and config will break silently).

1. Adapter — Implement LLMCLIAdapter: detect() must not raise; build() returns argv + optional stdin; parse / explain_failure for success and non-zero exits. Put prompt text on stdin and/or in argv as appropriate — the runner does not branch on a separate “delivery mode”; CLIInvocation carries what build() produced.
2. Registry — Add an entry to CLI_PROVIDER_REGISTRY in registry.py (adapter_factory, model_env_key). The dict key must match LLM_PROVIDER / ProviderOption.value / LLMProvider in app/config.py. _create_llm_client picks up registered CLI providers automatically (no new elif in llm_client.py for normal cases).
3. Config — Add the provider literal to LLMProvider and validators in app/config.py (same string as the registry key).
4. Wizard (optional) — If onboarding should offer the CLI: add a ProviderOption in app/cli/wizard/config.py with credential_kind="cli" and adapter_factory. flow.py already runs _run_cli_llm_onboarding for CLI providers and builds the saved-summary credential line from provider.label + adapter.auth_hint.
5. Typing — Prefer adapter_factory: Callable[[], LLMCLIAdapter] on ProviderOption so wizard and client stay aligned.

Binary resolution (recommended pattern)

Use binary_resolver.resolve_cli_binary(...) so all adapters share the same behavior.

Resolution order:

1. Explicit binary env var (<PROVIDER>_BIN, e.g. Codex CODEX_BIN) only if it points to a runnable file.
2. shutil.which(...) lookups for platform-specific binary names.
3. Fallback install locations from default_cli_fallback_paths(...).

Notes:

• Binary env vars are optional by default.
• Blank/invalid explicit paths are ignored; PATH/fallback resolution still runs.
• For Codex, keep this behavior: users can run with no CODEX_BIN.

Conventions

• No TTY: invocation must be suitable for subprocess.run without an interactive session.
• Probe vs run: detect() is cheap; CLIBackedLLMClient.invoke probes again before exec so missing auth fails fast with a clear error.
• Structured output: CLIBackedLLMClient.with_structured_output delegates to StructuredOutputClient (JSON-in-prompt), same pattern as API clients.
• Optional model envs: use <PROVIDER>_MODEL (see Per-provider env vars); always optional—if unset, rely on vendor CLI defaults.

Per-provider env vars (required for every new CLI)

Codex is the reference. Every subprocess LLM must expose the same shape of knobs:

Naming: derive <PROVIDER> from the registry / LLM_PROVIDER string: uppercase, then _BIN / _MODEL. Examples: codex → CODEX_BIN, CODEX_MODEL; a future gemini → GEMINI_BIN, GEMINI_MODEL.

Document both vars in the adapter module docstring or a one-line comment near binary_env_key / the registration entry so users and wizard copy stay aligned.

Auth probe pattern

detect() must return a CLIProbe with logged_in: bool | None — three states:

Recommended probe sequence (mirrors Codex):

1. Run <binary> --version — if it fails, return installed=False immediately.
2. Run <binary> <auth-status-command> — parse stdout/stderr to classify logged_in.
3. Write a _classify_<name>_auth(returncode, stdout, stderr) -> tuple[bool | None, str] helper. Check negative phrases first (e.g. "not logged in" before "logged in") to avoid substring false-positives.
4. Map network/timeout errors to None, not False — the user may be on a flaky connection and shouldn't be forced to re-authenticate.

See _classify_codex_auth in codex.py for a complete reference implementation.

OpenCode is multi-provider: users may rely on auth.json, environment API keys, or both. Run opencode auth list after --version and parse the reported credential/environment counts so detection matches the CLI (do not infer auth from the JSON file alone).

Kimi uses kimi login status after --version, then _check_kimi_auth_fallback() when the CLI did not positively confirm auth (logged_in not True): check KIMI_API_KEY, then API keys in ~/.kimi/config.toml (or KIMI_SHARE_DIR). API-key-only installs may omit “logged in” phrasing in login status, so the fallback mirrors real usage. The same fallback runs when login status times out or fails to spawn (logged_in=None), so a configured API key still counts as authenticated.

Subprocess environment allowlist

CLIBackedLLMClient passes only a safe subset of env vars to the subprocess via build_cli_subprocess_env in subprocess_env.py (_SAFE_SUBPROCESS_ENV_KEYS + _SAFE_SUBPROCESS_ENV_PREFIXES).

Shared HTTP/API overrides live in env_overrides.py: use nonempty_env_values(...) with OPENAI_PLATFORM_ENV_KEYS (Codex), HTTP_LLM_PROVIDER_ENV_KEYS (OpenCode), ANTHROPIC_CLI_ENV_KEYS (Claude Code), CURSOR_CLI_ENV_KEYS (Cursor Agent headless API key), or COPILOT_CLI_ENV_KEYS (Copilot CLI credential envs: COPILOT_GITHUB_TOKEN, GH_TOKEN, GITHUB_TOKEN) plus COPILOT_CLI_CONFIG_ENV_KEYS (COPILOT_HOME, COPILOT_MODEL, COPILOT_GH_HOST, GH_HOST). Extend those tuples when you add a matching API-key env to LLMSettings.

Kimi does not use those tuples today: OAuth/API material is covered by forwarding any KIMI_* keys via _SAFE_SUBPROCESS_ENV_PREFIXES; KimiAdapter.build() uses CLIInvocation(env=None) and relies on that allowlist.

The current prefix allowlist includes CODEX_, CURSOR_, CLAUDE_, OPENCODE_, KIMI_, and locale keys (LC_). COPILOT_ is deliberately NOT a prefix entry — COPILOT_GITHUB_TOKEN is a GitHub PAT, and a blanket prefix would leak it into every other CLI subprocess; the Copilot adapter forwards every Copilot-scoped env via its own CLIInvocation.env instead (see COPILOT_CLI_ENV_KEYS / COPILOT_CLI_CONFIG_ENV_KEYS in env_overrides.py).

If your CLI reads custom env vars (e.g. GEMINI_*) you must add the relevant prefix to _SAFE_SUBPROCESS_ENV_PREFIXES in subprocess_env.py, otherwise the subprocess will not receive those vars and authentication or configuration will silently fail. Add a test that asserts the required keys are forwarded.

Codex binary resolution (reference)

Order in CodexAdapter._resolve_binary (now delegated to shared resolver helpers):

1. CODEX_BIN if set and path is runnable (explicit override).
2. shutil.which("codex") (and Windows codex.cmd / codex.ps1).
3. _fallback_codex_paths() — conventional install locations; invalid or blank CODEX_BIN is ignored so PATH/fallbacks still apply.

Codex env quick reference (instance of the convention above)

All optional:

CODEX_MODEL=
CODEX_BIN=

• If CODEX_MODEL is unset, codex exec uses its default model behavior.
• If CODEX_BIN is unset, adapter resolution falls back to PATH + known install locations.

Provider checklist (copy/paste)

• Add adapter in app/integrations/llm_cli/.
• Define <PROVIDER>_BIN + <PROVIDER>_MODEL per Per-provider env vars; reuse resolve_cli_binary(..., explicit_env_key=...) for _resolve_binary.
• Implement detect() with --version + auth status checks; follow the three-state logged_in pattern above.
• Write _classify_<name>_auth — test against a real logged-in and logged-out session before merging.
• If the CLI reads custom env vars (e.g. GEMINI_*), add the prefix to _SAFE_SUBPROCESS_ENV_PREFIXES in subprocess_env.py.
• Register the provider in registry.py and add the same LLM_PROVIDER value in app/config.py.
• (Optional) Add wizard onboarding option in app/cli/wizard/config.py.
• Add tests under tests/integrations/llm_cli/ for detect/build/failure paths, including env forwarding.

Tests

• tests/integrations/llm_cli/ — adapter and runner unit tests; mock subprocess / shutil.which as needed.
• Platform-specific assertions must patch app.integrations.llm_cli.binary_resolver.sys.platform (not codex.sys.platform), because resolution lives in binary_resolver.py.
• npm_prefix_bin_dirs is @lru_cached; tests that vary env or platform should call npm_prefix_bin_dirs.cache_clear() before each case (or use a shared autouse fixture) to avoid stale cache across tests.