Configuration

Credentials

Open Settings in the workbench to set or replace credentials. The same values can be written from the CLI. The Gateway key is required for hosted models only—a local CLI model needs no credential at all. Logfire credentials are optional and scoped by purpose.

valcore config set-key

valcore config set-logfire-token

valcore config set-logfire-read-key

valcore config set-logfire-write-key

An optional Logfire frontend application traces the workbench in your browser. Its trace URL and restricted public token are set under Settings → Logfire Frontend Observability; see Browser observability.

Models and routes

A model string names one of two routes. Gateway routes reach a hosted model through the Pydantic AI Gateway and need a Gateway key. Local CLI routes reuse a coding-agent CLI already installed and logged in on this machine, and need no key at all. Anything that matches neither shape is rejected before a request is made.

Gateway models

Hosted model strings use gateway/<provider>:<model>. A bare provider model name such as claude-sonnet-5 fails fast with a configuration error rather than at call time.

gateway/anthropic:claude-sonnet-5   # the built-in default
gateway/openai:gpt-5
gateway/google:gemini-2.5-pro

Supported Gateway routes are anthropic, openai, google, google-cloud, bedrock, and groq. Suggestions in the editor come from the version of pydantic-ai installed with Valcore, but any well-formed Gateway model string can be entered.

Local CLI models

Local model strings name a CLI and nothing else. There is no model name after the CLI: each route shells out to that tool's own binary, which answers with whichever model it is already configured to use.

local/claude    # the claude binary (Claude Code)
local/codex     # the codex binary (Codex CLI)
local/cursor    # the cursor-agent binary (Cursor CLI)

See Running on a local CLI for requirements and the trade-offs against a Gateway model.

Running on a local CLI

A local CLI route lets Valcore evaluate without a Gateway key by driving a coding-agent CLI you have already installed and authenticated. This is the fastest way to try Valcore, and it keeps model spend on an existing subscription instead of a second credential.

Requirements

Each route requires that CLI's binary on your PATH and already logged in. Valcore does not manage the login.

Selecting one

Open Settings → Model Selection and pick a CLI from the Local CLI default dropdown. Valcore stores it as local_cli_default in ~/.valcore/config.toml and it becomes the default model for new evaluator versions and every generation call. Choose None (use the gateway) to clear it.

There is no dedicated config set- command for this. Outside the UI, edit the config file or set the environment variable:

# Open ~/.valcore/config.toml and add: local_cli_default = "claude"
valcore config edit

# Or override without touching the file, for one shell or a CI job
export VALCORE_DEFAULT_MODEL=local/claude

The Default model card on Overview shows the model that actually resolves, which is not always the stored local_cli_default—VALCORE_DEFAULT_MODEL outranks it. An individual evaluator version can still name a local or Gateway model explicitly, whatever the default is.

What a local model gives up

A local CLI is a different execution surface from a Gateway model, and Valcore enforces the difference rather than failing at run time.

Defaults and precedence

Runtime settings resolve from highest to lowest priority: an explicit command or API argument, a VALCORE_* environment variable, config.toml, then the built-in default.

local_cli_default outranks model within config.toml: a CLI selected in Settings → Model Selection wins over a stored Gateway model string, and clearing the selection falls back to model. Both are outranked by VALCORE_DEFAULT_MODEL.

An exported PYDANTIC_AI_GATEWAY_API_KEY or LOGFIRE_TOKEN takes precedence over its stored value. Logfire read and write API keys are read from the config file rather than exported to the environment.

Logfire project boundary

Most teams should use two projects. The source agent project contains production traces and any hosted Datasets you want to sample; the read key points there. The Valcore project receives workbench telemetry, Pydantic Experiment Runs, and published Datasets; the tracing token and write key point there.

If both roles genuinely use one project, valcore config set-logfire-key stores one API key as both read and write. Otherwise, keep the scopes separate.

See the Logfire integration guide for credential creation, exact scopes, setup, verification, and every Agent Evaluator, Dataset, and Experiment Run touchpoint.

Storage and security

Configuration is stored at ~/.valcore/config.toml. On macOS and Linux it is written with mode 0600; on Windows, access is controlled by the current user's filesystem ACLs. Settings masks stored credentials, and valcore config get reports Logfire credentials only as present or absent. The Gateway key is also masked unless --show-key is explicitly supplied.

valcore config get
valcore config get --json
valcore config path
valcore config edit

Valcore warns if the config file is group- or world-readable. See CLI & automation for relocating the workspace or choosing another database.