User reference / Run an experiment
1. Configuration
Install the tools, connect providers and choose where experiment records are stored.
Use Python 3.11 or later. From a checkout of the repository:
python -m venv .venv
source .venv/bin/activate
python -m pip install -e '.[report]'
slow-thinker --help
slow-thinker-report --help
The report extra installs Jinja2 and markdown-it-py. The planner can run without it, but interactive report generation requires it. On Windows, activate with .venv\Scripts\activate instead. This documentation describes the checked-in provider adapters and configurations, not a promise that every listed model is available to every API account.
The published base configuration inherits ../slow-thinker.keys.json. Create that file from the empty template:
cp slow-thinker.keys.example.json slow-thinker.keys.json
Fill the entries for the providers you use. Alternatively, leave the relevant entries empty and provide their environment variables. An empty keys file is still needed if the selected base chain references it.
| Provider | Environment variable |
|---|---|
openaiReviewed E 0.1.5 |
OPENAI_API_KEY |
anthropicReviewed E 0.1.5 |
ANTHROPIC_API_KEY |
googleReviewed E 0.1.5 |
GOOGLE_API_KEY |
xaiReviewed E 0.1.5 |
XAI_API_KEY |
deepseekReviewed E 0.1.5 |
DEEPSEEK_API_KEY |
alibabaReviewed E 0.1.5 |
DASHSCOPE_API_KEY |
Resolution is model entry api_key, then api_keys[provider], then the provider's environment variable. Missing keys stop configuration loading before the run. Saved model descriptions omit API keys. Raw prompts and responses are preserved, so the challenge text and any sensitive material you put in it become part of the trace and report.
The repository's config/slow-thinker.base.json is the starter profile. It configures three proposers, two refinements and three voters, with a separate analyst. The cheap, study and full JSON profiles inherit it and replace the relevant settings.
slow-thinker examples/software_incident_management_process.json \
--base config/slow-thinker.study.json
The task's own fields override the chosen base. Configuration merging replaces an entire key: defining llm_configs replaces that list; defining api_keys replaces that dictionary. It does not merge nested entries. See the complete configuration reference.
| Setting | Effect |
|---|---|
db_config: {} or omittedReviewed E 0.1.5 |
JSON storage, under runs_dir, default runs. This is the easiest way to inspect and share experiment data. |
Nonempty db_configReviewed E 0.1.5 |
PostgreSQL storage. Apply the supplied database schema before execution. The HTML output still goes to a filesystem directory. |
runs_dirReviewed E 0.1.5 |
JSON storage location when using files, and the default report output directory. Relative paths are resolved from the command's working directory. |
reportReviewed E 0.1.5 |
Defaults to true: after execution, generate reports if the report dependencies are installed. It does not request an LLM analysis. |
For database connection fields, schema setup and storage failure behavior, see Saved records & storage.
Check that the selected configuration references existing base files, every chosen provider has a key and the output directory is writable. The first real execution makes paid API calls. Help commands and report rendering alone do not run an experiment. Configure a small council first if you want to verify an account's model access.
Continue with case definition.
Source files used for this reference
Reviewed at 7361ea0b71a5.