User reference / Run an experiment
5. Analyse the run
Generate a separate assessment, choose the analyst and preserve successive analyses.
After a run has a selected plan, use its UUID and the base configuration for its storage and analyst:
slow-thinker-report \
--base config/slow-thinker.study.json \
--analyze d7442a5f-fd3b-438a-8816-91f4625f2492
This command makes additional paid model calls, saves a new analysis and regenerates reports. It leaves the proposals, votes and selected plan unchanged. --analyze all processes runs that have no stored analysis; it does not automatically upgrade old analyses. Specify a task UUID to add a fresh assessment of a previously analysed run.
| Stage | Information and output |
|---|---|
| One call per recorded roundReviewed A 0.1.4 · R 0.1.16 | Round 0: summarize each independent proposal, shared features and differences. Refinements: compare each slot to its previous version and peers, identify improvements, regressions, taken and rejected ideas, and judge convergence. These calls run in parallel. |
| Blind outcomeReviewed A 0.1.4 · R 0.1.16 | Assess and rank final proposals before seeing the votes. Compare the analyst's first choice with each initial proposal. |
| Process evaluationReviewed A 0.1.4 · R 0.1.16 | A separate call considers the collaboration, issues and suggested changes. It knows the analyst's first choice, not the vote. |
| Vote comparisonReviewed A 0.1.4 · R 0.1.16 | Reveal the actual winner and voting justifications, then ask whether the analyst agrees with the vote. |
| BriefReviewed A 0.1.4 · R 0.1.16 | Summarize what the preceding analysis already said in at most eight requested points. It is not a new evaluation of the original challenge. |
Analyst 0.1.4 uses recorded rounds + 4 calls. The stored call list is authoritative for an existing assessment. Some older CLI help and report placeholders describe fewer calls; they do not describe this current pipeline. Analyst 0.1.3 introduced a separate process evaluation; 0.1.4 added the Brief. See changelog.
analysis_llm selects its provider, model and reasoning settings. Without it, the first council model is used. A different analyst can disagree with a previous one without changing the experiment. The blind outcome hides the votes, not the identities of proposal authors: analysis prompts still include agent and model labels.
The analyst receives full proposal text. Refinement analysis also receives the report's title-based comparisons as evidence; it is not independent of those heuristic hints. Its inferences can be insightful or wrong. Read cited steps and raw exchanges when an attribution matters.
Every successful analysis is a separate UUID record. The newest is shown in report.html; earlier analyses get report.a<first-eight-analysis-id-characters>.html pages and remain accessible from About.
Answers are validated before saving. Round coverage and the final ranking must match the expected proposal numbers, and current process evaluation and Brief cannot be empty. If a call or validation fails, no partial analysis record is saved, although completed API calls may already have incurred cost. The failed attempt is therefore not represented by a complete analysis in report totals.
Continue with report generation, or inspect the technical analysis schema.
Source files used for this reference
Reviewed at 7361ea0b71a5.