Technical reference / Data & methods

Analysis pipeline & schema

The evidence supplied to the analyst, its structured answers and their validation.

Reviewed 2026-09-21Analysis generator 0.1.4

Call graph and information boundaries

For K recorded proposal rounds, analyst 0.1.4 requests K + 4 calls. All K round analyses run concurrently. Outcome, process evaluation, vote comparison and Brief then run sequentially, in that order. A record is built only after all calls and validations succeed.

Call	Inputs and output model
Round 0Reviewed A 0.1.4	Task and complete initial proposals → `InitialRoundAnalysis`.
Refinement roundReviewed A 0.1.4	Task, preceding and current proposals, deterministic closest-proposal diffs and title-origin hints → `RefinementRoundAnalysis`.
OutcomeReviewed A 0.1.4	Task, initial proposals, round-analysis summaries and final proposals, without votes → `OutcomeAnalysis`.
ProcessReviewed A 0.1.4	Task, initial/final proposals, round summaries and blind first choice → `ProcessEvaluation`.
Vote comparisonReviewed A 0.1.4	Task, blind ranking/reasons, saved votes/justifications and winner/counts → `VoteComparison`.
BriefReviewed A 0.1.4	Task/context plus the analyst's existing round, outcome, process and vote summaries → `Brief`.

The analysis uses full step renderings, not the executor's max_steps_in_summaries cap. It includes author/model labels. “Blind” means blind to voting outcome during initial final assessment; it does not mean anonymous authorship or independence from title-based evidence.

Round answer fields

Documented in A 0.1.4

Field	Type and meaning
`round_number`Reviewed A 0.1.4	Added by the pipeline to identify the assessed round.
`round_summary`Reviewed A 0.1.4	String summarizing the round.
Initial `shared`Reviewed A 0.1.4	List of common features; introduced in schema 5.
`differences`Reviewed A 0.1.4	List of what distinguishes the proposals; schema 5.
Refinement `converging`Reviewed A 0.1.4	Boolean judgment: are current proposals more similar to one another than the preceding round's?
Refinement `influences`Reviewed A 0.1.4	High-level list of who took what and what remained untaken; schema 5.
`proposals[].proposal`Reviewed A 0.1.4	Agent-slot number, checked against the round's expected proposals.
Initial `summary` / `approach`Reviewed A 0.1.4	Text summary and concise angle of a proposal.
Refinement `assessment`Reviewed A 0.1.4	Enum: improved, worsened, mixed, unchanged, no_previous_version.
`what_changed`Reviewed A 0.1.4	Text describing the change against the same-slot preceding version.
`improvements` / `regressions`Reviewed A 0.1.4	Lists of concrete gains/losses.
`taken` / `rejected`Reviewed A 0.1.4	Lists of structured `Influence` objects, available from schema 3.

Each Influence has from_proposal (source slot), steps (source step numbers, possibly empty), what (idea) and why (evidence of adoption or rejection). Step references accept numbers or labels such as S12; the parser extracts digit sequences. They are not UUID references. The prompt asks for evidence of rejection, not mere omission.

Final answer fields

Documented in A 0.1.4

Field	Type and interpretation
`summary`Reviewed A 0.1.4	String requested as a Markdown list summarizing final proposals and common ground.
`assessments[].proposal`Reviewed A 0.1.4	Final proposal's slot number.
`assessments[].fitness`Reviewed A 0.1.4	Enum: strong, adequate, weak.
`assessments[].strengths` / `.weaknesses`Reviewed A 0.1.4	Lists of concrete evaluation points.
`ranking`Reviewed A 0.1.4	Ordered list of final proposal numbers. Checked to cover the final alternatives exactly.
`ranking_reasons`Reviewed A 0.1.4	Explanation of ordering and tradeoffs.
`versus_initial[].proposal`Reviewed A 0.1.4	Initial proposal being used as comparison.
`versus_initial[].verdict`Reviewed A 0.1.4	Enum better/worse/similar, for analyst first choice relative to that initial proposal.
`versus_initial[].why` / `.how`Reviewed A 0.1.4	Reasons and concrete changes.
`improved_over_initial`Reviewed A 0.1.4	Boolean: analyst first choice is better than every initial proposal.
`improvement_summary`Reviewed A 0.1.4	Overall gains/losses.
`process_evaluation`Reviewed A 0.1.4	Separate process assessment text; must be nonempty in the current pipeline.
`process_issues` / `suggestions`Reviewed A 0.1.4	Lists of problems and proposed process/prompt improvements.
`vote_comparison.agrees` / `.comment`Reviewed A 0.1.4	Boolean and explanation after revealing actual votes.
`brief`Reviewed A 0.1.4	Nonempty final summary of the preceding assessment, requested in at most eight points.

These fields are merged into content.final; per-round answers form content.rounds; content.version is 5. Raw prompts/responses are concatenated with stage headers, and llm_calls stores a separate usage object for each stage.

Validation and what it does not prove

Documented in A 0.1.4

Pydantic validates shapes and enumerated categories. Several list fields tolerate a single sentence and normalize it to a one-item list. Round answers must cover the expected proposal slots, and the ranking must contain exactly the expected final slots. Empty process evaluations and briefs are rejected.

This is not an evidence verifier: the implementation does not comprehensively validate each cited source step against the actual text, prove that a described idea is present, establish factual correctness or calibrate fitness labels. Nor are requested bullet limits universally enforced. Treat structured output as an inspectable assessment, not a certified annotation dataset.

Compatibility

Documented in A 0.1.4

Renderer 0.1.16 supports analysis schemas 2–5. Schema 2 lacks structured influences; schema 3 adds them; schema 4 introduces blind final ranking and vote comparison; schema 5 adds the round lists. Later component builds can add fields while keeping schema 5, so read analyst_version and actual content as well as the schema number. Unsupported schemas trigger an outdated notice.

Source files used for this reference

Reviewed at 7361ea0b71a5.

← Documentation home Find a report element →