Skip to content

FE-1665: Validate explicit model snapshots for AI requests - #9675

Merged
kube merged 4 commits into
mainfrom
codex/fe-1665-snapshot-diagnostics
Sep 15, 2026
Merged

kube merged 4 commits into
mainfrom
codex/fe-1665-snapshot-diagnostics

Conversation

@kube

@kube kube commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The AI diagnostic tool describes validation of the captured net, and a result without errors says that no net-code diagnostics were found. It no longer claims that the whole model compiles.

This PR also tests and documents the snapshot request API, which is available on the base branch. The tests verify that editor diagnostic updates cannot complete another request and that disposing the client rejects pending requests.

Snapshot diagnostics

Links

Changes

  • Describe TypeScript and HIR diagnostics in the AI tool's schema
  • Report a diagnostic check without errors without claiming full compilation success
  • Keep React and React DOM subpaths external in the library build
  • Update Brunch browser assertions for the diagnostic result without errors
  • Document request correlation, isolation from editor sessions, and current limitations
  • Include a patch changeset for each published package

Known issues

Requests have no timeout; an unresponsive worker can leave a check pending. Moving diagnostics onto PetrinautInstance needs a headless contract, tracked in FE-754.

Test coverage

Core, editor, website, and Brunch builds, unit tests, TypeScript, and lint passed on the integrated stack. Architecture validation, the docs site build, and formatting also passed. The website suite needed one retry for an asynchronous history-loading assertion. Browser verification and its remaining limitations are recorded in PR #9678.

  • Language-client tests cover request correlation, unchanged diagnostics, editor diagnostic isolation, and disposal
  • AI panel and current-diagnostics tests cover repeated checks without errors and rejection of stale model results
  • Brunch browser tests check error-free results after model construction and compiler repair

How to test

  • Run yarn workspace @hashintel/petrinaut-core test:unit --run src/lsp/language-client.test.ts
  • Run yarn workspace @hashintel/petrinaut test:unit --run src/ui/views/Editor/panels/ai-assistant-panel.test.tsx src/ui/views/Editor/panels/ai-assistant-panel/read-current-diagnostics.test.ts

@kube kube self-assigned this Sep 11, 2026
@vercel

vercel Bot commented Sep 11, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hash Ready Ready Preview Sep 14, 2026 7:37pm UTC
petrinaut Ready Ready Preview Sep 14, 2026 7:37pm UTC
petrinaut-docs Ready Ready Preview Sep 14, 2026 7:37pm UTC
1 Skipped Deployment
Project Deployment Actions Updated
hashdotdesign-tokens Ignored Ignored Preview Sep 14, 2026 7:37pm UTC

Request Review

@github-actions github-actions Bot added area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > frontend Owned by the @frontend team type/eng > backend Owned by the @backend team area/apps labels Sep 11, 2026
@kube
kube added this pull request to stack #9679 September 11, 2026 14:48
@kube
kube marked this pull request as ready for review September 11, 2026 18:33
Copilot AI balanced review requested due to automatic review settings September 11, 2026 18:33
@cursor

cursor Bot commented Sep 11, 2026 •

Copy link
Copy Markdown

PR Summary

Low Risk
Changes are mostly messaging, documentation, tests, and bundler externals; AI diagnostic behavior is clarified rather than broadening mutation or auth paths.

Overview
The AI getNetCompilationErrors tool is described as validating a captured net snapshot and returning TypeScript and HIR diagnostics, not as proof the whole model compiles. A clean check now reports "No errors or warnings found in net function code." instead of claiming everything compiles; the same wording flows through formatDiagnosticsForAi, user-facing docs, and Brunch/browser integration expectations.

petrinaut-core gains unit tests that snapshot requestDiagnostics responses are correlated by request ID (editor diagnostic pushes cannot complete another caller’s check) and that disposing the language client rejects pending snapshot requests. Architecture docs add a snapshot diagnostics page and diagram; the AI assistant doc explains explicit checks vs. the Diagnostics tab.

@hashintel/petrinaut library build treats react and react-dom subpath imports as external (regex), and drawer/panel tests stub requestDiagnostics with a resolved empty result. Patch changesets cover petrinaut and petrinaut-core.

Reviewed by Cursor Bugbot for commit 4a068ce. Bugbot is set up for automated code reviews on this repo. Configure here.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Four unresolved findings remain, including one critical metric-ID collision and three moderate optimization/UI issues.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR adds correlated snapshot diagnostics for AI checks and expands browser optimization with pause/resume, constraints, parameter importance, sweep orchestration, and related UI/documentation updates.

Changes:

  • Correlates diagnostic requests with explicit model snapshots.
  • Adds optimization lifecycle, constraint, importance, and sweep workflows.
  • Updates simulation UI, tests, documentation, and changesets.

Final findings: Moderate (2 votes) for dropped paused-event importances; Critical (1 vote) for objective/auxiliary metric ID collisions; Moderate (2 votes) for unreleased sweep studies; and Moderate (1 vote) for stale experiment selections.

File summaries
File Review status
libs/@local/petrinaut-optimizer-core/tests/test_importance.py Reviewed.
libs/@local/petrinaut-optimizer-core/src/petrinaut_optimizer_core/pyodide_entry.py Reviewed.
libs/@local/petrinaut-optimizer-core/src/petrinaut_optimizer_core/importance.py Reviewed.
libs/@local/petrinaut-optimizer-core/src/petrinaut_optimizer_core/ask_tell.py Reviewed.
libs/@local/petrinaut-optimizer-core/src/petrinaut_optimizer_core/__init__.py Reviewed.
libs/@local/petrinaut-arch-docs/content/experiments/sweep-orchestration.mdx Reviewed.
libs/@local/petrinaut-arch-docs/content/experiments/parameter-sweeps.mdx Reviewed.
libs/@local/petrinaut-arch-docs/content/diagrams/sweep-surface-feed.d2 Reviewed.
libs/@local/petrinaut-arch-docs/content/diagrams/sweep-orchestration.d2 Reviewed.
libs/@local/petrinaut-arch-docs/content/diagrams/snapshot-diagnostics.d2 Reviewed.
libs/@local/petrinaut-arch-docs/content/ai/snapshot-diagnostics.mdx Reviewed.
libs/@hashintel/petrinaut/vite.config.ts Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/shared/experiment-progress.test.ts Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/simulate-view.test.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/simulate-view.stories.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/shared/surface-frame.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/shared/surface-field.ts Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/shared/summary-strip.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/shared/results.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/shared/drawer-frame/use-prefers-reduced-motion.ts Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/shared/drawer-frame/use-overflow-end.ts Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/shared/drawer-frame/use-header-engaged.ts Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/shared/drawer-frame/use-body-scrolled.ts Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/shared/drawer-frame/frame-test-helpers.ts Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/shared/drawer-frame/frame-layout-signature.ts Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/shared/drawer-frame/frame-columns.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/shared/drawer-frame/frame-animate-context.ts Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/shared/compute-backend-badge.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/shared/chart-card.test.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/shared/chart-card.stories.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/optimizations/view-optimization-drawer/study-summary-strip.test.ts Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/optimizations/view-optimization-drawer/shared/study-progress.ts Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/optimizations/view-optimization-drawer/optimization-metrics.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/optimizations/view-optimization-drawer/navigator-band.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/optimizations/study-results/study-steps.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/optimizations/study-results/study-phase.ts Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/optimizations/study-results/shared/study-progress.ts Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/optimizations/study-results/parameter-values.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/optimizations/study-results/objective-history-data.ts Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/optimizations/study-results/convergence.ts Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/optimizations/study-results/convergence.test.ts Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/optimizations/study-results/continue-control.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/optimizations/optimizations-view.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/optimizations/optimization-surface/surface-plot.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/optimizations/optimization-surface/surface-plot.test.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/optimizations/optimization-surface.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/optimizations/optimization-status.ts Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/optimizations/optimization-full-view.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/optimizations/optimization-full-view.stories.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/optimizations/create-optimization-drawer.test.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/metrics/create-metric-drawer.test.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/experiments/view-experiment-drawer/experiment-metrics.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/experiments/sweep-surface.stories.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/experiments/sweep-navigator/range-slider.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/experiments/sweep-navigator.stories.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/experiments/create-experiment-drawer.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/experiments/create-experiment-drawer.test.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/ai-assistant-panel/format-diagnostics-for-ai.ts Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/ai-assistant-panel/format-diagnostics-for-ai.test.ts Reviewed.
libs/@hashintel/petrinaut/src/ui/views/Editor/panels/ai-assistant-panel.test.tsx Reviewed.
libs/@hashintel/petrinaut/src/ui/index.ts Reviewed.
libs/@hashintel/petrinaut/src/react/state/editor-provider.tsx Reviewed.
libs/@hashintel/petrinaut/src/react/state/editor-provider.test.tsx Reviewed.
libs/@hashintel/petrinaut/src/react/state/editor-context.ts Reviewed.
libs/@hashintel/petrinaut/src/react/optimizations/sir-optimization-input.fixtures.ts Reviewed.
libs/@hashintel/petrinaut/src/react/optimizations/provider/connected-study.test.ts Reviewed.
libs/@hashintel/petrinaut/src/react/optimizations/channel/create-optimization-channel/trial-outcome.ts Reviewed.
libs/@hashintel/petrinaut/src/react/optimizations/channel/create-optimization-channel/trial-constraints.test.ts Reviewed.
libs/@hashintel/petrinaut/src/react/optimizations/channel/create-optimization-channel.test.ts Reviewed.
libs/@hashintel/petrinaut/src/react/navigation/index.tsx Reviewed.
libs/@hashintel/petrinaut/src/react/navigation/index.test.tsx Reviewed.
libs/@hashintel/petrinaut/src/react/lsp/provider.tsx Reviewed.
libs/@hashintel/petrinaut/src/react/lsp/context.ts Reviewed.
libs/@hashintel/petrinaut/src/react/index.ts Reviewed.
libs/@hashintel/petrinaut/src/react/hooks/use-petrinaut-mutations.test.tsx Reviewed.
libs/@hashintel/petrinaut/src/react/hooks/use-petrinaut-commands.test.tsx Reviewed.
libs/@hashintel/petrinaut/src/react/experiments/sweep-session/README.md Reviewed.
libs/@hashintel/petrinaut/src/react/experiments/sweep-session/cell-batch.ts Reviewed.
libs/@hashintel/petrinaut/src/react/experiments/provider/sweep-batch-instantiation/backend-lanes.ts Reviewed.
libs/@hashintel/petrinaut/src/react/experiments/provider/detached-objective.ts Critical finding (1 vote): auxiliary metric IDs can collide with the objective ID.
libs/@hashintel/petrinaut/src/react/experiments/provider/detached-objective.test.ts Reviewed.
libs/@hashintel/petrinaut/src/react/experiments/provider/create-experiment.ts Reviewed.
libs/@hashintel/petrinaut/src/react/experiments/provider.tsx Reviewed.
libs/@hashintel/petrinaut/src/panda-preset.ts Reviewed.
libs/@hashintel/petrinaut/src/main.ts Reviewed.
libs/@hashintel/petrinaut/docs/ai-assistant.md Reviewed.
libs/@hashintel/petrinaut-core/src/simulation/monte-carlo/metrics/user-defined.ts Reviewed.
libs/@hashintel/petrinaut-core/src/simulation/monte-carlo/metrics/user-defined.test.ts Reviewed.
libs/@hashintel/petrinaut-core/src/simulation/monte-carlo/metrics/types.ts Reviewed.
libs/@hashintel/petrinaut-core/src/optimization/optimization.test.ts Reviewed.
libs/@hashintel/petrinaut-core/src/optimization/browser/worker/study-runner/as-importances.ts Reviewed.
libs/@hashintel/petrinaut-core/src/optimization/browser/worker/study-runner/as-importances.test.ts Reviewed.
libs/@hashintel/petrinaut-core/src/optimization/browser/worker/study-runner.ts Reviewed.
libs/@hashintel/petrinaut-core/src/optimization/browser/worker/study-runner.pyodide.test.ts Reviewed.
libs/@hashintel/petrinaut-core/src/optimization/browser/worker/attach-optimizer-worker.ts Reviewed.
libs/@hashintel/petrinaut-core/src/optimization/browser/worker/attach-optimizer-worker.test.ts Reviewed.
libs/@hashintel/petrinaut-core/src/optimization/browser/run-log.ts Reviewed.
libs/@hashintel/petrinaut-core/src/optimization/browser/run-log.test.ts Reviewed.
libs/@hashintel/petrinaut-core/src/optimization/browser/README.md Reviewed.
libs/@hashintel/petrinaut-core/src/optimization/browser/python-sources.ts Reviewed.
libs/@hashintel/petrinaut-core/src/optimization/browser/messages.ts Reviewed.
libs/@hashintel/petrinaut-core/src/lsp/worker/protocol.ts Reviewed.
libs/@hashintel/petrinaut-core/src/lsp/worker/language-server.worker.ts Reviewed.
libs/@hashintel/petrinaut-core/src/lsp/lib/check-snapshot.ts Reviewed.
libs/@hashintel/petrinaut-core/src/lsp/lib/check-snapshot.test.ts Reviewed.
libs/@hashintel/petrinaut-core/src/lsp/language-client.ts Reviewed.
libs/@hashintel/petrinaut-core/src/lsp/language-client.test.ts Reviewed.
libs/@hashintel/petrinaut-core/src/index.ts Reviewed.
libs/@hashintel/petrinaut-core/src/hir/interpret.ts Reviewed.
libs/@hashintel/petrinaut-core/src/constraint/indicator-metric.ts Reviewed.
libs/@hashintel/petrinaut-core/src/constraint/constraint.ts Reviewed.
libs/@hashintel/petrinaut-core/src/ai.ts Reviewed.
libs/@hashintel/ds-components/src/main.ts Reviewed.
libs/@hashintel/ds-components/src/components/Drawer/drawer.tsx Reviewed.
apps/petrinaut-website/src/main/app/voice-interview/voice-browser-tools.integration.test.tsx Reviewed.
apps/petrinaut-website/src/examples/use-shared-search-navigation.ts Reviewed.
apps/petrinaut-website/src/examples/navigation-search.ts Reviewed.
apps/petrinaut-website/src/examples/navigation-search.test.ts Reviewed.
apps/petrinaut-website/src/examples/example-search.ts Reviewed.
apps/petrinaut-website/src/examples/example-search.test.ts Reviewed.
.claude/launch.json Reviewed.
.changeset/snapshot-language-diagnostics.md Reviewed.
.changeset/simulate-drawer-frame.md Reviewed.
.changeset/shared-results-view.md Reviewed.
.changeset/optimization-paused-event.md Reviewed.
.changeset/optimization-pause-resume.md Reviewed.
.changeset/optimization-parameter-importance.md Reviewed.
.changeset/optimization-importance-events.md Reviewed.
.changeset/optimization-dedicated-view.md Reviewed.
.changeset/optimization-constraint-verdicts.md Reviewed.
.changeset/optimization-constraint-margins.md Reviewed.
.changeset/ds-slider-plain-variant.md Reviewed.
.changeset/drawer-focus-refs-nullable.md Reviewed.
.changeset/await-ai-diagnostics.md Reviewed.
Review details

Suppressed comments (1)

libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/experiments/sweep-optimize-control.tsx:88

  • This state is initialized only on the first mount, but ViewExperimentDrawer keeps the same drawer mounted while the selected experiment can change. Switching to another experiment therefore leaves the prompt's metricId pointing at the previous experiment (and Start can reject it with “Pick a metric to optimize”); the same one-time initialization affects xAxisId, yAxisId, and metricId in sweep-surface.tsx:64-66, leaving the new surface blank or showing invalid selections. Reset these selections when the experiment changes, or key the experiment content by experiment.id.
  • Files reviewed: 170/209 changed files
  • Comments generated: 3
  • Review effort level: Lite (auto)

Note

Copilot is running an experiment and ran this review at Lite.


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread libs/@hashintel/petrinaut/src/react/experiments/provider/detached-objective.ts Outdated
Comment thread libs/@hashintel/petrinaut-core/src/optimization/browser/browser-optimization.ts Outdated
Comment thread libs/@hashintel/petrinaut/src/react/optimizations/provider.tsx
An error occurred while trying to automatically change base from claude/opt-proto-experiment-optimizer to claude/opt-proto-shared-results September 13, 2026 13:59
@kube
kube removed this pull request from stack #9679 September 13, 2026 22:49
@kube
kube changed the base branch from claude/opt-proto-experiment-optimizer to main September 13, 2026 22:51
@kube
kube removed this pull request from stack #9698 September 14, 2026 15:50
@kube
kube changed the base branch from main to codex/fe-1713-petrinaut-document-regressions September 14, 2026 15:51
@kube
kube added this pull request to stack #9717 September 14, 2026 15:51
@kube
kube force-pushed the codex/fe-1665-snapshot-diagnostics branch from aefac2a to 7847480 Compare September 14, 2026 16:00
@github-actions github-actions Bot added the area/tests New or updated tests label Sep 14, 2026
Base automatically changed from codex/fe-1713-petrinaut-document-regressions to main September 14, 2026 16:13
@CiaranMn
CiaranMn force-pushed the codex/fe-1665-snapshot-diagnostics branch from 7847480 to 7aa3a5d Compare September 14, 2026 16:13
Comment thread apps/brunch-agent/test/compiler-feedback.integration.ts Outdated

@lunelson lunelson left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved by Lu Nelson following an agent-assisted review of this PR against its base and FE-1665. No actionable introduced defects found in snapshot request correlation, disposal, or diagnostic wording. The current wording explicitly reports that no errors or warnings were found in net function code.

Verification on the integrated four-PR stack: core package build passed; 231 targeted tests across 11 files passed. This approval does not claim fresh live-browser verification.

@kube
kube requested a review from CiaranMn September 14, 2026 21:08
@kube
kube added this pull request to the merge queue Sep 15, 2026
Merged via the queue into main with commit 466cc6b Sep 15, 2026
79 checks passed
@kube
kube deleted the codex/fe-1665-snapshot-diagnostics branch September 15, 2026 01:18
@hash-release hash-release Bot mentioned this pull request Sep 15, 2026

This branch was successfully deployed

3 active deployments
Preview – hash — 4a068ce3 Deployed Sep 14, 2026 by vercel[bot]
Preview – petrinaut-docs — 4a068ce3 Deployed Sep 14, 2026 by vercel[bot]
Preview – petrinaut — 4a068ce3 Deployed Sep 14, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/apps area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) area/tests New or updated tests type/eng > backend Owned by the @backend team type/eng > frontend Owned by the @frontend team

Development

Successfully merging this pull request may close these issues.

4 participants