Skip to content

core/rawdb: freeze empty receipts when the header commits to none - #2474

Open
kamuikatsurgi wants to merge 2 commits into
developfrom
krishang/POS-3295-freezer-empty-receipts
Open

kamuikatsurgi wants to merge 2 commits into
developfrom
krishang/POS-3295-freezer-empty-receipts

Conversation

@kamuikatsurgi

@kamuikatsurgi kamuikatsurgi commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

Summary

Fixes POS-3295 (block receipts missing, can't freeze block N).

Between v2.3.2 and v2.8.0, splitReceiptsAndDeriveFields returned nil for blocks without (non state-sync) receipts. This mostly hit pre-Madhugiri sprint-end blocks whose only receipt was the state-sync one. InsertReceiptChain then stored a zero-length value under the block's receipts key. 839b3c6 fixed the writer, but databases that already have these entries (and snapshots taken from them) stay stuck:

  • freezeRange treats the empty value as missing and never gets past that block.
  • Every minute it re-appends up to 30k blocks to the ancient store and truncates them again. That causes periodic sync stalls, and the key-value store keeps growing.

The freezer now freezes rlp.EmptyList when the receipts value is empty and the block header commits to an empty receipt root. This is the same rule the eth handlers already use when serving receipts for such blocks. If the header commits to non-empty receipts, the freezer still errors as before.

Executed tests

  • New TestChainFreezerEmptyReceipts (fails without the fix); go test -race ./core/rawdb/...
  • golangci-lint clean on core/rawdb; diffguard passes with 100% mutation kill rate
  • Kurtosis devnet (2 validators, 2 RPC nodes). The images were built from develop and from this branch, with FullImmutabilityThreshold lowered to 256 for the test only so the freezer runs within minutes.
    • Setup: stopped both RPC nodes and replaced one block's receipts entry with an empty value, the same on-disk shape older versions left behind.
      • Node A, block 85: no txs, empty receipt root.
      • Node B, block 80: one real tx (control).
    • On develop: both nodes got stuck as reported in the ticket. Every minute they logged block receipts missing, can't freeze block 85 / ... block 80, and ancients stayed at 49 / 54 while head grew past 500.
    • On this branch, same data volumes:
      • Node A froze past block 85 in one cycle: ancients went from 49 to 528 and the error stopped.
      • After freezing, eth_getBlockReceipts(85) returns [], and block 80's tx receipt is served from the ancient store with status 1.
      • Node B (non-empty receipt root) still fails on block 80, as intended.

Rollout notes

Not consensus-affecting and backwards-compatible. Affected nodes start freezing again after the upgrade, with no resync needed.

Older versions could store a zero-length receipts value for blocks without
(non state-sync) receipts during sync. The chain freezer treated that as
missing data and failed on that block forever, re-appending and rolling back
up to a full batch every minute.

Freeze the canonical empty list instead when the block header's receipt root
is the empty root, matching how the eth handlers already serve such blocks.
A block that commits to non-empty receipts still fails as before.
@kamuikatsurgi
kamuikatsurgi marked this pull request as ready for review October 6, 2026 04:47
Copilot AI balanced review requested due to automatic review settings October 6, 2026 04:47

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

Tip: disable this comment in your organization's Code Review settings.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The focused fix safely validates the header commitment and includes coverage for both success and rejection paths.

Review effort: Balanced
Findings: None

What changed in this PR

Fixes freezer stalls caused by legacy empty receipt values when headers commit to an empty receipt root.

Changes:

  • Freezes rlp.EmptyList for safely verifiable empty receipts.
  • Preserves errors for non-empty receipt roots.
  • Adds regression and control tests.
File Description
core/​rawdb/​chain_freezer.go Handles legacy empty receipt values during freezing.
core/​rawdb/​freezer_test.go Tests empty and non-empty receipt-root behavior.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot AI balanced review requested due to automatic review settings October 6, 2026 04:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The focused fix preserves integrity checks and includes appropriate regression coverage.

Review effort: Balanced
Findings: None

@codecov

codecov Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 55.66%. Comparing base (4b95aff) to head (7e4969b).
⚠️ Report is 155 commits behind head on develop.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #2474      +/-   ##
===========================================
+ Coverage    55.62%   55.66%   +0.04%     
===========================================
  Files          918      918              
  Lines       167160   167169       +9     
===========================================
+ Hits         92978    93050      +72     
+ Misses       68714    68640      -74     
- Partials      5468     5479      +11     
Files with missing lines Coverage Δ
core/rawdb/chain_freezer.go 36.77% <100.00%> (+10.19%) ⬆️

... and 32 files with indirect coverage changes

Files with missing lines Coverage Δ
core/rawdb/chain_freezer.go 36.77% <100.00%> (+10.19%) ⬆️

... and 32 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

Copy link
Copy Markdown

This PR is stale because it has been open 21 days with no activity. Remove stale label or comment or this will be closed in 14 days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants