Skip to content

fix(core): support fallback keys for ignore-delete option - #462

Merged
lxy-9602 merged 2 commits into
alibaba:mainfrom
SteNicholas:PAIMON-461
Jul 30, 2026
Merged

lxy-9602 merged 2 commits into
alibaba:mainfrom
SteNicholas:PAIMON-461

Conversation

@SteNicholas

@SteNicholas SteNicholas commented Jul 29, 2026 •

Copy link
Copy Markdown
Contributor

Purpose

Linked issue: close #461

The ignore-delete semantics can also be configured through the merge-engine-specific keys
first-row.ignore-delete, deduplicate.ignore-delete and partial-update.ignore-delete, which are
declared as fallback keys of ignore-delete. CoreOptions only parsed ignore-delete and kept the
default false otherwise, so a table configured with first-row.ignore-delete=true was read as
ignore-delete=false. The first-row merge engine then rejected retract records with:

By default, First row merge engine can not accept DELETE/UPDATE_BEFORE records.
You can config 'first-row.ignore-delete' to ignore the DELETE/UPDATE_BEFORE records.

This change parses the fallback keys in fixed order (first-row.ignore-delete,
deduplicate.ignore-delete, partial-update.ignore-delete) when ignore-delete itself is absent,
keeping the primary key's precedence when both are present. The fallback key constants follow the
existing Options::FALLBACK_BLOB_DESCRIPTOR_FIELD pattern.

Tests

  • CoreOptionsTest.TestIgnoreDeleteFallbackKeys — each fallback key resolves, the primary key takes
    precedence over a fallback key, fallback keys are checked in fixed order, and an invalid value is
    rejected.
  • PrimaryKeyTableUtilsTest.TestCreateFirstRowMergeFunctionWithIgnoreDelete — a first-row merge
    function built from first-row.ignore-delete ignores a DELETE record and keeps the first row,
    identically to one built from ignore-delete; without either option retract records are still
    rejected.

API and Format

include/paimon/defs.h gains three new option-key constants
(Options::FALLBACK_FIRST_ROW_IGNORE_DELETE, Options::FALLBACK_DEDUPLICATE_IGNORE_DELETE,
Options::FALLBACK_PARTIAL_UPDATE_IGNORE_DELETE). This is additive; no existing symbol changes and
the storage format and protocol are unaffected.

Documentation

No new feature, so no documentation change. The new constants carry doc comments in
include/paimon/defs.h following the existing fallback-key convention.

Generative AI tooling

Generated-by: Claude Code (Claude Fable 5)

The ignore-delete semantics can also be configured through the
merge-engine-specific keys first-row.ignore-delete,
deduplicate.ignore-delete and partial-update.ignore-delete, but
CoreOptions only parsed ignore-delete and kept the default false, so
first-row compaction rejected DELETE/UPDATE_BEFORE records from tables
configured with first-row.ignore-delete=true.

Parse the fallback keys in fixed order when ignore-delete itself is
absent, keeping the primary key's precedence.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 29, 2026 09:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Adds support for parsing merge-engine-specific fallback keys for the ignore-delete option so that tables configured with keys like first-row.ignore-delete behave the same as ignore-delete, matching Java behavior and fixing #461.

Changes:

  • Update CoreOptions parsing to resolve ignore-delete from fallback keys in a fixed precedence order when the primary key is absent.
  • Add new Options::* constants for the fallback keys in public headers and common defs.
  • Add unit tests covering fallback resolution and first-row merge behavior with ignore-delete enabled via fallback.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/paimon/core/utils/primary_key_table_utils_test.cpp Adds integration-style test ensuring first-row merge respects ignore-delete when set via the fallback key.
src/paimon/core/core_options_test.cpp Adds unit tests verifying fallback key precedence/order and invalid value rejection.
src/paimon/core/core_options.cpp Implements fallback-key parsing logic for ignore-delete when the primary key is absent.
src/paimon/common/defs.cpp Defines string values for the new fallback option-key constants.
include/paimon/defs.h Exposes new fallback option-key constants with doc comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread include/paimon/defs.h Resolved
Comment thread src/paimon/core/core_options_test.cpp Resolved

@lxy-9602 lxy-9602 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@lxy-9602
lxy-9602 merged commit 575a3d8 into alibaba:main Jul 30, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] CoreOptions does not resolve Java fallback keys for ignore-delete (e.g. first-row.ignore-delete)

3 participants