Skip to content

[CALCITE-7795] SemanticTable FilterRequirement is dropped by certain query shapes - #5275

Open
rubenada wants to merge 2 commits into
apache:mainfrom
rubenada:CALCITE-7795
Open

rubenada wants to merge 2 commits into
apache:mainfrom
rubenada:CALCITE-7795

Conversation

@rubenada

@rubenada rubenada commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Jira Link

CALCITE-7795

Changes Proposed

Correct certain query shapes (set-ops, alias-with-column-list, PIVOT/UNPIVOT/MATCH_RECOGNIZE/table functions, subqueries, INSERT/UPDATE/MERGE/DELETE sources, cross-scope bypass purge) that silently drop or defuse the FilterRequirement.

@sonarqubecloud

Copy link
Copy Markdown

sameIdentifier_.forEach(remnantMustFilterFields::remove);
// Clear the remnant must-filter qualifieds imported through the
// same FROM item (namespace) as the filtered bypass field
if (qualified.namespace != null) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could it be that, within the same subquery, a bypass for one table incorrectly clears the remnant of another?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I understand, could you please provide an example?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I got your point, I tried to addressed this potential issue introducing the notion of "Group" in FilterRequirement.

final SqlValidatorNamespace operandNamespace =
requireNonNull(validator.getNamespace(operand),
() -> "namespace for " + operand);
final FilterRequirement requirement = operandNamespace.getFilterRequirement();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, one might need to consider that cross-bypass fields spanning set operation operands could result in valid queries being rejected.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean something like

// Only operands that still owe a must-filter constrain the setop's bypass set
if (!requirement.filterFields.isEmpty() || !requirement.remnantFilterFields.isEmpty()) {
  bypassFields = bypassFields == null
      ? requirement.bypassFields
      : bypassFields.intersect(requirement.bypassFields);
}

@liuyongvs

liuyongvs commented Oct 15, 2026 via email

Copy link
Copy Markdown
Contributor

@rubenada

Copy link
Copy Markdown
Contributor Author

@xuzifu666 , just a friendly reminder that there are a couple of comments waiting for your feedback here, nothing urgent, please take a look when you have some time

@xuzifu666 xuzifu666 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry!I missed the review notification for this PR earlier; I have no objections to the current changes. @rubenada

@sonarqubecloud

Copy link
Copy Markdown

@rubenada rubenada added LGTM-will-merge-soon Overall PR looks OK. Only minor things left. and removed LGTM-will-merge-soon Overall PR looks OK. Only minor things left. labels Oct 22, 2026
…rRequirement to have a per-table-instance tracking
@rubenada

Copy link
Copy Markdown
Contributor Author

@xuzifu666 I've added some extra code, which I think goes aligned with your original remarks. Could you please take another look at this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants