Skip to content

[Bug] Reject ES-incompatible field comparisons under elasticsearch_syntax - #86

Merged
Mikaayenson merged 8 commits into
masterfrom
bug/es-syntax-reject-field-comparisons
Aug 7, 2026
Merged

[Bug] Reject ES-incompatible field comparisons under elasticsearch_syntax#86
Mikaayenson merged 8 commits into
masterfrom
bug/es-syntax-reject-field-comparisons

Conversation

@Mikaayenson

@Mikaayenson Mikaayenson commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Under elasticsearch_syntax / reject_field_comparisons, reject comparisons where neither side folds to a constant (field-to-field), including through functions and arithmetic (e.g. a == b, length(a) == length(b) - 1).
  • Comparing a field expression to a constant remains valid (e.g. length(a) - 1 == 5), matching Elasticsearch stack behavior.
  • Not applied under elastic_endpoint_syntax (Endpoint allows field-to-field comparisons). Sequence aliases (as p0) remain allowed when aliases are enabled.
  • Motivated by detection-rules shipping an invalid query that Python EQL accepted but ES rejected (Comparisons against fields are not currently supported) (detection-rules#6611).

Test plan

  • flake8 eql tests
  • Full pytest tests/ locally (616 passed)
  • Edge-case matrix for ES allow/forbid + endpoint allow (25/25)
  • Scanned detection-rules 8.19/9.3/9.4/9.5: only NTLM field-vs-field fails (expected); fixed branch parses clean
  • Scanned endpoint-rules main: 1936/1936 OK

…ntax

Match Elasticsearch EQL by rejecting field-to-field comparisons and field
arithmetic in ==/!=/</> when elasticsearch_syntax is enabled, so SIEM CI
catches invalid queries (e.g. length(a) == length(b) - 1) before ship.
Comment thread CHANGELOG.md Outdated
Extract validation into a helper to reduce comparison() complexity
and wrap a long assertRaises line.
Cite ES EQL docs and the observed runtime rejection instead.
@Mikaayenson Mikaayenson self-assigned this Aug 6, 2026
@Mikaayenson Mikaayenson added bug Something isn't working elasticsearch Relates to the EQL in Elasticsearch effort labels Aug 6, 2026
Endpoint EQL allows field-to-field comparisons; gate the check via
reject_field_comparisons so elastic_endpoint_syntax is unaffected.
Comment thread eql/parser.py Outdated
Match ES: comparisons need one constant side. length(a)-1 == 5 is valid;
length(a) == length(b)-1 is not. Thanks Eric for the patch.
Comment thread eql/parser.py Outdated
Comment thread tests/test_parser.py

@eric-forte-elastic eric-forte-elastic left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two suggestions, otherwise I think this looks good 👍

Pass offender/op as _error kwargs so brace literals raise EqlSemanticError,
and cover alias exemptions under reject_field_comparisons.
Comment thread CHANGELOG.md Outdated
@Mikaayenson
Mikaayenson merged commit 16f9556 into master Aug 7, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working elasticsearch Relates to the EQL in Elasticsearch effort

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants