test(yara): write the non-ASCII malformed-rule fixture as UTF-8 - #504
Open
kevin9327 wants to merge 1 commit into
Open
test(yara): write the non-ASCII malformed-rule fixture as UTF-8#504kevin9327 wants to merge 1 commit into
kevin9327 wants to merge 1 commit into
Conversation
The malformed encoded-rule case is parametrized with a non-ASCII payload but writes it with Path.write_text and no encoding, so the file is encoded with the locale codec. Where that codec cannot represent the character the case raises UnicodeEncodeError during setup and never reaches the assertion that a malformed extra rule leaves the builtin rules loadable. _read_rule_source decodes rule bytes as UTF-8, so writing the fixture as UTF-8 also matches how the analyzer reads it. Signed-off-by: kevin9327 <kevin9327@users.noreply.github.com>
rng1995
approved these changes
Sep 12, 2026
rng1995
left a comment
Collaborator
There was a problem hiding this comment.
[SkillSpector Review]
Reviewed head 43a88ed955ee3fce022c4c020e5dcd940b16679f — APPROVE.
Writing the non-ASCII malformed-rule fixture explicitly as UTF-8 makes the test reach the intended invalid-base64 assertion on non-UTF-8 Windows locales and matches the production decoder. I found no required changes.
Required checks pass, but GitHub currently reports mergeStateStatus=BEHIND; update against current main and re-run required checks before merging.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
TestHelpers::test_malformed_extra_encoded_rule_does_not_block_builtin_rulesis parametrizedwith an ASCII payload and a non-ASCII one, then writes the fixture with no encoding:
Path.write_textwith noencodinguseslocale.getpreferredencoding(False). That is UTF-8on the CI image, but it is the ANSI code page on Windows and on any non-UTF-8 locale, and many
of those code pages cannot represent
é. The second parameter then raisesUnicodeEncodeErrorwhile writing the fixture, so the case never reaches its assertion — thatone malformed extra rule file does not stop the builtin rules from loading and matching. The
ASCII parameter passes, so the regression signal is silently halved wherever the locale is not
UTF-8.
The analyzer itself is explicit about the encoding it expects:
so the fixture should be written as UTF-8 rather than as whatever the ambient locale happens to
be.
Fix
Pass
encoding="utf-8"on that one write. The payload stays non-ASCII and stays invalidbase64, so what the case tests is unchanged; it just becomes byte-identical on every locale and
matches how
_read_rule_sourcereads it. Every otherwrite_textin this module writes pureASCII and is unaffected either way, so none of them are touched.
Reproduction
Windows 11, Python 3.12.10,
locale.getpreferredencoding(False)iscp949. Against unmodifiedmain(704bc95):cp949is this machine's code page; any ANSI code page withoutéreproduces it. With thischange, same command:
Both parameters now execute the assertion.
What must still hold
tests/nodes/analyzers/test_static_yara.pyin full:ruff check tests/—All checks passed!ruff format --check tests/—126 files already formattedDiff is
1 file changed, 1 insertion(+), 1 deletion(-).