format, pointers, bugc: specify and implement two-sorted (integer/bytes) expression evaluation - #286
Open
gnidan wants to merge 9 commits into
Open
format, pointers, bugc: specify and implement two-sorted (integer/bytes) expression evaluation#286gnidan wants to merge 9 commits into
gnidan wants to merge 9 commits into
Conversation
DRAFT for writer prose pass before the PR opens. Codifies the two-sorted semantics of pointer expressions: values are either unbounded non-negative integers (no width) or bytes (with a definite width). Arithmetic, JSON-number and odd-nibble-hex literals, $wordsize, and lookups are integers; even-nibble hex, $read, and $sizedN/$wordsized are bytes. An integer is accepted wherever an integer is expected (a bytes value is read as its big-endian integer value), but $concat and $keccak256 operands must be width-bearing bytes — a bare integer there is an error and must be resized first. Fixes the keccak256 example's bare-integer operand to be word-sized, matching what the mapping/string pointer examples already do.
Contributor
|
The evaluator used a single width-carrying Data representation for
every expression, so a JSON number, $wordsize, or an arithmetic result
had an incidental width that leaked into $concat and $keccak256: for
instance `{ $keccak256: [{ $wordsized: k }, 0] }` hashed 32 bytes
rather than 64.
evaluate() now returns a Value of one of two sorts, following the
expression schema:
- integer (bigint, no width): JSON numbers, $wordsize, lookups,
arithmetic results, and odd-digit hex literals
- bytes (Data, definite width): even-digit hex literals, $read, and
the resize forms
Arithmetic takes its operands as integers (bytes coerce big-endian)
and yields an integer. $concat and $keccak256 require bytes operands
and throw, naming the offending operand and suggesting $wordsized or
$sizedN, when given a bare integer. Resize is the only integer-to-bytes
bridge. Variables carry the sort of their defining expression; list
counts, conditionals, and region slot/offset/length coerce to
integers. Cursor.Region is unchanged: its Data-valued properties now
encode integers minimally, with bytes keeping their width.
The previously skipped keccak256 test is rewritten against these
rules, along with tests for each sort rule, both coercion directions,
and the bare-integer errors.
Claude-Session: https://claude.ai/code/session_01RJFyifZxcSXZchLFNTNPuT
The expression schema's root-level `$keccak256` example still hashed a bare JSON number and a bare lookup, which the two-sorted rules reject. Wrap both operands in `$wordsized`, as the Keccak256 example already does. pointers: add a test that evaluates every `$keccak256` / `$concat` expression found in the pointer schemas' examples against stubbed regions and variables, so an example whose operand lacks a width fails the suite rather than silently contradicting the spec. Claude-Session: https://claude.ai/code/session_01RJFyifZxcSXZchLFNTNPuT
The operands of $keccak256 must be width-bearing bytes. The mapping and dynamic-array slot helpers in irgen/debug/pointers.ts passed the slot operand bare (a number literal or $sum), which is invalid under the two-sorted expression rules and hashes the wrong number of bytes: the EVM hashes key‖slot as two 32-byte words, and a bare literal has no width. Wrap the slot operand in $wordsized; a nested $keccak256 result is already 32 bytes wide and is passed through unwrapped. Note that program output is not affected: variables.ts already wordsizes dynamic-array base slots, and mapping variables are emitted as their base slot only (translateComputeSlotChain and the access helpers have no production callers yet). Claude-Session: https://claude.ai/code/session_01KzkzZRucD49j4fRHT3Ndan
The guide extracts its code listings from src/evaluate.ts by symbol name, and the two-sorted rewrite collapsed the five per-operation arithmetic functions into one evaluateArithmetic, so the page failed to build. Replace the five arithmetic listings with one, introduce the integer/bytes sorts up front with listings of the exported Value type and namespace, and add listings of the evaluateInteger and evaluateBytesOperands helpers where coercion and width enforcement come up. Surrounding prose now describes integer-sort arithmetic, literal sorting by hex digit count, and the bytes requirement on $keccak256 / $concat operands. Claude-Session: https://claude.ai/code/session_01RJFyifZxcSXZchLFNTNPuT
The expression spec page still said odd-digit hex literals are fine and that values are left-padded to the width the context needs, which contradicts the two-sorted model: even-digit hex is bytes of the width written, odd-digit hex is an integer, and there is no implicit widening. Rewrite the literals prose, add an integers-and-bytes section ahead of the per-form sections, and have each form's prose name its result sort. The docs guide gains a compact integers-and-bytes section and its keccak256 examples now word-size keys and slots before hashing. Claude-Session: https://claude.ai/code/session_014otYPQPP9pvQabmY58Fyom
gnidan
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pointer expressions had an unstated gap: the width of a value was undefined for numeric literals, arithmetic results, and odd-digit hex, yet
$concatand$keccak256depend on operand widths — so a compiler and a debugger could legitimately disagree on the bytes produced. This specifies the evaluation model and brings the reference implementation, the compiler, and the docs along with it.Semantics
Expressions evaluate to a value of one of two sorts:
The forms are sorted accordingly. Integers: a JSON-number literal,
$wordsize, a variable or lookup, arithmetic ($sum/$difference/$product/$quotient/$remainder), and an odd-digit hex literal (which has no whole-byte width). Bytes: an even-digit hex literal (width = bytes written),$read(width = region length),$sizedN/$wordsized(width =N/ word size),$keccak256(width 32), and$concat(width = sum of operand widths).Coercion runs one way. Where an integer is expected — arithmetic operands, a list
count, a segmentslot/offset/length— a bytes value is read as the non-negative integer its bytes encode (big-endian). Where bytes are expected — the operands of$concatand$keccak256— the operand must be width-bearing; a bare integer there is an error and must be given a width with$sizedN/$wordsizedfirst. The resize forms are the only bridge from an integer to bytes; there is no implicit widening.This codifies what the mapping/string pointer examples already do (they word-size a key and slot before hashing). Two schema examples that hashed bare integers are corrected: the
keccak256definition's example and the expression schema's root-level example.Reference implementation
@ethdebug/pointersnow evaluates to a taggedValue— integer or bytes — with the sort rules above, coerces where an integer is expected, and throws on a bare-integer operand to$concator$keccak256with a message naming the operand and suggesting$wordsized/$sizedN. Variables carry the sort of their defining expression. This is package-internal:evaluateis not exported from the package index, and the publicCursor.Regionshape is unchanged, so no consumers needed edits. The previous single-width implementation gave a JSON-number literal minimal width (literal0was zero bytes), so a$keccak256over a bare slot silently hashed the wrong preimage; that can no longer happen.The evaluator's previously skipped keccak test is reinstated, and the evaluator test file now covers every sort rule, both coercion directions, and the error cases. A new test evaluates every
$keccak256/$concatoccurrence in the pointer schema examples so an example without operand widths fails the suite.Compiler
bugc's slot-computation helpers emitted
$keccak256with a bare slot operand. They now word-size it, with a nested$keccak256passed through since it is already 32-byte bytes. These helpers have no production callers today — bugc emits a mapping variable as its base slot — so emitted debug info was already valid; the helpers and their pinned tests are corrected so that any future use is too.Docs
The expression spec page no longer says values are "assumed to be left-padded to the bytes width appropriate for the context"; it states the two sorts, the even/odd hex rule, and the width-bearing requirement. The pointer expressions guide gains an "Integers and bytes" section and word-sizes the operands in its hashing examples. The evaluation implementation guide follows the new evaluator's shape.
Note: the integer/bytes sorting discipline is meant to carry over into a future extraction of the expression language into a shared facility.