Skip to content

Move the exception handling logic from K to kevm-pyk#2816

Open
anvacaru wants to merge 11 commits intomasterfrom
driver-refactor-1
Open

Move the exception handling logic from K to kevm-pyk#2816
anvacaru wants to merge 11 commits intomasterfrom
driver-refactor-1

Conversation

@anvacaru
Copy link
Copy Markdown
Contributor

@anvacaru anvacaru commented Dec 4, 2025

Refactoring the logic that handles the exceptions in conformance tests and moving it from driver.md to the kevm-pyk test harness.

changes in driver.md:

  • removing the exception production from EthereumCommand sort and its associated rules from driver.md.
  • removing unused status production from driver.md
  • removing a typo in #allPostKeys set

changes to the GST iterator in kevm-pyk/src/kevm_pyk/interpreter.py:

  • added a _resolve_exception function that reads the expectException and hasBigInt fields from the GST test json.

  • the iterator now returns if the test is expected to fail.

  • changes in kevm-pyk/src/tests/utils.py:

  • optimizing the _test function:

    • removing some redundant if-statements
    • computing the list of tests in a GST file ahead of the execution loop.
  • adding new functions to fetch the cell value.

  • updating the _assert_exit_code_zero function to check if the test should throw an exception when the interpreter returns a non-zero exit code.

Removing the expectException from a test that is supposed to fail is caught.

INFO     tests.utils:utils.py:119 Running test: /home/anvacaru/dev/evm-semantics/master/tests/ethereum-tests/BlockchainTests/GeneralStateTests/Pyspecs/cancun/eip4844_blobs/insufficient_balance_blob_tx.json - src/GeneralStateTestsFiller/Pyspecs/cancun/eip4844_blobs/test_blob_txs.py::test_insufficient_balance_blob_tx[fork_Cancun-blockchain_test--exact_balance_minus_1-tx_max_fee_per_blob_gas_1-no_calldata-tx_value_0-tx_max_priority_fee_per_gas_0-tx_max_fee_per_gas_14-access_list]
INFO     pyk.utils:utils.py:556 [PID=739809][exec] kore-print /dev/stdin --definition /home/anvacaru/.cache/kdist-8d0e547/evm-semantics/llvm --output pretty
INFO     pyk.utils:utils.py:603 [PID=739809][stdo] <kevm>
INFO     pyk.utils:utils.py:603 [PID=739809][stdo]   <k>
INFO     pyk.utils:utils.py:603 [PID=739809][stdo]     check "blockHeader" : { "blobGasUsed" : 131072 , .JSONs } ~> check "blockHeader" : { "bloom" : "0x0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000...
INFO     pyk.utils:utils.py:603 [PID=739809][stdo]   </k>
INFO     pyk.utils:utils.py:603 [PID=739809][stdo]   <exit-code>

@anvacaru anvacaru marked this pull request as ready for review December 12, 2025 11:53
@anvacaru anvacaru requested review from juliankuners and palinatolmach and removed request for juliankuners December 15, 2025 07:11
Copy link
Copy Markdown
Contributor

@juliankuners juliankuners left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see comments. Also, I did not deep dive into the testing mechanisms, so a second review might be nice.

Comment on lines 605 to 610
kore_pattern_list = [
(name, kore)
for (name, kore) in iterate_gst(
(name, kore, exception_metadata)
for (name, kore, exception_metadata) in iterate_gst(
json_read, options.mode, options.chainid, options.usegas, schedule=options.schedule
)
]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't this instead be re-written as this?

Suggested change
kore_pattern_list = [
(name, kore)
for (name, kore) in iterate_gst(
(name, kore, exception_metadata)
for (name, kore, exception_metadata) in iterate_gst(
json_read, options.mode, options.chainid, options.usegas, schedule=options.schedule
)
]
kore_pattern_list = list(
iterate_gst(
json_read, options.mode, options.chainid, options.usegas, schedule=options.schedule
)
)

Comment on lines +640 to +645
except CalledProcessError:
if exception_expected:
_LOGGER.info(f'Test {name} failed as expected')
continue
else:
raise
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If an exception is expected, but no CalledProcessError is thrown, this check will be skipped. If this needs to be enforced, it should, e.g., check whether an exception happened outside of the except block.

Comment on lines 659 to 664
kore_pattern_list = [
(name, kore)
for (name, kore) in iterate_gst(
(name, kore, exception_metadata)
for (name, kore, exception_metadata) in iterate_gst(
json_read, options.mode, options.chainid, options.usegas, schedule=options.schedule
)
]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same list comprehension as above can be applied here as well

Comment on lines +64 to +67
ethereum_cell = kevm_cell.args[5] # type: ignore[attr-defined]
evm_cell = ethereum_cell.args[0] # type: ignore[attr-defined]
status_code_cell = evm_cell.args[1] # type: ignore[attr-defined]
status_code = status_code_cell.args[0] # type: ignore[attr-defined]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These magic number indices look very susceptible to semantic changes to me. Can you replace them with a more reproducible mechanism like labels or something?

Comment on lines +44 to +46
elif exception_expected:
_assert_exit_code_exception(kevm_cell)
return
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function name _assert_exit_code_zero is not appropriate anymore when exception_expected = True.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants