feat(specs,fixtures,consume): Support Payload-Building tests #1175

marioevz · 2025-02-04T23:37:15Z

🗒️ Description

Adds support for writing tests based on the following discussion: #1143

~~Requires #1210~~ - Merged

This is in the experimental phase, and a single EIP-7702 test has been included, which consists of a test that sends a single and valid type-4 transaction to the clients and then requests a payload, and the built payload must contain the transaction.

🔗 Related Issues

✅ Checklist

All: Set appropriate labels for the changes.
All: Considered squashing commits to improve commit history.
All: Added an entry to CHANGELOG.md.
All: Considered updating the online docs in the ./docs/ directory.
Tests: All converted JSON/YML tests from ethereum/tests have been added to converted-ethereum-tests.txt.
Tests: A PR with removal of converted JSON/YML tests from ethereum/tests have been opened.
Tests: Included the type and version of evm t8n tool used to locally execute test cases: e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.

fix(consume/engine): Assert no valid transactions left in mempool

spencer-tb

Really cool PR! Gave this a run today and it works well :)

Most of my comments are tiny tweaks with minimal changes to the logic! I'd be happy to get this in soon for more Prague coverage and then follow through on some wider changes in the long term to reap its benefits across all (valid) tests!

A part of me thinks that it could be better to move this within its own consume command, like consume payload, or creating sub-commands:

consume engine blocks (existing)
consume engine txs

Although I'd like to simmer on the idea a bit more. And preferably it would be another PR.

spencer-tb · 2025-02-21T13:59:18Z

src/ethereum_test_fixtures/payload_building.py

+    If an `invalidates` list is specified, when the transaction is included in a
+    block, none of the transactions in the `invalidates` list should be included afterwards
+    (same block or subsequent block).


How do you find invalidated_txs?

Suggested change

If an `invalidates` list is specified, when the transaction is included in a

block, none of the transactions in the `invalidates` list should be included afterwards

(same block or subsequent block).

If an `invalidated_txs` set is specified, when the primary transaction is included in a

block, none of the transactions in `invalidated_txs` should be included in that block

or any subsequent block.

spencer-tb · 2025-02-21T14:00:03Z

src/ethereum_test_fixtures/payload_building.py

+    """
+    Error that should be returned by the client when sending the transaction.
+    """
+    invalidates: Set[Hash] = Field(default_factory=set)


Similar to above.

Suggested change

invalidates: Set[Hash] = Field(default_factory=set)

invalidated_txs: Set[Hash] = Field(default_factory=set)

spencer-tb · 2025-02-21T14:06:25Z

src/ethereum_test_fixtures/payload_building.py

+
+    step_type: Literal["payload_build"] = "payload_build"
+
+    id: int


Suggested change

id: int

payload_id: int

spencer-tb · 2025-02-21T14:06:37Z

src/ethereum_test_fixtures/payload_building.py

+    Monotonically increasing integer that identifies the payload, with zero being
+    the genesis block.
+    """
+    parent_id: int


Suggested change

parent_id: int

parent_payload_id: int

spencer-tb · 2025-02-21T14:20:46Z

src/ethereum_test_specs/payload_building.py

+    The allocations are expected to be compounded with the expected allocation of previously
+    and future included transactions.


Suggested change

The allocations are expected to be compounded with the expected allocation of previously

and future included transactions.

The allocations are expected to be compounded with the expected allocation of previous

and future transactions included in a block.

spencer-tb · 2025-02-21T14:34:26Z

src/ethereum_test_specs/payload_building.py

+                raise Exception(f"Unsupported step type: {step}")
+
+        return PayloadBuildingFixture(
+            fork=network_info,


Should we pass in fork: Fork here instead of network info? And then use that directly to get the engine versions within test_via_engine.py, i.e fixture.fork.engine_..._version? We could remove the 3 engine_ fields from PayloadBuildingFixture by applying the latter?

spencer-tb · 2025-02-21T14:35:53Z

src/ethereum_test_specs/payload_building.py

+    def execute(
+        self,
+        *,
+        fork: Fork,
+        execute_format: ExecuteFormat,
+        eips: Optional[List[int]] = None,
+    ) -> BaseExecute:
+        """Generate the list of test fixtures."""
+        raise Exception(f"Unsupported execute format: {execute_format}")


It looks like we could remove this as its not abstract in BaseTest :)

spencer-tb · 2025-02-21T14:46:48Z

src/pytest_plugins/consume/hive_simulators/engine/test_via_engine.py

@@ -5,9 +5,23 @@
 Each `engine_newPayloadVX` is verified against the appropriate VALID/INVALID responses.


We should update docstring. It can probably be reduced to specify the different types of consume engine tests

spencer-tb · 2025-02-21T14:59:30Z

src/pytest_plugins/consume/hive_simulators/engine/test_via_engine.py

@@ -91,3 +105,199 @@ def test_blockchain_via_engine(
                        assert (
                            forkchoice_response.payload_status.status == PayloadStatusEnum.VALID
                        ), f"unexpected status: {forkchoice_response}"
+
+
+def check_post(expected_post: Alloc, eth_rpc: EthRPC):


Maybe check_post_from_rpc?

I'm happy with this here, although we could also use this as an oppurtunity to create a src/ethereum_test_rpc/helpers.py file and move it there with more granular methods like check_balance, check_code, etc - wrapping them all within the check_post function. I'm imagining a future where we may use these elsewhere

spencer-tb · 2025-02-21T15:28:35Z

tests/prague/eip7702_set_code_tx/test_payload_building.py

+        steps=[
+            *txs,
+            Payload(),  # Should include some of the transactions
+            Payload(),  # Empty payload


Can we automate any of the Payload steps in the framework? I.e always having a second empty payload for tests with invalidated txs?

marioevz force-pushed the payload-building-spec branch 3 times, most recently from 2969ba8 to 8ff396f Compare February 12, 2025 22:09

marioevz changed the base branch from main to fixtures-consume-refactor February 12, 2025 22:09

marioevz mentioned this pull request Feb 12, 2025

refactor(fixtures,consume): Make BaseFixture able to parse any format #1210

Merged

8 tasks

marioevz force-pushed the fixtures-consume-refactor branch from 1c91b20 to 6672261 Compare February 14, 2025 16:54

Base automatically changed from fixtures-consume-refactor to main February 14, 2025 17:01

marioevz added 3 commits February 20, 2025 15:49

new(fixtures,spec): Payload building test spec and fixture

1212a48

new(consume): Payload building in consume engine

d48e775

fix(consume/engine): Assert no valid transactions left in mempool

new(tests): EIP-7702: Payload building tests

a0de4aa

marioevz force-pushed the payload-building-spec branch from e061a5d to a0de4aa Compare February 20, 2025 15:49

marioevz requested a review from danceratopz February 20, 2025 15:50

marioevz marked this pull request as ready for review February 20, 2025 15:50

spencer-tb reviewed Feb 21, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(specs,fixtures,consume): Support Payload-Building tests #1175

feat(specs,fixtures,consume): Support Payload-Building tests #1175

marioevz commented Feb 4, 2025 •

edited

Loading

spencer-tb left a comment •

edited

Loading

spencer-tb Feb 21, 2025

spencer-tb Feb 21, 2025

spencer-tb Feb 21, 2025

spencer-tb Feb 21, 2025

spencer-tb Feb 21, 2025

spencer-tb Feb 21, 2025

spencer-tb Feb 21, 2025

spencer-tb Feb 21, 2025

spencer-tb Feb 21, 2025

spencer-tb Feb 21, 2025

	invalidates: Set[Hash] = Field(default_factory=set)
	invalidated_txs: Set[Hash] = Field(default_factory=set)


		step_type: Literal["payload_build"] = "payload_build"

		id: int

		The allocations are expected to be compounded with the expected allocation of previously
		and future included transactions.

		@@ -5,9 +5,23 @@
		Each `engine_newPayloadVX` is verified against the appropriate VALID/INVALID responses.

feat(specs,fixtures,consume): Support Payload-Building tests #1175

Are you sure you want to change the base?

feat(specs,fixtures,consume): Support Payload-Building tests #1175

Conversation

marioevz commented Feb 4, 2025 • edited Loading

🗒️ Description

🔗 Related Issues

✅ Checklist

spencer-tb left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marioevz commented Feb 4, 2025 •

edited

Loading

spencer-tb left a comment •

edited

Loading