[dip-213] decoupled execution by zekun000 · Pull Request #214 · diem/dip

zekun000 · 2021-11-24T23:22:47Z

for #213

dahliamalkhi

wonderful proposal! see stylistic/clarification comments inline.

dips/dip-213.md

dahliamalkhi · 2021-11-27T09:59:14Z

dips/dip-213.md

+To prevent consensus going too fast and create ever-growing backlog for other stages, back pressure is implemented to stop consensus making progress if the difference between committed block and ordered block is large.
+
+## Reconfiguration
+A reconfiguration transaction is the last transaction of an epoch. Execution recognizes this type of transaction and buffer manager would stop processing any blocks after the reconfiguration block.


Suggested change

A reconfiguration transaction is the last transaction of an epoch. Execution recognizes this type of transaction and buffer manager would stop processing any blocks after the reconfiguration block.

Recall that a reconfiguration transaction must be the last transaction of an epoch. Execution recognizes this type of transaction and BufferManager stops processing blocks succeeding the reconfiguration block.

what about the empty/special block that must follow a reconfiguration block to reach a 2-chain commit?

those empty blocks won't be committed (both before and after), they're just used to facilitate the commit of reconfiguration block.

dahliamalkhi · 2021-11-27T10:00:56Z

dips/dip-213.md

+
+## Reconfiguration
+A reconfiguration transaction is the last transaction of an epoch. Execution recognizes this type of transaction and buffer manager would stop processing any blocks after the reconfiguration block.
+Consensus would stop once the backpressure is triggered, blocks after the reconfiguration block would be discarded and transactions would be retried in next epoch.


not clear. Please explain

dips/dip-213.md

dahliamalkhi · 2021-12-10T00:32:44Z

dips/dip-213.md

+`StateComputer` is the trait that consensus uses for `compute`, `commit` or `sync_to` blocks.
+A simple `ordering_state_computer` is implemented to bypass execution and send blocks to next stage when ordered.
+
+## BufferManager


it's really a block state-machine, but that's of course overloaded.

how about ConsensusStageManager or BFTStageManager or SMRStageManager?

dips/dip-213.md

dahliamalkhi · 2021-12-13T18:51:23Z

dips/dip-213.md

+Recall that a reconfiguration transaction must be the last transaction of an epoch. Before this proposal, consensus-ordering is aware of 
+reconfiguration and leader proposes empty suffix. After the proposal, execution on each node recognizes this type of transaction and BlockStateManager stops processing any blocks succeeding the reconfiguration block.
+Ordered blocks after the reconfiguration block are discarded and transactions inside those blocks would be retried in next epoch.
+Imagine a few block batches [1, 2] [r, 3], [4], [5, 6], r represents a block has reconfiguration transaction. The ledger info is aggregated on block 3 (transactions in 3 are skipped) and 4, 5, 6 are discarded.


what is the purpose of the bracket notation? what do you mean by "block batches" (transactions batched into blocks?)

Suggested change

Imagine a few block batches [1, 2] [r, 3], [4], [5, 6], r represents a block has reconfiguration transaction. The ledger info is aggregated on block 3 (transactions in 3 are skipped) and 4, 5, 6 are discarded.

Imagine a few block batches [1, 2] [r, 3], [4], [5, 6], r represents a block that contains a reconfiguration transaction. The ledger info is aggregated on block 3 (transactions in 3 are skipped) and 4, 5, 6 are discarded.

it's a batches of blocks that certified by a ledger info, not every block gets a ledger info in case of timeout

dahliamalkhi · 2021-12-13T18:52:18Z

dips/dip-213.md

+After the reconfiguration block gets committed, the old epoch instance is dropped, a new epoch is instantiated via a virtual genesis block derived from the last ledger info in previous epoch.
+
+## Upgrade
+To enable the feature in existing blockchain requires coordination via reconfiguration, on-chain consensus config would be updated to a newer version supporting the upgrade protocol.


Suggested change

To enable the feature in existing blockchain requires coordination via reconfiguration, on-chain consensus config would be updated to a newer version supporting the upgrade protocol.

To enable the feature in an existing blockchain requires coordination via reconfiguration, on-chain consensus config would be updated to a newer version supporting the upgrade protocol.

dahliamalkhi · 2021-12-13T18:54:48Z

dips/dip-213.md

+
+## Reconfiguration
+Recall that a reconfiguration transaction must be the last transaction of an epoch. Before this proposal, consensus-ordering is aware of 
+reconfiguration and leader proposes empty suffix. After the proposal, execution on each node recognizes this type of transaction and BlockStateManager stops processing any blocks succeeding the reconfiguration block.


does this mean that the consensus stage continues moving forward, oblivious of the reconfiguration transaction, until either execution catches up with the reconfig tx or back-pressure prevents progress? if yes, perhaps add this clarification to the text?

it moves forward until it hits backpressure and then blocks until next epoch

mimoo

wow this is great, is this implemented already? Are there any downsides of this approach?

mimoo · 2022-03-16T01:33:45Z

dips/dip-213.md

+- `Ordered` block is sent to execution and advanced to `Executed` after receiving execution result.
+- `Executed` block is sent to safety rules and advanced to `Signed` after receiving a signature.
+- Signature on `Signed` block is broadcasted to all validators. At all stages except `Aggregated`, signatures from other validators on the block are collected.
+- A block in either `Executed` or `Signed` stage is advanced to `Aggregated` after receiving a quorum of signatures.


I would suggest "certification of correct execution", as 2f+1 have agreed that this is the correct execution output

zekun000 · 2022-03-16T18:22:55Z

@mimoo yes it's already implemented. one trade-off is that it adds one hop latency due to the extra execution certification

mimoo · 2022-03-16T19:27:13Z

oh right, makes sense! thanks

dahliamalkhi suggested changes Nov 27, 2021

View reviewed changes

zekun000 force-pushed the dip213 branch from 77b6b27 to 6ca6fc3 Compare November 29, 2021 23:20

zekun000 requested a review from dahliamalkhi November 30, 2021 01:55

dahliamalkhi suggested changes Dec 10, 2021

View reviewed changes

zekun000 force-pushed the dip213 branch from 6ca6fc3 to abe0abf Compare December 11, 2021 02:45

dahliamalkhi suggested changes Dec 13, 2021

View reviewed changes

[dip-213] decoupled execution

7dc44ee

zekun000 force-pushed the dip213 branch from abe0abf to 7dc44ee Compare December 13, 2021 19:41

mimoo reviewed Mar 16, 2022

View reviewed changes

	A reconfiguration transaction is the last transaction of an epoch. Execution recognizes this type of transaction and buffer manager would stop processing any blocks after the reconfiguration block.
	Recall that a reconfiguration transaction must be the last transaction of an epoch. Execution recognizes this type of transaction and BufferManager stops processing blocks succeeding the reconfiguration block.

	Imagine a few block batches [1, 2] [r, 3], [4], [5, 6], r represents a block has reconfiguration transaction. The ledger info is aggregated on block 3 (transactions in 3 are skipped) and 4, 5, 6 are discarded.
	Imagine a few block batches [1, 2] [r, 3], [4], [5, 6], r represents a block that contains a reconfiguration transaction. The ledger info is aggregated on block 3 (transactions in 3 are skipped) and 4, 5, 6 are discarded.

	To enable the feature in existing blockchain requires coordination via reconfiguration, on-chain consensus config would be updated to a newer version supporting the upgrade protocol.
	To enable the feature in an existing blockchain requires coordination via reconfiguration, on-chain consensus config would be updated to a newer version supporting the upgrade protocol.

Conversation

zekun000 commented Nov 24, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dahliamalkhi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mimoo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zekun000 commented Mar 16, 2022

Uh oh!

mimoo commented Mar 16, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

zekun000 commented Nov 24, 2021 •

edited

Loading