Dispute distribution implementation #3282

eskimor · 2021-06-17T12:08:24Z

This PR implements the dispute distribution as described in the guide it also integrates dispute distribution with other subsystems.

In particular:

Implement dispute distribution
Lets the RuntimeInfo work with a subsystem sender only, instead of a full SubsystemContext.
Make dispute participation report back on the availability of the candidate
Make dispute coordinator report back on the validity of an import (spam protection)
Some general cleanup/code de-duplication
Make request/response API more expressive

Todos:

Vote recovery will be a second PR, disputes should already be fully functional without it, see paritytech/polkadot-sdk#965

Closes #3163 .
Closes #3399 .

- Cleanup dead disputes - Update sends for new sessions - Retry on errors

node/network/dispute-distribution/src/sender/mod.rs

and use them in statement-distribution.

We only need a `SubsystemSender` not a full `SubsystemContext`.

Make it possible to consume stuff without clones.

rphmeier · 2021-06-20T04:24:29Z

Vote recovery will be a second PR, disputes should already be fully functional without it.

Please file an issue! And yes, agreed. Best to split

eskimor · 2021-06-20T17:10:32Z

Noiice! Thanks for the tip!!

…

On 6/20/21 7:18 AM, Andreas Doerr wrote: ***@***.**** commented on this pull request. ------------------------------------------------------------------------ In node/network/dispute-distribution/src/sender/mod.rs <#3282 (comment)>: > +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Polkadot. If not, see <http://www.gnu.org/licenses/>. + + +use std::collections::HashMap; +use std::collections::HashSet; +use std::collections::hash_map::Entry; + +use futures::Future; +use futures::channel::mpsc; +use futures::channel::oneshot; +use futures::future::RemoteHandle; + +use polkadot_node_network_protocol::IfDisconnected; rust-analyzer has an |Import Granularity| config item. For me, |preserve| works best. This will just follow along with the current structure. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3282 (comment)>, or unsubscribe </~https://github.com/notifications/unsubscribe-auth/AALUZ2K5TCZQR4ZH3QCA35LTTV22NANCNFSM463RCMKA>.

node/network/dispute-distribution/src/receiver/mod.rs

node/subsystem-test-helpers/src/lib.rs

rphmeier · 2021-07-08T03:59:37Z

node/subsystem-test-helpers/src/mock.rs

+
+
+/// Get mock keystore with `Ferdie` key.
+pub fn make_ferdie_keystore() -> SyncCryptoStorePtr {


rphmeier · 2021-07-08T04:00:08Z

node/subsystem-util/src/runtime/mod.rs

 		&mut self,
-		ctx: &mut Context,
+		sender: &mut Sender,


Glad to see this change.

rphmeier · 2021-07-08T04:01:09Z

node/subsystem/src/messages.rs

+		/// This is, we either discarded the votes, just record them because we
+		/// casted our vote already or recovered availability for the candidate
+		/// successfully.
+		pending_confirmation: oneshot::Sender<ImportStatementsResult>


The dispute coordinator accepts all statements on all candidates, even undisputed ones. I implemented this in #3348. So there is some middle ground between "once we've cast our vote or recovered availability".

Not 100% sure what you mean, but I clarified the comment a bit. Hopefully it makes sense - it is quite late over here already.

node/subsystem/src/messages.rs

node/network/dispute-distribution/src/lib.rs

node/subsystem/src/messages.rs

node/network/dispute-distribution/src/lib.rs

node/network/dispute-distribution/src/receiver/error.rs

Seems to be stale. All remarks have been addressed.

node/core/dispute-coordinator/src/lib.rs

node/network/dispute-distribution/src/metrics.rs

node/overseer/src/lib.rs

drahnr

Post merge review - LGTM 👍 a few tiny nits

Incorporating post merge review remarks.

eskimor added 10 commits June 14, 2021 16:27

Dispute protocol.

8c83835

Dispute distribution protocol.

125dd32

Get network requests routed.

2c395e5

WIP: Basic dispute sender logic.

7c53c69

Basic validator determination logic.

0bd8713

WIP: Getting things to typecheck.

608f84c

Slightly larger timeout.

a78fbc9

More typechecking stuff.

53f0c15

Cleanup.

6abf1f3

Finished most of the sending logic.

b541062

github-actions bot added the A3-in_progress Pull request is in progress. No review needed at this stage. label Jun 17, 2021

eskimor added 3 commits June 17, 2021 15:20

Handle active leaves updates

5883c08

- Cleanup dead disputes - Update sends for new sessions - Retry on errors

Pass sessions in already.

4c4e3a7

Startup dispute sending.

bd8bb95

bkchr reviewed Jun 18, 2021

View reviewed changes

node/network/dispute-distribution/src/sender/mod.rs Outdated Show resolved Hide resolved

eskimor added 8 commits June 18, 2021 10:50

Provide incoming decoding facilities

4ebe127

and use them in statement-distribution.

Relaxed runtime util requirements.

95351bd

We only need a `SubsystemSender` not a full `SubsystemContext`.

Better usability of incoming requests.

863b1d9

Make it possible to consume stuff without clones.

Add basic receiver functionality.

35d2cad

Cleanup + fixes for sender.

da8abac

One more sender fix.

c997aed

Start receiver.

06770db

Make sure to send responses back.

2378c7e

eskimor added 4 commits June 21, 2021 11:01

WIP: Exposed authority discovery

3910cf4

Merge branch 'master' into rk-dispute-distribution-impl

840a046

Make tests pass.

5b10c78

Fully featured receiver.

d2aa4ff

eskimor commented Jun 21, 2021

View reviewed changes

node/network/dispute-distribution/src/receiver/mod.rs Outdated Show resolved Hide resolved

rphmeier reviewed Jul 8, 2021

View reviewed changes

node/subsystem-test-helpers/src/lib.rs Outdated Show resolved Hide resolved

rphmeier reviewed Jul 8, 2021

View reviewed changes

node/subsystem-util/src/runtime/mod.rs

&mut self,

ctx: &mut Context,

sender: &mut Sender,

Copy link

Contributor

rphmeier Jul 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Glad to see this change.

rphmeier reviewed Jul 8, 2021

View reviewed changes

node/subsystem/src/messages.rs Outdated Show resolved Hide resolved

rphmeier reviewed Jul 8, 2021

View reviewed changes

node/network/dispute-distribution/src/lib.rs Outdated Show resolved Hide resolved

rphmeier reviewed Jul 8, 2021

View reviewed changes

node/network/dispute-distribution/src/lib.rs Outdated Show resolved Hide resolved

rphmeier reviewed Jul 8, 2021

View reviewed changes

node/network/dispute-distribution/src/lib.rs Outdated Show resolved Hide resolved

rphmeier reviewed Jul 8, 2021

View reviewed changes

node/subsystem/src/messages.rs Outdated Show resolved Hide resolved

rphmeier reviewed Jul 8, 2021

View reviewed changes

node/network/dispute-distribution/src/lib.rs Show resolved Hide resolved

eskimor added 6 commits July 8, 2021 18:43

Merge branch 'master' into rk-dispute-distribution-impl

90b46f4

Unwire dispute distribution.

210b853

Review remarks.

58b1bee

Review remarks.

59c76c8

Better docs.

3cd91cb

Merge branch 'master' into rk-dispute-distribution-impl

dedbaa8

rphmeier reviewed Jul 8, 2021

View reviewed changes

node/network/dispute-distribution/src/receiver/error.rs Show resolved Hide resolved

rphmeier approved these changes Jul 8, 2021

View reviewed changes

eskimor merged commit f9d71f8 into master Jul 9, 2021

eskimor deleted the rk-dispute-distribution-impl branch July 9, 2021 02:29

drahnr reviewed Jul 9, 2021

View reviewed changes

node/core/dispute-coordinator/src/lib.rs Show resolved Hide resolved

drahnr reviewed Jul 9, 2021

View reviewed changes

node/core/dispute-coordinator/src/lib.rs Show resolved Hide resolved

drahnr reviewed Jul 9, 2021

View reviewed changes

node/network/dispute-distribution/src/metrics.rs Show resolved Hide resolved

drahnr reviewed Jul 9, 2021

View reviewed changes

node/overseer/src/lib.rs Show resolved Hide resolved

drahnr reviewed Jul 9, 2021

View reviewed changes

eskimor added a commit that referenced this pull request Aug 4, 2021

Follow up for #3282

0f5e5ae

Incorporating post merge review remarks.

eskimor added a commit that referenced this pull request Aug 6, 2021

Follow up for #3282 (#3571)

3ef9924

Incorporating post merge review remarks.

github-actions bot mentioned this pull request Aug 24, 2021

Update substrate/polkadot/cumulus from v0.9.8 to v0.9.9 moonbeam-foundation/moonbeam#729

Closed

eskimor mentioned this pull request Nov 11, 2022

Member request polkadot-fellows/seeding#57

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dispute distribution implementation #3282

Dispute distribution implementation #3282

eskimor commented Jun 17, 2021 •

edited

Loading

rphmeier commented Jun 20, 2021

eskimor commented Jun 20, 2021 via email

rphmeier Jul 8, 2021

rphmeier Jul 8, 2021

rphmeier Jul 8, 2021

eskimor Jul 8, 2021 •

edited

Loading

drahnr left a comment



		/// Get mock keystore with `Ferdie` key.
		pub fn make_ferdie_keystore() -> SyncCryptoStorePtr {

Dispute distribution implementation #3282

Dispute distribution implementation #3282

Conversation

eskimor commented Jun 17, 2021 • edited Loading

In particular:

Todos:

rphmeier commented Jun 20, 2021

eskimor commented Jun 20, 2021 via email

rphmeier Jul 8, 2021

Choose a reason for hiding this comment

rphmeier Jul 8, 2021

Choose a reason for hiding this comment

rphmeier Jul 8, 2021

Choose a reason for hiding this comment

eskimor Jul 8, 2021 • edited Loading

Choose a reason for hiding this comment

drahnr left a comment

Choose a reason for hiding this comment

eskimor commented Jun 17, 2021 •

edited

Loading

eskimor Jul 8, 2021 •

edited

Loading