feat(provider): add 'CollectionAdded' Provider event #1131
Conversation
| /// Makes a an RPC endpoint that uses a QUIC transport | ||
| fn make_rpc_endpoint( |
There was a problem hiding this comment.
went on a wild goose chase looking for a way to construct an RPC endpoint to give to the builder, and at one point wanted to move this, only to discover node.controller(). This comment is a momento of hour(s) lost.
There was a problem hiding this comment.
Sorry about this. So the idea is that you always want a cheap in memory way to control your node via RPC like calls.
Basically like an in memory message channel between you and the node, but nicer than the usual pattern of sending messages that contain a oneshot sender for the reply. This is what the controller is for.
| )) { | ||
| warn!("failed to send CollectionAdded event: {:?}", e); |
There was a problem hiding this comment.
this might be overkill, but I figured I'd err on the side of being pedantic
iroh/src/node.rs
Outdated
| while let Ok(msg) = events.recv().await { | ||
| match msg { | ||
| Event::ByteProvide(e) => { | ||
| if let iroh_bytes::provider::Event::CollectionAdded { hash } = e { | ||
| got_hash = Some(hash); | ||
| break; | ||
| } | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
This test could be expanded in the future for more extensive provider event testing
| .await | ||
| .unwrap(); | ||
|
|
||
| let _drop_guard = node.cancel_token().drop_guard(); |
There was a problem hiding this comment.
I am not sure, @flub ?
b5
force-pushed
the
collection-added-event
branch
from
729e44e to
5a16586
Compare
There was a problem hiding this comment.
This seems quite reasonable.
However, please be aware that there is a problem with using the events for any bookkeeping. I frequently have the problem that the many_files test fails because as you optimize things it can produce events very quickly, and the broadcast channel has only a fixed buffer size before it starts dropping messages.
If you want to use these events for anything important, we would have to have a way to prevent the dropping. Unbounded channels, or blocking when you are unable to send.
See failing test in ci, because of exactly that reason |
iroh-bytes/src/provider.rs
Outdated
| @@ -34,6 +34,11 @@ pub use ticket::Ticket; | |||
| /// Events emitted by the provider informing about the current status. | |||
| #[derive(Debug, Clone)] | |||
| pub enum Event { | |||
| /// A new collection has been added via an RPC call | |||
There was a problem hiding this comment.
Can we make this more generic, so it always happens, not just on RPC call, like the other events?
There was a problem hiding this comment.
After talking it over with @flub, they're one-and-the-same. I've updated the comment to match.
I've confirmed that: All of these events require a provider (duh), and that all call sites where a provider calls collection::create_collection, the provider is wrapping with a CollectionCreated event. This works rn b/c everything is flowing through node::provide0.
We should keep an eye on this for the moment when we can do getting & providing in the same process "mode".
|
(Feel free to mark this as offtopic, I couldn't find a repo for iroh.network) @b5 https://iroh.network shows |
|
@Winterhuman, lol yeah the with regard to https://iroh.network, nice find 😄 . It isn't open source atm, and isn't scheduled for wide release until later this year. If you'd like an invite to test it out, feel free to email me using [my github handle] @ n0.computer. Thanks! |
|
Dunno what this did, but seems tests run forever. Canceling the current batch, but needs looking into it. |
b5
force-pushed
the
collection-added-event
branch
from
5e46d1a to
e1e09cb
Compare
Need this for iroh.network, which listens to an in-process node for provider collection construction.
I'm piggybacking this to also add a test that does minimal node setup, which should form the basis of writing examples later. Learned many things in the process.