-
Notifications
You must be signed in to change notification settings - Fork 184
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor(iroh)!: remove tags from downloader #2348
Conversation
I saw that there is lots of logic related to tags in the downloader. Do we really need this? One nice thing about temp tags or tags in general is that you can assign them even before you have the data. So you can do the whole tag and temp tag manager outside the downloader it seems. Or am I missing somehting?
definitely looks like a nice simplification, but @Frando will have to answer if this is missing something crucial |
Aha, interesting. I somehow did not realize that we can create tags before we have the data, I thought only temptags could do that. So I think this is fine. We should document this somewhere on the downloader though (it's public API) that users need to create tags themselves otherwise their newly downloaded blobs will be garbage collected at any time. |
Now I remember something: The advantage of doing the tagging in the downloader is that if you queue the same downloader multiple times from the client API with SetTagOption::Auto you get only one auto tag. Whereas with this PR you'd get multiple auto tags. I think this was the original reason I moved the tagging into the downloader and added the TagSet. |
Not sure that warrants the additional complexity. also, handing the same auto tag to 2 places is a bit dangerous, given that I think currently we are using auto tags as a workaround for the fact that temp tags are not available on the rpc client. E.g. you got one place that does download with auto, then some validation, then removes the auto tag and replaces it with something permanent/named. The other side will not notice this, so they will have an auto tag that is not actually tagging anything anymore. The assumption with an auto tag is that you own that tag, and handing out the same auto tag to 2 places means that you don't. In general I think most uses for SetTagOption::Auto will be replaced by temp tags once we have them in the rpc client. We can make auto just some optional string prefix and a sufficiently large random number, then it does not even have to be part of the rpc api. There is also the question whether to tag before or after. E.g. if you do a download you really care about you would assign a tag before you have the data to make sure the data is fully protected even against a crash/restart as it trickles in. There are so many ways to work with tags that I really think the downloader should not have anything to do with it and the job of protecting the data in some way should be up to the caller. |
please document the breaking changes |
3abfe6e
to
969f3d4
Compare
Description
Remove tags from downloader. Adding temp or persistent tags is now the responsibility of the caller.
Breaking Changes
tag
is removeduntagged
is removedtag
is removedDocs are changed to indicate that tagging is now responibility of the caller...
Notes & open questions
Change checklist