Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Consolidate the blobs API #42

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open

refactor: Consolidate the blobs API #42

wants to merge 13 commits into from

Conversation

rklaehn
Copy link
Collaborator

@rklaehn rklaehn commented Jan 2, 2025

Description

The API has grown quite a bit. This is an attempt to structure it a bit better.

There is going to be a much larger change later when we add multi-provider downloads. That requires changing the entire progress story. But for now this just attempts a small clean up.

Breaking Changes

  • fns that import into or export from the db are moved into the store module
    • get::db::blob_info moved to store
    • get::db::valid_ranges moved to store
    • get::db::get_to_db moved to store
    • get::db::get_to_db_in_steps moved to store
    • get::db::GetStateNeedsConn moved to store
    • get::db::BlobId moved to store
    • get::db::BlobInfo moved to store
    • get::db::DownloadProgress moved to store
    • get::db::GetState moved to store
    • export::export_blob moved to store
    • export::export moved to store
    • export::export_collection moved to store
    • export::ExportProgress moved to store
  • format/collection is feature gated
  • DownloadProgress renamed to DownloadProgressEvent and exported in rpc/client/blobs
  • BlobProgress renamed to BlobProgressEvent and exported in rpc/client/blobs

Notes & open questions

Maybe not for this PR, but there is one open question. Basically to OO or not to OO.

There are a large number of fns that need a store. Currently these are fns that just take a impl Store as first or second parameter. I could make these member functions of the trait. E.g. handle_connection(store, conn, events) could be store.handle_connection(conn, events). Would that be easier to discover for people?

Change checklist

  • Self-review.
  • Documentation updates following the style guide, if relevant.
  • Tests if relevant.
  • All breaking changes documented.

Copy link

github-actions bot commented Jan 2, 2025

Documentation for this PR has been generated and is available at: https://n0-computer.github.io/iroh-blobs/pr/42/docs/iroh_blobs/

Last updated: 2025-01-03T16:36:33Z

@rklaehn rklaehn changed the title Consolidate the blobs API refactor: Consolidate the blobs API Jan 2, 2025
@rklaehn rklaehn marked this pull request as ready for review January 2, 2025 10:38
@rklaehn rklaehn marked this pull request as draft January 2, 2025 10:41
@rklaehn rklaehn requested a review from dignifiedquire January 3, 2025 15:25
@rklaehn rklaehn marked this pull request as ready for review January 3, 2025 15:25
@rklaehn rklaehn self-assigned this Jan 3, 2025
get::{db::get_to_db_in_steps, error::GetError},
store::Store,
get::Error,
store::{get_to_db_in_steps, FetchState, FetchStateNeedsConn, Store},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so there is get and fetch now? this is confusing 😅

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

everything that needs a store is in the store module. I might even make it a fn on the store traits (see the OO or not OO question)

One reason for this is that I think it would be neat if you could hide the entire store behind a ff, but to do that you must make sure that store stuff is not all over the place, otherwise you will go crazy.


/// Progress updates for the get operation.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum DownloadProgress {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we normalize get, download and fetch? they sound like the same thing for most users

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Normalize to what? get, download or fetch? I guess the most descriptive is download, given that it is a downloader.

But then get is more accurate for the case where the stuff is already local.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 🏗 In progress
Development

Successfully merging this pull request may close these issues.

2 participants