Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move account_member_index to a plugin #682

Closed
abitmore opened this issue Feb 20, 2018 · 6 comments
Closed

Move account_member_index to a plugin #682

abitmore opened this issue Feb 20, 2018 · 6 comments

Comments

@abitmore
Copy link
Member

account_member_index is only being used in API nodes. Block producing nodes and seed nodes don't need it. We can save RAM and CPU if it's moved to a plugin (by not loading it).

@abitmore abitmore added this to the Future Non-Consensus-Changing Release milestone Feb 20, 2018
@jmjatlanta
Copy link
Contributor

jmjatlanta commented Mar 21, 2018

Note: I am not claiming this issue. I am just gathering requirements.

Programming tasks:

  1. Create a plugin that adds the secondary index account_member_index to the account_index.
    • the account_member_index should no longer be added in db_init.cpp.
  2. Load this new plugin if the application has (or will have) an rpc-endpoint or rpc-tls-endpoint, as this indicates it is a node that provides API access.

Testing tasks:

  1. Verify that a node that has no rpc endpoint does not have the secondary index
  2. Verify that a node that has an rpc endpoint has the secondary index, and that a query that uses that index can find entries successfully.

Edit: updated based on 1 comment below.

@pmconrad
Copy link
Contributor

  1. An API node is a node that is listening on a rpc-endpoint or rpc-tls-endpoint
  2. This should work without a replay. Upon shutdown, only the objects in the index are dumped. On the next startup the index (and its secondaries) are populated by reading the objects from disk.

@jmjatlanta
Copy link
Contributor

jmjatlanta commented Mar 22, 2018

A SWAG of an estimate:

  • Buildout of plugin: 0.5h
  • Removal of index in db_init.cpp and addition of index in plugin: 1h
  • Load plugin at correct time 1h
  • Create and complete tests 2h

Please provide your comments, and I will adjust.

@abitmore
Copy link
Member Author

To got this done, we may need to adjust implementation of certain API's, e.g. check if the plugin is loaded and etc.

However, as mentioned in #683, this will result only 2-3% performance gain (measured in time). I don't know how much RAM it will save, guess it shouldn't be significant since a node enabled witness plugin only now uses 1.5G RAM only. Then we decide whether do it now, likely the answer is "no" or "low priority".

@pmconrad
Copy link
Contributor

we may need to adjust implementation of certain API's, e.g. check if the plugin is loaded and etc.

It would be generally useful if libraries/application could automatically make use of APIs provided by the enabled plugins, without requiring knowledge (or specific code) for these APIs. See also #65 .

@pmconrad
Copy link
Contributor

Resolved by #1918

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants