Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add lib.yang.list hash trie to store YANG lists #1490

Merged
merged 84 commits into from
Dec 15, 2022

Conversation

eugeneia
Copy link
Member

@eugeneia eugeneia commented Nov 18, 2022

This adds a hash trie implementation to be used for storing all YANG lists. The motivation for this is twofold:

  • before YANG lists could be represented as three different data structures: lib.ctable, lib.cltable, and native Lua tables. Ctable and cltable representations are important optimizations as they can store huge configurations (think routing tables) efficiently. With this PR all lists are represented as lib.yang.list hash tries, which should give both the efficiency of a specialized data structure and the ease of access of native tables via the use of metatables.
  • a downside of ctables and cltables was that they are unordered collections. YANG however specifies the ordered-by statement for lists. lib.yang.list tries are ordered and hence support this statement.

Implementation wise this PR:

  • enables LUA52COMPAT to provide full operator overloading for lib.yang.list
  • enabling LUA52COMPAT uncovered some unrealiability in StackTracePlus. I’ve checked in the latest version, but we still need to fallback to debug.traceback() sometimes.

Functionality wise this PR affects:

  • Snabb lwAFTR: we are aiming for feature parity. There is a new extra step when loading the initial configuration where the YANG list is compiled to a ctable binding table. Data-plane behavior should be unchanged.
  • Snabbflow (snabb ipfix probe): flow director classes in schema use ordered-by user, and leaves referring to other key leaves now use the leafref type.
  • Future users of lib.yang and lib.ptree: more types of configuration schemas should be able to be represented efficiently now. lib.yang.list stores strings off-heap, so huge lists with string keys or members should be more ecomonical.

Todo:

  • tests pass
  • benchmarks should run and not have degraded in performance

@eugeneia eugeneia self-assigned this Nov 18, 2022
@eugeneia eugeneia added merged release-balaclava December 2022 release labels Nov 28, 2022
eugeneia added a commit to eugeneia/snabb that referenced this pull request Nov 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant