Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#168] feat(core): Refactor the ID field of entity to assign a unique id for this field #348

Merged
merged 1 commit into from
Sep 8, 2023

Conversation

jerryshao
Copy link
Contributor

What changes were proposed in this pull request?

This PR proposes to make some changes to entity's ID field.

  1. Assign a unique ID for each entity when creating.
  2. Remove the parent ID field for catalog, schema and table entity (since it is useless currently).

Why are the changes needed?

This is the subtask for #250 . With unique ID assigned to each entity, we could leverage this unique ID as a "record" or "watermark" to be bound between the underlying sources and graviton store, which can guarantee the SSOT of entities.

Fix: #168

Does this PR introduce any user-facing change?

  1. This change removes the parent id field of catalog, schema, and table entity's proto definition.

How was this patch tested?

With the existing UTs.

@jerryshao jerryshao self-assigned this Sep 7, 2023
@github-actions
Copy link

github-actions bot commented Sep 7, 2023

Code Coverage Report

Overall Project 63.38% -0.23% 🟢
Files changed 71.79% 🟢

Module Coverage
core 71.46% -0.72% 🟢
catalog-hive 58.82% 🟢
Files
Module File Coverage
core HasIdentifier.java 100% 🟢
SchemaEntitySerDe.java 100% 🟢
TableEntitySerde.java 100% 🟢
Entity.java 96.94% 🟢
BaseMetalakeSerDe.java 90.83% 🟢
CatalogEntitySerDe.java 85.71% 🟢
BaseTable.java 80.81% 🟢
BaseSchema.java 79.78% 🟢
BaseMetalake.java 79.4% 🟢
MetalakeManager.java 78.16% -1.44% 🟢
CatalogEntity.java 73.56% -11.53% 🔴
CatalogManager.java 61.99% 🟢
GravitonEnv.java 0% -22.31% 🔴
catalog-hive HiveTable.java 94.4% 🟢
HiveSchema.java 83.95% 🟢
HiveCatalogOperations.java 66.07% 🟢

@yuqi1129
Copy link
Contributor

yuqi1129 commented Sep 7, 2023 via email

Copy link
Contributor

@yuqi1129 yuqi1129 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jerryshao jerryshao merged commit c08c842 into apache:main Sep 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Improvement] Refactor the ID field in entity
2 participants