Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(rust): Allow setting and reading custom schema-level IPC metadata #20066

Merged
merged 6 commits into from
Dec 3, 2024
Merged

feat(rust): Allow setting and reading custom schema-level IPC metadata #20066

merged 6 commits into from
Dec 3, 2024

Conversation

lukapeschke
Copy link
Contributor

closes #17560

@lukapeschke
feat(rust): allow to pass custom schema-level metadata in IpcStreamWr…
f932840 
…iter

Signed-off-by: Luka Peschke <luka.peschke@toucantoco.com>
@lukapeschke lukapeschke changed the title feat(rust): allow to pass custom schema-level metadata in ipc writers feat(rust): Allow setting custom schema-level metadata in ipc writers Nov 29, 2024
@github-actions github-actions bot added enhancement New feature or an improvement of an existing feature rust Related to Rust Polars and removed title needs formatting labels Nov 29, 2024
@lukapeschke
feat(rust): allow to pass custom schema-level metadata in IpcWriter
9451872 
Signed-off-by: Luka Peschke <luka.peschke@toucantoco.com>
@lukapeschke
feat: allow reading custom schema metadata in ipc file and stream rea…
3053a4d 
…ders

Signed-off-by: Luka Peschke <luka.peschke@toucantoco.com>
@lukapeschke lukapeschke force-pushed the schema-metadata-for-ipc branch from c422103 to 3053a4d Compare November 29, 2024 14:28
@lukapeschke lukapeschke changed the title feat(rust): Allow setting custom schema-level metadata in ipc writers feat(rust): Allow setting and reading custom schema-level IPC metadata Nov 29, 2024
@codecov Codecov
Copy link

codecov bot commented Nov 29, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 66.66667% with 29 lines in your changes missing coverage. Please review.

Project coverage is 79.52%. Comparing base (1f104f9) to head (6fb3f28).
Report is 18 commits behind head on main.

Files with missing lines Patch % Lines
crates/polars-io/src/ipc/ipc_file.rs 0.00% 7 Missing ⚠️
crates/polars-io/src/ipc/ipc_stream.rs 30.00% 7 Missing ⚠️
crates/polars-arrow/src/io/ipc/write/schema.rs 64.28% 5 Missing ⚠️
crates/polars-io/src/ipc/write.rs 42.85% 4 Missing ⚠️
crates/polars-arrow/src/io/ipc/write/stream.rs 66.66% 3 Missing ⚠️
crates/polars-arrow/src/io/ipc/write/writer.rs 80.00% 3 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main   #20066      +/-   ##
==========================================
- Coverage   79.53%   79.52%   -0.01%     
==========================================
  Files        1563     1563              
  Lines      217124   217198      +74     
  Branches     2464     2464              
==========================================
+ Hits       172680   172733      +53     
- Misses      43884    43905      +21     
  Partials      560      560

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@lukapeschke lukapeschke marked this pull request as ready for review November 29, 2024 16:10
@lukapeschke lukapeschke mentioned this pull request Nov 29, 2024
Closed
@lukapeschke
fix: only write metadata to file footer in IpcWriter
78654fc 
Signed-off-by: Luka Peschke <luka.peschke@toucantoco.com>
@lukapeschke
refactor: Arc<Option<_>> -> Option<Arc<_>>
4840c1e 
Signed-off-by: Luka Peschke <luka.peschke@toucantoco.com>
@lukapeschke
Merge branch 'main' into schema-metadata-for-ipc
6fb3f28
@nameexhaustion
Copy link
Collaborator

This looks good to me now - the final merge approval would need to be from @ritchie46

@ritchie46 ritchie46 merged commit 0263868 into pola-rs:main Dec 3, 2024
20 checks passed
@lukapeschke lukapeschke deleted the schema-metadata-for-ipc branch December 3, 2024 08:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ritchie46 ritchie46 Awaiting requested review from ritchie46 ritchie46 is a code owner

@orlp orlp Awaiting requested review from orlp orlp is a code owner

@c-peters c-peters Awaiting requested review from c-peters c-peters is a code owner

@coastalwhite coastalwhite Awaiting requested review from coastalwhite coastalwhite is a code owner

@nameexhaustion nameexhaustion Awaiting requested review from nameexhaustion

Assignees
No one assigned
Labels
enhancement New feature or an improvement of an existing feature rust Related to Rust Polars
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Support reading and writing custom IPC metadata
3 participants
@lukapeschke @nameexhaustion @ritchie46