Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: listing, no manifest setup #1122

Merged
merged 1 commit into from
Jan 23, 2025

Conversation

nikhilsinhaparseable
Copy link
Contributor

@nikhilsinhaparseable nikhilsinhaparseable commented Jan 20, 2025

the PR solves cases where entire data has no manifest
also fixes the prefix creation for listing

how to verify:

  1. checkout parseable main branch
  2. run parseable locally with minio
  3. open another terminal and run mc trace to analyse the API calls happening to minio
  4. load some data in a stream
  5. use query API - provide the start date and end date as old date where there was no data in the stream
  6. analyse in mc trace that list calls do not happen at all -- this is because of the 1st bug where listing does not happen when entire time range of the query is before the manifest creation date
  7. again use the query API - this time, prove the start date as old date where there was no data in the stream, end date can be today's date
  8. analyse in mc trace that the list calls happen now but the prefix does not contain the stream name

now, when you checkout this PR, you will find both the issues have been fixed.

the PR solves cases where entire data has no manifest
also fixes the prefix creation for listing
@coveralls
Copy link

Pull Request Test Coverage Report for Build 12867825671

Details

  • 0 of 5 (0.0%) changed or added relevant lines in 2 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-0.003%) to 12.711%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/query/listing_table_builder.rs 0 1 0.0%
src/query/stream_schema_provider.rs 0 4 0.0%
Totals Coverage Status
Change from base Build 12855418418: -0.003%
Covered Lines: 2459
Relevant Lines: 19345

💛 - Coveralls

@nikhilsinhaparseable nikhilsinhaparseable changed the title fix: listing bug fix: listing, no manifest setup Jan 21, 2025
@nitisht nitisht merged commit 750140b into parseablehq:main Jan 23, 2025
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants