Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add warnings to PineconeDocumentStore about indexing metadata if filters return no documents #3086

Merged
merged 5 commits into from
Aug 30, 2022

Conversation

Namoush
Copy link
Contributor

@Namoush Namoush commented Aug 22, 2022

Related Issues

Proposed Changes:

This pull request adds a warning for query_by_embedding and get_all_documents_generator in PineconeDocumentStore when a filter is used through an user action but no results are obtained. As it is referenced in this issue #3021, there's no way to know which metadata fields are indexed currently, so these warnings serve as a hint for users to revisit their queries in the circumstances referenced above (used a filter + no results obtained).

How did you test it?

query_by_embedding tested ✅(Edited Tutorial 1 with some help from this notebook here that was added as testing material in the PineconeDocumentStore implementation PR #2254
get_all_documents_generator tested ✅ through the upstream method get_all_documents

Notes for the reviewer

I have preemptively opened the pull request as it is encouraged in the CONTRIBUTING section.
I am still exploring the framework and this is my first os pull request so bear with me 😅.

Checklist

@Namoush Namoush requested a review from a team as a code owner August 22, 2022 15:16
@Namoush Namoush requested review from bogdankostic and removed request for a team August 22, 2022 15:16
@Namoush Namoush marked this pull request as draft August 22, 2022 16:05
@ZanSara ZanSara requested review from ZanSara and removed request for bogdankostic August 23, 2022 07:41
@ZanSara
Copy link
Contributor

ZanSara commented Aug 24, 2022

Hello @Namoush! Thank you for this PR 🙂 There is a big PineconeDocumentStore refactoring by the Pinecone team being merged today, so I will hold this one up until it's done. There are probably going to be some conflicts, so keep an eye out for them.

@Namoush
Copy link
Contributor Author

Namoush commented Aug 26, 2022

Thanks for merging main's changes into this branch + solving the conflicts @ZanSara . I did fix the if clause evaluations and manually tested both flows.

@Namoush Namoush marked this pull request as ready for review August 26, 2022 11:48
@ZanSara
Copy link
Contributor

ZanSara commented Aug 30, 2022

Hey @Namoush, I've merged main again as you stumbled upon a new pylint warning that prevented the CI from running. If the tests are all green I'm going to merge this PR today 🙂

Copy link
Contributor

@ZanSara ZanSara left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@ZanSara ZanSara merged commit 911a2fa into deepset-ai:main Aug 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Pinecone: warn users about indexing metadata if filters return no documents
2 participants