Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support more than one answer/document output per node in pipeline.eval() #1850

Closed
tstadel opened this issue Dec 6, 2021 · 0 comments · Fixed by #1854
Closed

Support more than one answer/document output per node in pipeline.eval() #1850

tstadel opened this issue Dec 6, 2021 · 0 comments · Fixed by #1854
Assignees
@julian-risch @tstadel
Labels
topic:eval type:feature New feature or request

Comments

@tstadel
Copy link
Member

tstadel commented Dec 6, 2021 •
edited
Loading

According to #1719 we need to support multiple outputs per node and type (answer/document).
Needs to be done:

  • introduce new column in eval dataframes ('node_input')
  • set default value 'prediction' to new column during pipeline.eval()
  • extend EvaluationResult to receive node_input and filter for it, also set default value 'prediction'
    (node_input value for 'perfect retriever' would be 'label')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@julian-risch julian-risch

@tstadel tstadel

Labels
topic:eval type:feature New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

introduce node_input param deepset-ai/haystack
2 participants
@julian-risch @tstadel