Skip to content

Commit

Permalink
update essential ds tasks & task-hierarchy w/ Reasoning -> Scientific…
Browse files Browse the repository at this point in the history 
… Reasoning
  • Loading branch information
@garyhlai
garyhlai committed Oct 19, 2021
1 parent ae4a373 commit 9368d76
Show file tree
Hide file tree
Showing 3 changed files with 70 additions and 62 deletions.
2 changes: 1 addition & 1 deletion doc/task-hierarchy.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@
- `Hallucination`: Given a context, generate imaginary content e.g. given a sentence, generate a story/poem.
- `Hate Speech Detection`
- `Hypernym Discovery`
- `Incorrect Answer Generation`
- `Intent Detection`
- `Language Identification`
- `Mathematics`
Expand Down Expand Up @@ -66,6 +65,7 @@
- `Reasoning -> Reasoning with Symbols`: Tasks where symbols represent various things e.g. if X is the number of apples in the freeze today morning and Y is the number remaining after I ate a few apples, X-Y is the number of apples I ate.
- `Reasoning -> Spatial Reasoning`
- `Reasoning -> Temporal Reasoning`
- `Reasoning -> Scientific Reasoning`
- `Relation Prediction`
- `Relevancy Estimation`
- `Review Generation`
Expand Down
34 changes: 19 additions & 15 deletions tasks/task043_essential_terms_answering_incomplete_questions.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,11 @@
"Daniel Khashabi"
],
"Source": [
"essential"
"essential (https://aclanthology.org/K17-1010.pdf)"
],
"Categories": [
"Answer Generation"
"Question Answering -> Multiple Choice Question Answering",
"Reasoning -> Scientific Reasoning"
],
"Definition": "In this task, you need to answer basic science questions. For each question, several terms are intentionally masked with ***. This masking can cover very few or many of the question words. Clearly if masked terms are important, the modified question will be impossible to answer. Please indicate the correct answer with one of the following responses: \"A\", \"B\", \"C\", \"D\" or \"E\". While \"A\"-\"D\" correspond to the answer options provided in the input, \"E\" represents \"I don't know\" for questions that do not provide enough information. Respond via \"E\" if the question is not answerable.\nThings to avoid: Do not generate anything else apart from one of the following characters: 'A', 'B, 'C', 'D', 'E'.\nEmphasis & Caution: - ",
"Positive Examples": [
Expand Down Expand Up @@ -190,7 +191,7 @@
]
},
{
"input": "Question: *** of any *** *** ___. (A) when the particles of the substance are able to slide past one another (B) at 0\u00c2\u00b0C (C) when a liquid changes to a solid, regardless of temperature (D) when energy is added to a substanc.",
"input": "Question: *** of any *** *** ___. (A) when the particles of the substance are able to slide past one another (B) at 0°C (C) when a liquid changes to a solid, regardless of temperature (D) when energy is added to a substanc.",
"output": [
"E."
]
Expand Down Expand Up @@ -624,7 +625,7 @@
]
},
{
"input": "Question: *** of any substance *** ___. (A) when the particles of the substance are able to slide past one another (B) at 0\u00c2\u00b0C (C) when a liquid changes to a solid, regardless of temperature (D) when energy is added to a substanc.",
"input": "Question: *** of any substance *** ___. (A) when the particles of the substance are able to slide past one another (B) at 0°C (C) when a liquid changes to a solid, regardless of temperature (D) when energy is added to a substanc.",
"output": [
"E."
]
Expand Down Expand Up @@ -1491,7 +1492,7 @@
]
},
{
"input": "Question: *** of *** *** *** ___. (A) when the particles of the substance are able to slide past one another (B) at 0\u00c2\u00b0C (C) when a liquid changes to a solid, regardless of temperature (D) when energy is added to a substanc.",
"input": "Question: *** of *** *** *** ___. (A) when the particles of the substance are able to slide past one another (B) at 0°C (C) when a liquid changes to a solid, regardless of temperature (D) when energy is added to a substanc.",
"output": [
"E."
]
Expand Down Expand Up @@ -2151,7 +2152,7 @@
]
},
{
"input": "Question: *** *** *** *** *** *** ***? (A) Summers in Tucson are hot and dry. (B) Winters in Flagstaff are cold and snowy. (C) The temperature is 55\u00c2\u00b0 F in Winslow today. (D) The Sun will set in Yuma today at 6:45 p.m.",
"input": "Question: *** *** *** *** *** *** ***? (A) Summers in Tucson are hot and dry. (B) Winters in Flagstaff are cold and snowy. (C) The temperature is 55° F in Winslow today. (D) The Sun will set in Yuma today at 6:45 p.m.",
"output": [
"E."
]
Expand Down Expand Up @@ -2770,7 +2771,7 @@
]
},
{
"input": "Question: *** of any substance occurs ___. (A) when the particles of the substance are able to slide past one another (B) at 0\u00c2\u00b0C (C) when a liquid changes to a solid, regardless of temperature (D) when energy is added to a substanc.",
"input": "Question: *** of any substance occurs ___. (A) when the particles of the substance are able to slide past one another (B) at 0°C (C) when a liquid changes to a solid, regardless of temperature (D) when energy is added to a substanc.",
"output": [
"E."
]
Expand Down Expand Up @@ -2968,7 +2969,7 @@
]
},
{
"input": "Question: Which statement is a description of weather? (A) Summers in Tucson are hot and dry. (B) Winters in Flagstaff are cold and snowy. (C) The temperature is 55\u00c2\u00b0 F in Winslow today. (D) The Sun will set in Yuma today at 6:45 p.m.",
"input": "Question: Which statement is a description of weather? (A) Summers in Tucson are hot and dry. (B) Winters in Flagstaff are cold and snowy. (C) The temperature is 55° F in Winslow today. (D) The Sun will set in Yuma today at 6:45 p.m.",
"output": [
"C."
]
Expand Down Expand Up @@ -3783,7 +3784,7 @@
]
},
{
"input": "Question: Freezing of *** substance occurs ___. (A) when the particles of the substance are able to slide past one another (B) at 0\u00c2\u00b0C (C) when a liquid changes to a solid, regardless of temperature (D) when energy is added to a substanc.",
"input": "Question: Freezing of *** substance occurs ___. (A) when the particles of the substance are able to slide past one another (B) at 0°C (C) when a liquid changes to a solid, regardless of temperature (D) when energy is added to a substanc.",
"output": [
"E."
]
Expand Down Expand Up @@ -5651,7 +5652,7 @@
]
},
{
"input": "Question: *** statement *** *** description *** weather? (A) Summers in Tucson are hot and dry. (B) Winters in Flagstaff are cold and snowy. (C) The temperature is 55\u00c2\u00b0 F in Winslow today. (D) The Sun will set in Yuma today at 6:45 p.m.",
"input": "Question: *** statement *** *** description *** weather? (A) Summers in Tucson are hot and dry. (B) Winters in Flagstaff are cold and snowy. (C) The temperature is 55° F in Winslow today. (D) The Sun will set in Yuma today at 6:45 p.m.",
"output": [
"C."
]
Expand Down Expand Up @@ -6205,7 +6206,7 @@
]
},
{
"input": "Question: *** *** *** *** description *** weather? (A) Summers in Tucson are hot and dry. (B) Winters in Flagstaff are cold and snowy. (C) The temperature is 55\u00c2\u00b0 F in Winslow today. (D) The Sun will set in Yuma today at 6:45 p.m.",
"input": "Question: *** *** *** *** description *** weather? (A) Summers in Tucson are hot and dry. (B) Winters in Flagstaff are cold and snowy. (C) The temperature is 55° F in Winslow today. (D) The Sun will set in Yuma today at 6:45 p.m.",
"output": [
"E."
]
Expand Down Expand Up @@ -8695,7 +8696,7 @@
]
},
{
"input": "Question: Which *** is a *** of ***? (A) Summers in Tucson are hot and dry. (B) Winters in Flagstaff are cold and snowy. (C) The temperature is 55\u00c2\u00b0 F in Winslow today. (D) The Sun will set in Yuma today at 6:45 p.m.",
"input": "Question: Which *** is a *** of ***? (A) Summers in Tucson are hot and dry. (B) Winters in Flagstaff are cold and snowy. (C) The temperature is 55° F in Winslow today. (D) The Sun will set in Yuma today at 6:45 p.m.",
"output": [
"E."
]
Expand Down Expand Up @@ -9052,7 +9053,7 @@
]
},
{
"input": "Question: Which statement is a *** of ***? (A) Summers in Tucson are hot and dry. (B) Winters in Flagstaff are cold and snowy. (C) The temperature is 55\u00c2\u00b0 F in Winslow today. (D) The Sun will set in Yuma today at 6:45 p.m.",
"input": "Question: Which statement is a *** of ***? (A) Summers in Tucson are hot and dry. (B) Winters in Flagstaff are cold and snowy. (C) The temperature is 55° F in Winslow today. (D) The Sun will set in Yuma today at 6:45 p.m.",
"output": [
"E."
]
Expand Down Expand Up @@ -10136,7 +10137,7 @@
]
},
{
"input": "Question: *** of *** *** occurs ___. (A) when the particles of the substance are able to slide past one another (B) at 0\u00c2\u00b0C (C) when a liquid changes to a solid, regardless of temperature (D) when energy is added to a substanc.",
"input": "Question: *** of *** *** occurs ___. (A) when the particles of the substance are able to slide past one another (B) at 0°C (C) when a liquid changes to a solid, regardless of temperature (D) when energy is added to a substanc.",
"output": [
"E."
]
Expand Down Expand Up @@ -11882,5 +11883,8 @@
],
"Instruction_language": [
"English"
],
"Domains": [
"Natural Science"
]
}
}
Loading

0 comments on commit 9368d76

Please sign in to comment.