Task 116 Com2sense task - Commonsense reasoning

[Palipoor]

A commonsense reasoning task based on the Com2Sense dataset.
https://arxiv.org/abs/2106.00969

[swarooprm]

Please rename your task to task116 since task115 is already submitted in the latest Pull request.

[danyaljj]

task_116 -> task116
I would suggest changing "sentence is true" to whether "the provided event is plausible".

[Palipoor]

That makes sense, thanks.

[danyaljj]

For the last column, maybe also add Classification, too.

[danyaljj]

I am not sure what it means for a sentence to be "true" or "false".
I would suggest changing the phrasing of the instruction, to something along the line of, "you need to decide whether provided story in the following sentence is a plausible scenario and it matches your commonsense".

[danyaljj]

Similarly, instead of saying "For true sentences output 'True'" (which sounds like a recursive statement), say, "if the given event is a plausible one, output 'True'" and so on.

[danyaljj]

Expand this a bit, something like this: The instructions and the instance are extracted from the Com2Sense ()

[danyaljj]

Add Classification, too.

[danyaljj]

That's all the thoughts I had.
After addressing them, @swarooprm feel free to merge when you're happy.

[danyaljj]

@Palipoor pinging the thread to make sure you have seen the comments above.

[Palipoor]

@danyaljj Yes I have seen them, I was just too busy these days. Will handle them soon.

[danyaljj]

Thanks! This is good to go, from my perspective. ✅

[swarooprm]

Looks good! Merging this task.
However, I think you can create more tasks from the same dataset. Section 2.2. contain multiple steps in the crowdsourcing process, you can convert some of those to tasks. To clarify, any input-output can constitute a task. A task does not necessarily have to be an end task in a dataset. Feel free to create new PR if you find any new task using this dataset.