Tasks 1407-1417: DART, ajgt_twitter_ar, and youtube_caption_corrections.

[RushangKaria]

Created a set of tasks from the DART, ajgt_twitter_ar and youtube_caption_corrections datasets available on huggingface.

Tasks

• DART
  • Question Generation
  • Sentence Similarity Classification
  • Text Generation
  • Relationship Extraction
  • Subject/Predicate/Object Identification
• ajgt_twitter_ar
  • Sentiment Classification
• youtube_captions_corrections
  • Grammar Correction
  • Incorrect Grammar Classification
  • Slot Filling
  • Slot Generation

[danyaljj]

Please make sure to set the "Categories": [""] and "Domains": [""] fields according to the taxonomy presented here.

[swarooprm]

@Mehrad0711, wondering if you can help review these tasks?

[Mehrad0711]

Things look good in general. Although, I have some suggestions:

task1334_dart_question_generation:
Definition: "specified in the at least" --> "specified in at least"
Definition: "such that a it is a well-formed question" --> "such that it is a well-formed question"

"The input is a list of triplets of the form [subject, predicate, object] and the output is a list of questions that are sentences based on the triplets but with the subject and/or object replaced with slots."
It seems all outputs are single sentences not a list? Also slots need to be defined. Do you mean blank lines by slots?

Probably add a line that says: "All input triplets need not be used in the question for it to be valid."

task1336_dart_text_generation:
Do all input triplets need to be used in the question for it to be valid? Please indicate that in the Definition.
Is a question valid if it contains more information than what is provided in the input triplets?

I noticed input triplets like this: ['[TABLECONTEXT]', '[TITLE]', 'Olivia McKoy']] Are [.] indicating special tokens or placeholders? Please describe what these are in the definition.

task1342_youtube_caption_corrections_grammar_correction:
The positive examples can be more varied. It seems all three have the same mistakes. Also in the "explanation", you may wanna describe what is fixed.
In definition, if you can add more examples of grammar mistakes that need to be fixed, it would be helpful for the model to understand what needs attention. I see in the other task you have "unknown" and "special" errors. You may add some examples for those.

task1343_youtube_caption_corrections_incorrect_grammar_classification:
Please expand on what (unknown error) and (special error) each represents. What errors classify as unknown vs special?

task1344_youtube_caption_corrections_slot_filling:
I'm not sure if this is a reasonable task. There are a lot of words that can fill out the slots and achieve the objective but output only considers one of them to be correct. We may wanna drop this task.

[swarooprm]

@RushangKaria, could you update your files based on reviewer comments?

[RushangKaria]

Hi @swarooprm

Yes, ill make the necessary changes and push a new set of changes. Thanks for the reviews!

[swarooprm]

Hi @swarooprm

Yes, ill make the necessary changes and push a new set of changes. Thanks for the reviews!

@RushangKaria, a gentle reminder!

[RushangKaria]

@swarooprm Sorry, got caught up in some other stuff. Should have an update soon.

[RushangKaria]

Made the changes. Some files have no matching or relevant domain so I've left those unchanged (without a domain).

[Mehrad0711]

Thanks for making the changes. LGTM now!

[swarooprm]

@RushangKaria, can you rename the files to 1407-1417 so that we can go ahead and merge this PR?

[RushangKaria]

Hi @swarooprm,

Yes, Ill rename the files and create a new PR for it.