You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
According to issue /~https://github.com/mlcommons/submission_training_1.0/issues/39, the number of training samples is 117266. Many submissions do hardcode this value, even though the reference does not. On that specific issue the submission in question used a different value.
The decision was to add train_samples, and eval_samples as hyperparameters + the related compliance checker rules so we avoid such issues in the future.
The text was updated successfully, but these errors were encountered:
Good point. Marek, since it is all the benchmarks, if you need any help let me know.
So, restating the problem: In the 1.0 submission training and eval samples were found to be off for a couple of submissions. This happened also in 0.7 and went undetected.
Let's add compliance checks to avoid such issues in the future.
emizan76
changed the title
Add hparams and compliance checks for SSD
Add hparams and compliance checks for training and eval samples for all benchmarks
Jun 16, 2021
According to issue /~https://github.com/mlcommons/submission_training_1.0/issues/39, the number of training samples is 117266. Many submissions do hardcode this value, even though the reference does not. On that specific issue the submission in question used a different value.
The decision was to add train_samples, and eval_samples as hyperparameters + the related compliance checker rules so we avoid such issues in the future.
The text was updated successfully, but these errors were encountered: