This repository has been archived by the owner on Dec 16, 2022. It is now read-only.
pieces for multitask learning #2369
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
DeNeutoy
approved these changes
Jan 17, 2019
| from allennlp.data.instance import Instance | ||
| from allennlp.data.iterators.data_iterator import DataIterator | ||
|
|
||
| @DataIterator.register("homogeneous-batch") |
There was a problem hiding this comment.
By convention this should be registered as homogeneous_batch
| If false, it will do the tensorization anew each iteration. | ||
| track_epoch : ``bool``, optional, (default = False) | ||
| If true, each instance will get a ``MetadataField`` containing the epoch number. | ||
| partition_key : ``str``, optional, (default = "dataset") |
There was a problem hiding this comment.
This is a bit gross, I wonder if it's better to allow setting an "origin" attribute on an Instance or something. Maybe not for this PR
There was a problem hiding this comment.
it needs to make it into the model, so if you did that you'd have to change all the "batch to tensor" logic to account for that and then make sure it doesn't somehow collide with other tensors and so on
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
as discussed at happy hour, this includes an
InterleavingDatasetReaderwhich wraps multiple other dataset readers and interleaves their instances (adding aMetadataFieldindicating which dataset each instance came from) and aHomogeneousBatchIterator, which assumes such aMetadataFieldexists and constructs batches that are homogeneous with respect to its value.The only "weird" thing is that the file_path passed to
InterleavingDatasetReader.read()needs to be a JSON-serialized dict { wrapped_reader_key -> file_path }. We discussed alternative designs like passing in a directory and requiring each wrapped reader to know to look for a specific file under the provided directory; I felt like that seemed a little bit too prescriptive about data layout and harder to configure.I believe that with these pieces most of the multitask things that S2 research wants to do should be relatively easy. (Notably, with the file-path-as-JSON-dict innovation, we can just use the usual
Trainer😬 )FYI @amandalynne