Add "sub_module" argument in PretrainedTransformerMismatchedEmbedder (#…

…5580)

* Add "submodule" argument in PretrainedTransformerMismatchedEmbedder

* Update CHANGELOG.md

CHANGELOG.md

@@ -17,6 +17,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 - We can now transparently read compressed input files during prediction.
 - LZMA compression is now supported.
+- Added the argument `sub_module` in `PretrainedTransformerMismatchedEmbedder`
 
 
 ## [v2.9.0](/~https://github.com/allenai/allennlp/releases/tag/v2.9.0) - 2022-01-27

allennlp/modules/token_embedders/pretrained_transformer_mismatched_embedder.py

@@ -26,6 +26,10 @@ class PretrainedTransformerMismatchedEmbedder(TokenEmbedder):
         through the transformer model independently, and concatenate the final representations.
         Should be set to the same value as the `max_length` option on the
         `PretrainedTransformerMismatchedIndexer`.
+    sub_module: `str`, optional (default = `None`)
+        The name of a submodule of the transformer to be used as the embedder. Some transformers naturally act
+        as embedders such as BERT. However, other models consist of encoder and decoder, in which case we just
+        want to use the encoder.
     train_parameters: `bool`, optional (default = `True`)
         If this is `True`, the transformer weights get updated during training.
     last_layer_only: `bool`, optional (default = `True`)
@@ -65,6 +69,7 @@ def __init__(
         self,
         model_name: str,
         max_length: int = None,
+        sub_module: str = None,
         train_parameters: bool = True,
         last_layer_only: bool = True,
         override_weights_file: Optional[str] = None,
@@ -80,6 +85,7 @@ def __init__(
         self._matched_embedder = PretrainedTransformerEmbedder(
             model_name,
             max_length=max_length,
+            sub_module=sub_module,
             train_parameters=train_parameters,
             last_layer_only=last_layer_only,
             override_weights_file=override_weights_file,