Skip to content
This repository has been archived by the owner on Dec 16, 2022. It is now read-only.

Commit

Permalink
Update documentation for bert_pooler.py (#3181)
Browse files Browse the repository at this point in the history
* Update documentation for bert_pooler.py

Documenting that the BertPooler actually returns just the [CLS] token from the BERT paper, followed by a non-linear transformation.

* Update bert_pooler.py
  • Loading branch information
Swabha Swayamdipta authored and DeNeutoy committed Aug 21, 2019
1 parent 112d8d0 commit 817814b
Showing 1 changed file with 4 additions and 2 deletions.
6 changes: 4 additions & 2 deletions allennlp/modules/seq2vec_encoders/bert_pooler.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,10 @@
@Seq2VecEncoder.register("bert_pooler")
class BertPooler(Seq2VecEncoder):
"""
The pooling layer at the end of the BERT model. If you want to use the pretrained
BERT model to build a classifier and you want to use the AllenNLP token-indexer ->
The pooling layer at the end of the BERT model. This returns an embedding for the
[CLS] token, after passing it through a non-linear tanh activation; the non-linear layer
is also part of the BERT model. If you want to use the pretrained BERT model
to build a classifier and you want to use the AllenNLP token-indexer ->
token-embedder -> seq2vec encoder setup, this is the Seq2VecEncoder to use.
(For example, if you want to experiment with other embedding / encoding combinations.)
Expand Down

0 comments on commit 817814b

Please sign in to comment.