Display activation functions as modules.

[mklimasz]

Currently some of the activation functions (and therefore list of activation functions in FeedForward or any custom model with list / lambda based one) wouldn't be displayed in model textual summary (by summary I mean print(model)).

I provide wrapper around lambda based implementations of activation functions (non PyTorch ones) and make corresponding change in FeedForward model.

Example result can be seen in test file.

[mklimasz]

I'm not sure what is failing (cannot see details). I guess tests are failing (test that broke on my branch also seems to be broken on master when I ran that locally).

[matt-gardner]

Thanks for the PR! You can see the failures by logging into team city as a guest. The failures all come from tests that currently use non-modules to specify the activation. Looks like it's two one-line fixes, here:

allennlp/allennlp/tests/modules/seq2seq_encoders/compose_encoder_test.py

Line 37 in c497103

input_dim=input_dim, num_layers=1, activations=torch.relu, hidden_dims=output_dim

and here:

allennlp/allennlp/tests/modules/seq2seq_encoders/feedforward_encoder_test.py

Line 12 in c497103

feedforward = FeedForward(input_dim=10, num_layers=1, hidden_dims=10, activations="linear")

While you're at it, can you change the name of this class to be TestComposeEncoder?

allennlp/allennlp/tests/modules/seq2seq_encoders/compose_encoder_test.py

Line 42 in c497103

class TestPassThroughEncoder(AllenNlpTestCase):

You'll also need to run black to pass some of our code formatting CI checks.

[matt-gardner]

Also, can you update the docstring to reflect that the input must be a module, not just a function, and give an example; e.g., torch.nn.ReLU instead of torch.relu.

[bryant1410]

Oh, wow, nice! I opened #4101 and I just see this. Nice that someone's working on it. DIfferent motivation but same patch!

[bryant1410]

LGTM

[mklimasz]

Hey guys, can we get this merged?