refine test_understand_sentiment_lstm #5781
Conversation
| data = [(x[0][:chop_len], x[1]) for x in data if len(x[0]) >= chop_len] | ||
|
|
||
| return data[:batch_len] | ||
| return data[:batch_size] |
There was a problem hiding this comment.
static RNN requires all the data points within a batch have the same sequence length. In this chop_data, we make all the data to have length 80. However, some are less than 80, so we simply abandoned them.
This chop_data should only be used in the sanity check.
There was a problem hiding this comment.
Do you mean that batch_size and batch_len have different means? if so, why does lstm_net use batch_size?
There was a problem hiding this comment.
batch_size and batch_len are the same thing. After looking into the revised code more, I think it does fix #5780. I am approving this.
@Canpio Please confirm that data reader is reading 100 data points per batch, and we are preprocessing each batch into 50 data points per batch and each data point is of length 80.
fixes #5780