Some questions about the code

[Carrotljw]

Hello, does the language model of this paper use transformer? I saw this part in the code, but it seems that I did not see the relevant introduction in the paper.Hope to get your answer, thank you~

[brianhie]

Hi @Carrotljw, there is some code that tries to implement a transformer, but the main model that we use is a BiLSTM.

[Carrotljw]

In each training, one of the amino acids in the sequence will be removed, and then the front part and the back part will be passed into the same LSTM to predict what the removed amino acid is. Could I understand it this way?

[brianhie]

Yup, that is right!

[Carrotljw]

Thank you for your patience to answer~ Your work understands the evolution of the virus from a new perspective. it’s very meaningful and interesting. I hope I could have the opportunity to discuss it with you in the future. 😊

[Carrotljw]

Sorry I have new questions, I don't know how semantic embedding is obtained. I read this part of the code but didn't understand it.
My puzzle: Is it to input the original sequence and the mutated sequence into the trained model and get their respective semantic embeddings?

• If so, these sequences are entered together with mutation sites, but the mutation sites are removed during model training, so how do I enter into the model to get semantic embedding😭
• Could you briefly explain the structure of the code？🤓

[brianhie]

Please see this function for computing semantic embeddings:

viral-mutation/bin/mutation.py

Line 183 in 27dcc5a

def embed_seqs(args, model, seqs, vocabulary,

. We use the output of the last hidden layer of the BiLSTM as the embeding:

viral-mutation/bin/language_model.py

Line 71 in 27dcc5a

def transform(self, X_cat, lengths, embed_fname=None):