If you choose this second option, there are three possibilities you can use to gather all the input Tensors Nevertheless, in the vocabulary size growth in RoBERTa allows to encode almost any word or subword without using the unknown token, compared to BERT. This gives a considerable advantage to