The model is not pretrained Due to lack of resources maybe someday.
A reproduction of Bert model using Pytorch .