AI is becoming state-of-the-art across scientific fields, giving novel solutions to age-old problems. In genomic prediction, Machine Learning methods could not outperform linear regressions in a general way yet, but are becoming closer. An important feature when working with genomic data, which is non other than a long sequence of information, is to account for the linkage disequilibrium, i.e. dependencies between genome variations that do not need to be close in the genome, and variate with respect to the reference genome. To explode this feature, we evaluate Transformers, known for their great performance with long sequences. We worked with two databases: the first one composed of Yeast SNPs seeking to predict the growth of each individual in two different environments and the second one composed of Wheat SNPs seeking to predict four phenotypes. We compare the results with different linear models (BRR, BayesA, BayesB, BayesC and BayesL) typically used for genomic prediction and also with XGBoost, commonly known to have well performance in the area. We conclude that Transformers have shown to be a competitive model for genomic prediction, even tho it does not achieve the state-of-the-art yet.