Different variants of the XCiT: Cross-Covariance Image Transformers models  are now available on TensorFlow Hub including models for off-the-shelf inference as well as feature extractors.
 Ali, A., Touvron, H., Caron, M., Bojanowski, P., Douze, M., Joulin, A., Laptev, I., Neverova, N., Synnaeve, G., Verbeek, J., and Jegou, H. 2021. XCiT: Cross-Covariance Image Transformers. In Advances in Neural Information Processing Systems (pp. 20014–20027). Curran Associates, Inc…