Different variants of the XCiT: Cross-Covariance Image Transformers models [1] are now available on TensorFlow Hub including models for off-the-shelf inference as well as feature extractors.
https://tfhub.dev/rishit-dagli/collections/xcit/
References:
[1] Ali, A., Touvron, H., Caron, M., Bojanowski, P., Douze, M., Joulin, A., Laptev, I., Neverova, N., Synnaeve, G., Verbeek, J., and Jegou, H. 2021. XCiT: Cross-Covariance Image Transformers. In Advances in Neural Information Processing Systems (pp. 20014–20027). Curran Associates, Inc…