Here is my implementation of the “Conformer: Convolution-augmented Transformer” paper. It reduces the local inductive bias of transformer and achieves the best of both worlds (transformers for content-based global interactions and CNNs to exploit local features) by combining convolution neural networks and transformers to model both local and global dependencies
1 Like