Sentence 1: Sourav Ganguly is the greatest captain in BCCI.
Sentence 2: Ricky Ponting is the greatest captain in Cricket Australia.
Do these two sentences contradict/entail each other or are they neutral? In NLP, this problem is known as textual entailment and is a part of the GLUE benchmark for language understanding.
On social media platforms, to better curate and moderate content, we often need to utilize multiple sources of data to understand their semantic behavior. This is where multimodal entailment can be useful. In my latest post, I introduce the basics of this topic and present a set of baseline models for the Multimodal Entailment dataset recently introduced by Google. Some recipes include “modality dropout”, cross-attention, and class-imbalance mitigation.
- Blog post: Multimodal entailment
- Code: GitHub - sayakpaul/Multimodal-Entailment-Baseline: This repository shows how to implement a basic model for multimodal entailment.
Fun fact: This marks the 100th example on keras.io.