Abstract

Sarcasm represents a sophisticated linguistic phenomenon, frequently encountered on contemporary social media platforms. Multi-modal sarcasm detection aims to ascertain whether a given sample, comprising both text and image, exhibits sarcastic intent. Despite commendable achievements by existing methods, existing studies on multimodal sarcasm detection heavily rely on textual modality and unavoidably induce the spurious correlations between textual words and labels. This greatly hinders the model generalization ability. To address this problem, we define the task of out-of-distribution (OOD) multimodal sarcasm detection. This task aims to estimate and mitigate the bad effect of textual modality for strong OOD generalization. To achieve this task, we propose a novel de-biasing multimodal sarcasm detection method with contrastive learning. In particular, we first design counterfactual data augmentation to construct the positive samples with dissimilar bias and negative samples with similar bias. Then, we devise an adapted debiasing contrastive learning mechanism to make model learn the robust task-relevant feature and remove the effect of the bias feature based augmented samples. Through extensive experiments on a widely-utilized dataset, we demonstrate the superiority of our framework over state-of-the-art approache.