HyperAIHyperAI

Command Palette

Search for a command to run...

4 months ago

A Large Self-Annotated Corpus for Sarcasm

Mikhail Khodak; Nikunj Saunshi; Kiran Vodrahalli

A Large Self-Annotated Corpus for Sarcasm

Abstract

We introduce the Self-Annotated Reddit Corpus (SARC), a large corpus for sarcasm research and for training and evaluating systems for sarcasm detection. The corpus has 1.3 million sarcastic statements -- 10 times more than any previous dataset -- and many times more instances of non-sarcastic statements, allowing for learning in both balanced and unbalanced label regimes. Each statement is furthermore self-annotated -- sarcasm is labeled by the author, not an independent annotator -- and provided with user, topic, and conversation context. We evaluate the corpus for accuracy, construct benchmarks for sarcasm detection, and evaluate baseline methods.

Code Repositories

Benchmarks

BenchmarkMethodologyMetrics
sarcasm-detection-on-sarc-all-balBag-of-Bigrams
Accuracy: 75.8
sarcasm-detection-on-sarc-pol-balBag-of-Bigrams
Accuracy: 76.5
sarcasm-detection-on-sarc-pol-unbalBag-of-Words
Avg F1: 27.0

Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing
Get Started

Hyper Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp