HyperAIHyperAI

Command Palette

Search for a command to run...

3 months ago

Hostility Detection in Hindi leveraging Pre-Trained Language Models

Ojasv Kamal Adarsh Kumar Tejas Vaidhya

Hostility Detection in Hindi leveraging Pre-Trained Language Models

Abstract

Hostile content on social platforms is ever increasing. This has led to the need for proper detection of hostile posts so that appropriate action can be taken to tackle them. Though a lot of work has been done recently in the English Language to solve the problem of hostile content online, similar works in Indian Languages are quite hard to find. This paper presents a transfer learning based approach to classify social media (i.e Twitter, Facebook, etc.) posts in Hindi Devanagari script as Hostile or Non-Hostile. Hostile posts are further analyzed to determine if they are Hateful, Fake, Defamation, and Offensive. This paper harnesses attention based pre-trained models fine-tuned on Hindi data with Hostile-Non hostile task as Auxiliary and fusing its features for further sub-tasks classification. Through this approach, we establish a robust and consistent model without any ensembling or complex pre-processing. We have presented the results from our approach in CONSTRAINT-2021 Shared Task on hostile post detection where our model performs extremely well with 3rd runner up in terms of Weighted Fine-Grained F1 Score.

Code Repositories

kamalojasv181/Hostility-Detection-in-Hindi-Posts
Official
pytorch
Mentioned in GitHub

Benchmarks

BenchmarkMethodologyMetrics
fake-news-detection-on-hostility-detectionAuxiliary IndicBert
F1 score: 0.7741
hate-speech-detection-on-hostility-detectionAuxiliary IndicBert
F1 score: 0.5725

Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing
Get Started

Hyper Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp