HyperAIHyperAI

Command Palette

Search for a command to run...

3 months ago

Multi-task Sentence Encoding Model for Semantic Retrieval in Question Answering Systems

Qiang Huang Jianhui Bu Weijian Xie Shengwen Yang Weijia Wu Liping Liu

Multi-task Sentence Encoding Model for Semantic Retrieval in Question Answering Systems

Abstract

Question Answering (QA) systems are used to provide proper responses to users' questions automatically. Sentence matching is an essential task in the QA systems and is usually reformulated as a Paraphrase Identification (PI) problem. Given a question, the aim of the task is to find the most similar question from a QA knowledge base. In this paper, we propose a Multi-task Sentence Encoding Model (MSEM) for the PI problem, wherein a connected graph is employed to depict the relation between sentences, and a multi-task learning model is applied to address both the sentence matching and sentence intent classification problem. In addition, we implement a general semantic retrieval framework that combines our proposed model and the Approximate Nearest Neighbor (ANN) technology, which enables us to find the most similar question from all available candidates very quickly during online serving. The experiments show the superiority of our proposed method as compared with the existing sentence matching models.

Benchmarks

BenchmarkMethodologyMetrics
paraphrase-identification-on-quora-questionMSEM
Accuracy: 88.86

Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing
Get Started

Hyper Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp
Multi-task Sentence Encoding Model for Semantic Retrieval in Question Answering Systems | Papers | HyperAI