HyperAIHyperAI

Command Palette

Search for a command to run...

3 months ago

Fusing task-oriented and open-domain dialogues in conversational agents

Tom Young Frank Xing Vlad Pandelea Jinjie Ni Erik Cambria

Fusing task-oriented and open-domain dialogues in conversational agents

Abstract

The goal of building intelligent dialogue systems has largely been separately pursued under two paradigms: task-oriented dialogue (TOD) systems, which perform goal-oriented functions, and open-domain dialogue (ODD) systems, which focus on non-goal-oriented chitchat. The two dialogue modes can potentially be intertwined together seamlessly in the same conversation, as easily done by a friendly human assistant. Such ability is desirable in conversational agents, as the integration makes them more accessible and useful. Our paper addresses this problem of fusing TODs and ODDs in multi-turn dialogues. Based on the popular TOD dataset MultiWOZ, we build a new dataset FusedChat, by rewriting the existing TOD turns and adding new ODD turns. This procedure constructs conversation sessions containing exchanges from both dialogue modes. It features inter-mode contextual dependency, i.e., the dialogue turns from the two modes depend on each other. Rich dependency patterns including co-reference and ellipsis are features. The new dataset, with 60k new human-written ODD turns and 5k re-written TOD turns, offers a benchmark to test a dialogue model's ability to perform inter-mode conversations. This is a more challenging task since the model has to determine the appropriate dialogue mode and generate the response based on the inter-mode context. But such models would better mimic human-level conversation capabilities. We evaluate baseline models on this task, including classification-based two-stage models and two-in-one fused models. We publicly release FusedChat and the baselines to propel future work on inter-mode dialogue systems https://github.com/tomyoung903/FusedChat.

Code Repositories

tomyoung903/fusedchat
Official
pytorch
Mentioned in GitHub

Benchmarks

BenchmarkMethodologyMetrics
dialogue-generation-on-fusedchatTwo-in-one model
BLEU: 12.05
Inform: 70.4
Inform_mct: 90.1
Joint SA: 0.592
PPL: 10.49
SSA: 0.50
Sensibleness: 0.52
Slot Accuracy: 0.972
Specificity: 0.47
Success: 57.0
Success_mct: 72.7
dialogue-generation-on-fusedchatClassification-based model
BLEU: 12.17
Inform: 75.1
Inform_mct: 90.8
Joint SA: 0.600
PPL: 10.50
SSA: 0.55
Sensibleness: 0.58
Slot Accuracy: 0.973
Specificity: 0.51
Success: 60.9
Success_mct: 74.4

Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing
Get Started

Hyper Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp