HyperAIHyperAI

Command Palette

Search for a command to run...

5 months ago

DeliData: A dataset for deliberation in multi-party problem solving

Georgi Karadzhov; Tom Stafford; Andreas Vlachos

DeliData: A dataset for deliberation in multi-party problem solving

Abstract

Group deliberation enables people to collaborate and solve problems, however, it is understudied due to a lack of resources. To this end, we introduce the first publicly available dataset containing collaborative conversations on solving a well-established cognitive task, consisting of 500 group dialogues and 14k utterances. In 64% of these conversations, the group members are able to find a better solution than they had identified individually, and in 43.8% of the groups who had a correct answer as their final solution, none of the participants had solved the task correctly by themselves. Furthermore, we propose a novel annotation schema that captures deliberation cues and release all 14k utterances annotated with it. Finally, we use the proposed dataset to develop and evaluate two methods for generating deliberation utterances. The data collection platform, dataset and annotated corpus are publicly available at https://delibot.xyz.

Benchmarks

BenchmarkMethodologyMetrics
problem-solving-deliberation-on-delidataConversation + Sol. Tracker
AUC: 0.53
problem-solving-deliberation-on-delidataBag-Of-Annotations + Sol. Tracker
AUC: 0.62
problem-solving-deliberation-on-delidataMajority Class Baseline
AUC: 0.50

Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing
Get Started

Hyper Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp
DeliData: A dataset for deliberation in multi-party problem solving | Papers | HyperAI