Title: "Aspect-Aware Response Generation for Multimodal Dialogue System"
a) This work is accepted in ACM Transactions on Intelligent Systems and Technology (TIST), 12(2), 1-33.
b) This dataset is intended only for non-commercial, educational and/or research purposes only.
c) For access to the datasets and any associated queries, please reach us at
iitpainlpmlresourcerequest@gmail.comd) This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License.
e) The dataset is allowed to be used in any publication, only upon citation.
BibTex:
@article{10.1145/3430752,
author = {Firdaus, Mauajama and Thakur, Nidhi and Ekbal, Asif},
title = {Aspect-Aware Response Generation for Multimodal Dialogue System},
year = {2021},
issue_date = {April 2021},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
volume = {12},
number = {2},
issn = {2157-6904},
doi = {10.1145/3430752},
abstract = {Multimodality in dialogue systems has opened up new frontiers for the creation of robust conversational agents. Any multimodal system aims at bridging the gap between language and vision by leveraging diverse and often complementary information from image, audio, and video, as well as text. For every task-oriented dialog system, different aspects of the product or service are crucial for satisfying the user’s demands. Based upon the aspect, the user decides upon selecting the product or service. The ability to generate responses with the specified aspects in a goal-oriented dialogue setup facilitates user satisfaction by fulfilling the user’s goals. Therefore, in our current work, we propose the task of aspect controlled response generation in a multimodal task-oriented dialog system. We employ a multimodal hierarchical memory network for generating responses that utilize information from both text and images. As there was no readily available data for building such multimodal systems, we create a Multi-Domain Multi-Modal Dialog (MDMMD++) dataset. The dataset comprises the conversations having both text and images belonging to the four different domains, such as hotels, restaurants, electronics, and furniture. Quantitative and qualitative analysis on the newly created MDMMD++ dataset shows that the proposed methodology outperforms the baseline models for the proposed task of aspect controlled response generation.},
journal = {ACM Trans. Intell. Syst. Technol.},
month = {feb},
articleno = {15},
numpages = {33},
keywords = {Multimodal dialogue system, memory network, response generation}
}
