Emotional dialog generation via multiple classifiers based on a generative adversarial network
School of Computer and Information, Hefei University of Technology, Hefei 230601, China
Abstract
Keywords: Emotional dialog generation ; Sequence-to-sequence model ; Emotion classification ; Generative adversarial networks ; Multiple classifiers
Content



Algorithm 1 Adversarial training of the model |
---|
Input: Real dialog (the emotion category of target sentence Y is e):R{X, Y} Output: Trained Dialog Generator: 1: Initialize 2: Pre-train 3: Generate Fake dialogs F{X, Y} using 4: Pre-train 5: while model not converges do 6: for each generative step do 7: Generate fake dialog (F) using 8: Calculate penalty 9: Update 10: end 11: for each discriminative step do 12: Generate fake dialog (F) using 13: Update 14: end 15: end 16: return |
Algorithm 2 Emotion discriminative model training |
---|
Input: Real dialog (R) with emotion category e: R Fake dialog (F) with distinct emotion category: F{ Output: Trained Emotional Discriminator: 1: Initialize 2: while model not converges do 3: for each emotional discriminative step do 4: Update 5: end 6: end 7: return |
Algorithm 3 Calculate fluency score |
---|
Input: corpus, Sentence to be evaluated; Output: the fluency score of the input sentence; 1: Count the number of Bi-gram of dialog corpus 2: Bi-gram count dict 3: Count the number of Tri-gram of dialog corpus. 4: Tri-gram count dict 5: Calculate Tri-gram transfer probability by Eq. 12 6: Tri-gram transition probability dict 7: Sort the 8: The size of sorted probability list is: 9: 10: 11: 12: 13: if 14: return 15: for 16:if 17: 18: else if 19: 20:end 21: 22: return |

Target Emotion | |||||||
---|---|---|---|---|---|---|---|
Dataset | Source Emotion | Other | Liking | Sadness | Disgust | Anger | Happiness |
NLPW | Other | 0.419 | 0.271 | 0.290 | 0.301 | 0.291 | 0.276 |
Liking | 0.187 | 0.381 | 0.182 | 0.180 | 0.159 | 0.276 | |
Sadness | 0.095 | 0.083 | 0.212 | 0.098 | 0.111 | 0.099 | |
Disgust | 0.149 | 0.110 | 0.145 | 0.259 | 0.199 | 0.120 | |
Anger | 0.065 | 0.040 | 0.064 | 0.080 | 0.147 | 0.057 | |
Happiness | 0.085 | 0.115 | 0.108 | 0.082 | 0.094 | 0.171 | |
XHJ | Other | 0.450 | 0.393 | 0.377 | 0.388 | 0.374 | 0.382 |
Liking | 0.132 | 0.227 | 0.119 | 0.124 | 0.109 | 0.144 | |
Sadness | 0.087 | 0.079 | 0.172 | 0.080 | 0.072 | 0.106 | |
Disgust | 0.173 | 0.155 | 0.170 | 0.246 | 0.183 | 0.160 | |
Anger | 0.102 | 0.091 | 0.109 | 0.110 | 0.212 | 0.101 | |
Happiness | 0.056 | 0.053 | 0.053 | 0.052 | 0.049 | 0.108 |
Emotion Accuracy | |||||||
---|---|---|---|---|---|---|---|
Dataset | Model | Other | Liking | Sadness | Disgust | Anger | Happiness |
NLPW | Seq2Seq | 0.286 | 0.121 | 0.089 | 0.128 | 0.212 | 0.191 |
EHMCG | 0.421 | 0.354 | 0.374 | 0.289 | 0.211 | 0.195 | |
EM-SeqGAN | 0.572 | 0.487 | 0.548 | 0.376 | 0.295 | 0.201 | |
EMC-GAN | 0.740 | 0.687 | 0.723 | 0.588 | 0.392 | 0.236 | |
XHJ | Seq2Seq | 0.293 | 0.176 | 0.064 | 0.227 | 0.177 | 0.094 |
EHMCG | 0.563 | 0.379 | 0.374 | 0.458 | 0.385 | 0.295 | |
EM-SeqGAN | 0.647 | 0.563 | 0.487 | 0.567 | 0.426 | 0.375 | |
EMC-GAN | 0.870 | 0.794 | 0.761 | 0.732 | 0.765 | 0.701 |
Option | Very Good | Good | Normal | Bad | Very Bad |
---|---|---|---|---|---|
Score | 5 | 4 | 3 | 2 | 1 |
Coherence Evaluate Scores | |||||||
---|---|---|---|---|---|---|---|
Dataset | Model | Other | Liking | Sadness | Disgust | Anger | Happiness |
NLPW | Seq2Seq | 1.306 | 1.067 | 1.451 | 1.085 | 1.181 | 1.051 |
EHMCG | 1.403 | 1.192 | 1.387 | 1.236 | 1.335 | 1.096 | |
EM-SeqGAN | 1.732 | 1.563 | 1.734 | 1.522 | 1.403 | 1.225 | |
EMC-GAN | 2.277 | 2.875 | 2.115 | 1.881 | 1.972 | 1.245 | |
XHJ | Seq2Seq | 1.127 | 1.361 | 1.229 | 1.111 | 1.147 | 1.263 |
EHMCG | 1.256 | 1.452 | 1.248 | 1.324 | 1.223 | 1.371 | |
EM-SeqGAN | 1.820 | 1.726 | 1.339 | 1.514 | 1.330 | 1.207 | |
EMC-GAN | 3.407 | 2.542 | 3.180 | 1.931 | 1.561 | 2.255 |
Fluency Score | |||||||
---|---|---|---|---|---|---|---|
Dataset | Model | Other | Liking | Sadness | Disgust | Anger | Happiness |
NLPW | Seq2Seq | -0.189 | -0.193 | -0.192 | -0.193 | -0.192 | -0.194 |
EHMCG | 0.405 | 0.727 | 1.133 | 0.526 | 1.238 | 0.943 | |
EM-SeqGAN | 0.586 | 0.875 | 1.875 | 1.034 | 1.337 | 1.237 | |
EMC-GAN | 0.854 | 1.710 | 2.617 | 1.512 | 2.498 | 1.706 | |
XHJ | Seq2Seq | -0.124 | -0.123 | -0.124 | -0.125 | -0.124 | -0.123 |
EHMCG | 3.356 | 5.528 | 7.526 | 6.776 | 5.882 | 4.581 | |
EM-SeqGAN | 4.652 | 7.238 | 8.832 | 7.774 | 6.237 | 6.735 | |
EMC-GAN | 6.300 | 9.239 | 11.33 | 10.33 | 10.26 | 11.26 |
Fluency Evaluate Score | |||||||
---|---|---|---|---|---|---|---|
Dataset | Model | Other | Liking | Sadness | Disgust | Anger | Happiness |
NLPW | Seq2Seq | 1.193 | 1.267 | 1.251 | 1.202 | 1.114 | 1.351 |
EHMCG | 1.253 | 1.196 | 1.325 | 1.269 | 1.156 | 1.183 | |
EM-SeqGAN | 2.013 | 2.162 | 2.067 | 1.849 | 1.758 | 1.657 | |
EMC-GAN | 2.424 | 2.875 | 2.365 | 2.476 | 2.272 | 1.984 | |
XHJ | Seq2Seq | 1.287 | 1.111 | 1.567 | 1.311 | 1.265 | 1.187 |
EHMCG | 1.257 | 1.284 | 1.732 | 1.455 | 1.325 | 1.173 | |
EM-SeqGAN | 2.471 | 2.648 | 2.741 | 1.846 | 1.775 | 2.659 | |
EMC-GAN | 3.717 | 3.760 | 4.480 | 2.835 | 2.960 | 3.326 |
Model | Post(translate) | Emotion | Response(translate) |
---|---|---|---|
Seq2Seq |
我命运多坎坷啊你都预见不到! How rough my fate has been, you never saw it coming! |
Anger |
关我什么事? What is it to me? |
懒得理你。 Talk to the hand. |
Disgust |
切~,我是傻。 Cut ~, I am stupid. |
|
你吹牛。 You brag. |
Happiness |
就是啊、已经注意到了。 Yeah, I've noticed. |
|
我也要参加。 I'll be there, too. |
Liking |
谢谢您的关注!祝您周末愉快。 Thank you for your attention!Have a nice weekend. |
|
牙齿不痛了? No pain in your tooth? |
Sadness |
我爬山好痛苦... It was painful climbing the mountain... |
|
来的话,如何找你? How can I get to you if I come? |
Other |
星期六晚。 Saturday night. |
|
EMC-GAN |
不是我吓你,你晚上真的会做噩梦,小狗会变成鬼找你。 It’s not that I scare you, you really have nightmares at night, and the puppy will turn into a ghost to find you. |
Anger |
吓到我了! Scared me! |
青岛,一座来过就不想再来的城市! Qingdao, a city that has been here but never wants to come again! |
Disgust |
你怎么这么说话呢?赤裸裸的羡慕嫉妒。 Why are you talking like this? Naked envy and jealousy. |
|
又是第一名。 Number one again. |
Happiness |
感觉心情不错很好。 I feel in a good mood. |
|
我爱你。 I love you. |
Liking |
真的啊~,超级无敌帅。 Really, super invincible handsome. |
|
我不是故意的亲爱的。 I didn't mean to, honey. |
Sadness |
太伤心了… It's so sad... |
|
你是机器人嘛?回复这么快。 Are you a robot? Reply so quickly. |
Other |
我又不是,我怎么知道。 I am not, how do I know. |
Reference
Young T, Cambria E, Chaturvedi I, Zhou H, Biswas S, Huang M. Augmenting end-to-end dialogue systems with commonsense knowledge. Thirty-Second AAAI Conference on Artificial Intelligence. New Orleans, Louisiana, USA, 2018, 4970–4977
Young T, Pandelea V, Poria S, Cambria E. Dialogue systems with audio context. Neurocomputing, 2020, 388: 102–109 DOI:10.1016/j.neucom.2019.12.126
Xu H T, Peng H Y, Xie H R, Cambria E, Zhou L Y, Zheng W G. End-to-End latent-variable task-oriented dialogue system with exact log-likelihood optimization. World Wide Web, 2020, 23(3): 1989–2002 DOI:10.1007/s11280-019-00688-8
Zhang Z, Liao L Z, Huang M L, Zhu X Y, Chua T S. Neural multimodal belief tracker with adaptive attention for dialogue systems. In: The World Wide Web Conference on-WWW'19. San Francisco, CA, USA, ACM Press, 2019, 2401–2412 DOI:10.1145/3308558.3313598
Ma Y K, Nguyen K L, Xing F Z, Cambria E. A survey on empathetic dialogue systems. Information Fusion, 2020 DOI:10.1016/j.inffus.2020.06.011
Paschke A, Boley H. Rule responder: rule-based agents for the semantic-pragmatic web. International Journal on Artificial Intelligence Tools, 2011, 20(6): 1043–1081 DOI:10.1142/s0218213011000528
Xu M H, Li P J, Yang H R, Ren P J, Ren Z C, Chen Z M, Ma J. A neural topical expansion framework for unstructured persona-oriented dialogue generation. 2020
Cho K, van Merrienboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y. Learning phrase representations using RNN encoder–decoder for statistical machine translation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, Qatar, Association for Computational Linguistics, 2014, 1724–1734 DOI:10.3115/v1/d14-1179
Sutskever I, Vinyals O, Le Q V. Sequence to sequence learning with neural networks. 2014
Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. 2014
Vinyals O, Le Q. A neural conversational model. 2015
Goodfellow I J, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial networks. 2014
Yu L, Zhang W, Wang J, Yu Y. SeqGAN: sequence generative adversarial nets with policy gradient. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. San Francisco, California, AAAI, 2017, 2852-2858
Kaelbling L P, Littman M L, Moore A W. Reinforcement learning: a survey. Journal of Artificial Intelligence Research, 1996, 4(1): 237–285 DOI:10.1613/jair.301
Sutton R S, McAllester D A, Singh S P, Mansour Y. Policy gradient methods for reinforcement learning with function approximation. In: Advances in Neural Information Processing Systems 12, NIPS Conference. Denver, Colorado, USA, 1999, 1057–1063
Li J W, Monroe W, Shi T L, Jean S, Ritter A, Jurafsky D. Adversarial learning for neural dialogue generation. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen, Denmark, Association for Computational Linguistics, 2017, 2157–2169 DOI:10.18653/v1/d17-1230
Cui S B, Lian R Z, Jiang D, Song Y F, Bao S Q, Jiang Y. DAL: dual adversarial learning for dialogue generation. 2019
Salovey P, Mayer J D. Emotional intelligence. Imagination, Cognition and Personality, 1990, 9(3): 185–211 DOI:10.2190/dugg-p24e-52wk-6cdg
Ghosh S, Chollet M, Laksana E, Morency L P, Scherer S. Affect-LM: a neural language model for customizable affective text generation. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). PA, USA, Association for Computational Linguistics, 2017, 634–642 DOI:10.18653/v1/p17-1059
Rashkin H, Smith E M, Li M, Boureau Y L. Towards empathetic open-domain conversation models: a new benchmark and dataset. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy, Association for Computational Linguistics, 2019 DOI:10.18653/v1/p19-1534
Zhou H, Huang M L, Zhang T Y, Zhu X Y, Liu B. Emotional chatting machine: emotional conversation generation with internal and external memory. 2017
Wang K, Wan X. SentiGAN: Generating sentimental texts via mixture adversarial networks. In: Twenty-Seventh International Joint Conforence on Artificial Intelligence. Stockholm, Sweden, 2018, 4446-4452 DOI:10.24963/ijcai.2018/618
Sun X, Peng X Q, Ding S. Emotional human-machine conversation generation based on long short-term memory. Cognitive Computation, 2018, 10(3): 389–397 DOI:10.1007/s12559-017-9539-4
Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. 2014
Graves A, Mohamed A R, Hinton G. Speech recognition with deep recurrent neural networks. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. Vancouver, BC, Canada, IEEE, 2013, 6645–6649 DOI:10.1109/icassp.2013.6638947
Salimans T, Goodfellow I, Zaremba W, Cheung V, Radford A, Chen X. Improved techniques for training GANs. Advances In Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016. Barcelona, Spain, 2016, 2226–2234
Lai S, Xu L, Liu K, Zhao, J. Recurrent convolutional neural networks for text classification. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence. Austin, Texas, AAAI, 2015, 2267–2273
Srivastava R K, Greff K, Schmidhuber J. Highway networks. 2015
Zhang X, LeCun Y. Text Understanding from Scratch. 2015
Liu D. Approaches to Chinese word analysis; utterance segmentation and automatic evaluation of machine translation. Beijing: Chinese Academy of Sciences, 2004
Doddington G. Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In: Proceedings of the second international conference on Human Language Technology Research-. San Diego, California, Morristown, NJ, USA, Association for Computational Linguistics, 2002, 138–145 DOI:10.3115/1289189.1289273
Zhang H P, Yu H K, Xiong D Y, Liu Q. HHMM-based Chinese lexical analyzer ICTCLAS. In: Proceedings of the second SIGHAN workshop on Chinese language processing. NJ, USA, Association for Computational Linguistics, 2003 DOI:10.3115/1119250.1119280
Sun X, Chen X M, Pei Z M, Ren F J. Emotional human machine conversation generation based on SeqGAN. In: 2018 First Asian Conference on Affective Computing and Intelligent Interaction (ACII Asia). Beijing, China, IEEE, 2018, 1–6 DOI:10.1109/aciiasia.2018.8470388
Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M, Kudlur M, Levenberg J, Monga R, Moore S, Derek G. Tensorflow: A system for large-scale machine learning. 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2016). GA, USA, 2016, 265–283