Knowledge Acquisition and Dialogue
Research Team

Research Summary

Communication through language is an essential tool for robots to cooperate with users in daily lives. This research team focuses on modeling intention estimation, thinking, and inference on verbal and non-verbal communications. We are working on the following topics: action management of dialogue robots, spoken dialogue related technologies modeling thinking, inference, and topic selection of robots knowledge expressions grounded on robot behaviors and dialogue contents incorporating emotion and language information empathy, emotion, and trust in dialogue.

Main Research Fields
  • Natural language processing
  • Spoken language processing
  • Intelligence information processing
Keywords
  • Spoken dialogue systems
  • Multimodal information processing
  • Language understanding
  • Language generation
  • Embodied dialogue
  • Robot intention
  • Dialogue policy learning
Research theme
  • Environment understanding from verbal and non-verbal information
  • Inference on knowledge
  • Memory representation of experiments and knowledges
  • Action decision
  • Dialogue response generation
  • Robot action generation

Koichiro Yoshino

Koichiro Yoshino

History

2009
Bachelor of Arts, Keio University
2011
Master of Informatics, Kyoto University
2014
Ph.D. Informatics, Kyoto University
2014
JSPS Research Fellow (PD), ACCMS, Kyoto University
2015
Project Assistant Professor, Graduate School of Information Science, NAIST
2016
Assistant Professor, Graduate School off Information Science, NAIST
2016
PRESTO Researcher, JST
2020
Team Leader, GRP, RIKEN

He is a Visiting Researcher at AIP, RIKEN from 2017. From 2019 to 2020, he was a visitor researcher of HHU Düsseldorf, Germany.

Award

2013
SIG Research Award, The Japanese Society for Artificial Intelligence
2018
Outstanding Journal Paper Award, The Association for Natural Language Processing
2019
Best Paper Award, ACL2019 NLP for Conversational AI Workshop
2020
Outstanding Paper Award, The Association for Natural Language Processing
2020
SIG Research Award, The Japanese Society for Artificial Intelligence
2020
Best Paper Award, IWSDS2020

Members

Akishige Yuguchi
Postdoctoral Researcher
Seiya Kawano
Postdoctoral Researcher
Hideko Habe
Technical Staff I
Shohei Tanaka
Junior Research Associate
Kana Miyamoto
Junior Research Associate
Takamasa Iio
Visiting Researcher
Seitaro Shinagawa
Visiting Researcher
Daichi Yoshihara
Research Part-time Worker II
Taiki Nakamura
Student Trainee
Shota Kanezaki
Student Trainee
Kenta Yamamoto
Student Trainee
Nobuhiro Ueda
Student Trainee

Former member

Muteki Arioka
Research Part-time Worker I (2020/08-2022/03)
Liu Qianying
Student Trainee (2021/04-2022/03)
Rino Naka
Research Intern (2021/08-2021/09)

Research results

Explanation by the robot itself

We have developed a framework that allows a robot to explain what it has done in natural language. The robot can understand the actions it has taken and explain them in language using the trajectory of the robot's motors and first-person images. This technology improves the explainability of the robot's actions, and even when an action fails, the robot can explain what it tried to do and what happened as a result.

Explanation by the robot itself

Dialogue system that makes witty recommendations.

Dialogue systems need to take appropriate actions not only for clear user requests such as “Please launch the camera app” but also for ambiguous and vague ones such as “I love the view here.” We built such a reflective dialogue system by collecting a corpus that includes pairs of ambiguous requests and corresponding reflective actions. A proposed Positive/Unlabeled (PU) learning method achieved higher classification performance than a conventional PU learning method.

Dialogue system that makes witty recommendations.

Selected Publications

  1. Akishige Yuguchi, Seiya Kawano, Koichiro Yoshino, Carlos Ishi, Yasutomo Kawanishi, Yutaka Nakamura, Takashi Minato, Yasuki Saito and Michihiko Minoh.
    "Butsukusa: A Conversational Mobile Robot Describing Its Own Observations and Internal States."
    In Proceedings of 17th ACM/IEEE International Conference on Human-Robot Interaction, 2022.
  2. Koichiro Yoshino.
    "Embodied Dialogue by Autonomous Robot."
    The third workshop of natural language processing for conversational AI (NLP4ConvAI) at the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP2021); Invited Talk, 2021.
  3. Sara Asai, Seitaro Shinagawa, Koichiro Yoshino, Sakriani Sakti, Satoshi Nakamura.
    "Eliciting Cooperative Persuasive Dialogue by Multimodal Emotional Robot."
    International Workshop on Spoken Dialogue Systems Technology (IWSDS) 2021.
  4. Seiya Kawano, Koichiro Yoshino and Satoshi Nakamura.
    "Controlled Neural Response Generation by Given Dialogue Acts Based on Label-aware Adversarial Learning"
    Transaction of The Japanese Society for Artificial Intelligence,Vol.36, No.4, July 2021.
  5. Shohei Tanaka, Koichiro Yoshino, Katsuhito Sudoh and Satoshi Nakamura.
    "ARTA: Collection and Classification of Ambiguous Requests and Thoughtful Actions"
    Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL 2021), Singapore, Singapore, July 2021
  6. Seiya Kawano, Koichiro Yoshino, David Traum and Satoshi Nakamura.
    "Dialogue Structure Parsing on Multi-Floor Dialogue Based on Multi-Task Learning"
    In Proceedings of the first ROBOT-DIAL workshop at IJCAI 2020.
  7. Seiya Kawano, Masahiro Mizukami, Koichiro Yoshino and Satoshi Nakamura.
    "Entrainable Neural Conversation Model based on Reinforcement Learning"
    IEEE Access, Vol.8, pp.178283 - 178294, September 2020.
  8. Tung The Nguyen, Koichiro Yoshino, Sakriani Sakti and Satoshi Nakamura.:
    "Policy Reuse for Dialog Management Using Action-Relation Probability"
    IEEE Access, Vol.8, pp.159639-159649, 2020
  9. Luis Fernando D'Haro, Koichiro Yoshino, Chiori Hori, Tim K. Marks, Lazaros Polymenakos, Jonathan K. Kummerfeld, Michel Galley, and Xiang Gao.:
    "Overview of the seventh Dialog System Technology Challenge: DSTC7"
    Computer Speech & Language, Volume 62, 2020
  10. Seitaro Shinagawa, Koichiro Yoshino, Seyed Hossein Alavi, Kallirroi Georgila, David Traum, Sakriani Sakti, and Satoshi Nakamura.:
    "An Interactive Image Editing System using an Uncertainty-based Confirmation Strategy"
    IEEE Access, Vol.8, pp.98471 - 98480, 2020
  11. The Tung Nguyen, Koichiro Yoshino, Sakriani Sakti, Satoshi Nakamura.:
    "Dialog Management of Healthcare Consulting System by Utilizing Deceptive Information"
    Transactions of the Japanese Society for Artificial Intelligence, Vol.35, No.1, pp.DSI-C_1-12, 2020
  12. Nurul Lubis, Sakti Sakriani, Koichiro Yoshino, Satoshi Nakamura.:
    "Positive Emotion Elicitation in Chat-based Dialogue Systems"
    IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), Vol.27, Issue.4, pp.866--877, April 2019
  13. Koichiro Yoshino, Kana Ikeuchi, Katsuhito Sudoh, Satoshi Nakamura.:
    "Improving Spoken Language Understanding by Wisdom of Crowds"
    In Proceedings of The 28th International Conference on Computational Linguistics (COLING2020), Barcelona (virtual), Spain, December 2020 (accepted).
  14. Koichiro Yoshino, Kohei Wakimoto, Yuta Nishimura and Satoshi Nakamura.:
    "Caption Generation of Robot Behaviors based on Unsupervised Learning of Action Segments"
    In Proceedings of International Workshop on Spoken Dialogue Systems Technology (IWSDS) 2020, Madrid (virtual), Spain, Sep. 2020.
  15. Seiya Kawano, Koichiro Yoshino, Satoshi Nakamura.:
    "Neural Conversation Model Controllable by Given Dialogue Act Based on Adversarial Learning and Label-aware Objective"
    The 12th International Conference on Natural Language Generation (INLG2019), Tokyo, Japan, October 2019.
  16. Shohei Tanaka, Koichiro Yoshino, Katsuhito Sudoh, Satoshi Nakamura.:
    "Conversational Response Re-ranking Based on Event Causality and Role Factored Tensor Event Embedding"
    The 1st workshop on NLP for ConvAI workshop in ACL conference, Florence, Italy, July 2019

Links

Contact Information

koichiro.yoshino [at] riken.jp