Knowledge Acquisition and Dialogue
Research Team

Research Summary

Communication through language is an essential tool for robots to cooperate with users in daily lives. This research team focuses on modeling intention estimation, thinking, and inference on verbal and non-verbal communications. We are working on the following topics: action management of dialogue robots, spoken dialogue related technologies modeling thinking, inference, and topic selection of robots knowledge expressions grounded on robot behaviors and dialogue contents incorporating emotion and language information empathy, emotion, and trust in dialogue.

Main Research Fields
  • Natural language processing
  • Spoken language processing
  • Intelligence information processing
  • Spoken dialogue systems
  • Multimodal information processing
  • Language understanding
  • Language generation
  • Embodied dialogue
  • Robot intention
  • Dialogue policy learning
Research theme
  • Environment understanding from verbal and non-verbal information
  • Inference on knowledge
  • Memory representation of experiments and knowledges
  • Action decision
  • Dialogue response generation
  • Robot action generation

Koichiro Yoshino

Koichiro Yoshino


Bachelor of Arts, Keio University
Master of Informatics, Kyoto University
Ph.D. Informatics, Kyoto University
JSPS Research Fellow (PD), ACCMS, Kyoto University
Project Assistant Professor, Graduate School of Information Science, NAIST
Assistant Professor, Graduate School off Information Science, NAIST
PRESTO Researcher, JST
Team Leader, GRP, RIKEN

He is a Visiting Researcher at AIP, RIKEN from 2017. From 2019 to 2020, he was a visitor researcher of HHU Düsseldorf, Germany.


SIG Research Award, The Japanese Society for Artificial Intelligence
Outstanding Journal Paper Award, The Association for Natural Language Processing
Best Paper Award, ACL2019 NLP for Conversational AI Workshop
Outstanding Paper Award, The Association for Natural Language Processing
SIG Research Award, The Japanese Society for Artificial Intelligence
Best Paper Award, IWSDS2020


Research Scientist
Seiya Kawano
Research Scientist
Angel Garcia contreras
Postdoctoral Researcher
Hideko Habe
Technical Staff I
Akishige Yuguchi
Visiting Scientist
Shun Inazumi
Junior Research Associate and Student Trainee
Kazuyo Onishi
Junior Research Associate and Student Trainee
Kai Yoshida
Research Part-time Worker I
Lee Sangmyeong
Research Part-time Worker I and Student Trainee
Yoko Matsui
Research Part-time Worker II
Hyuga Nakaguro
Research Part-time Worker II and Student Trainee
Takahiro Hiura
Research Part-time Worker II and Student Trainee
Takuma miwa
Research Part-time Worker II and Student Trainee
Mao Yamaguchi
Administrative Part-time Worker II
Kazuma Fujita
Student Trainee
Muhammad Yeza Baihaqi
Student Trainee
Nakaso yoshihiro
Student Trainee

Former member

Kana Miyamoto
Junior Research Associate(2022/04-2022-05)
Shohei Tanaka
Junior Research Associate(2021/04-2023/03)
Seitaro Shinagawa
Visiting Scientist(2022/5~2024/4/30)
Muteki Arioka
Research Part-time Worker I(2020/08-2022/03)
Daichi Yoshihara
Research Part-time Worker II and Student Trainee(2022/05-2023/03)
Konosuke Yamasaki
Research Part-time Worker II and Student Trainee(2022/06-2024/03)
Kanta Watanabe
Research Part-time Worker II and Student Trainee(2022/06-2024/03)
Liu Qianying
Student Trainee(2021/04-2022/03)
Taiki Nakamura
Student Trainee(2021/08-2023/03)
Yasutaka Odo
Student Trainee(2022/08-2022/09)
Kenta Yamamoto
Student Trainee(2021/08-2023/03)
Nobuhiro Ueda
Student Trainee(2021/08-2024/03)
Shota Kanezaki
Student Trainee(2021/12-2024/03)
Ryuichi Sumida
Student Trainee(2023/8-2024/03)
Kataoka Shotaro
Student Trainee(2023/9-2024/03)
Mizoguchi Tsukito
Student Trainee(2023/9-2024/03)
Shang-Chi Tsai
Student Trainee(2023/11-2024/03)
Rino Naka
Research Intern(2021/08-2021/09)
Issei Matsumoto
Research Intern(2022/9)
Wen-Yu Chang
Research Intern(2023/7-9)

Research results

Explanation by the robot itself

We have developed a framework that allows a robot to explain what it has done in natural language. The robot can understand the actions it has taken and explain them in language using the trajectory of the robot's motors and first-person images. This technology improves the explainability of the robot's actions, and even when an action fails, the robot can explain what it tried to do and what happened as a result.

Explanation by the robot itself

Dialogue system that makes witty recommendations.

Dialogue systems need to take appropriate actions not only for clear user requests such as “Please launch the camera app” but also for ambiguous and vague ones such as “I love the view here.” We built such a reflective dialogue system by collecting a corpus that includes pairs of ambiguous requests and corresponding reflective actions. A proposed Positive/Unlabeled (PU) learning method achieved higher classification performance than a conventional PU learning method.

Dialogue system that makes witty recommendations.

Selected Publications

  1. Shohei Tanaka, Koichiro Yoshino, Katsuhito Sudoh, Satoshi Nakamura.
    "Reflective action selection based on positive-unlabeled learning and causality detection model."
    Computer Speech & Language, 101463. 2023.
  2. Seiya Kawano, Muteki Arioka, Akishige Yuguchi, Kenta Yamamoto, Koji Inoue, Tatsuya Kawahara, Satoshi Nakamura and Koichiro Yoshino.
    "Multimodal Persuasive Dialogue Corpus using Teleoperated Android."
    In Proceedings of INTERSPEECH2022, pp.2308-2312, Incheon, Korea, September 2022.
  3. Yuya Nakano, Seiya Kawano, Koichiro Yoshino, Katsuhito Sudoh and Satoshi Nakamura.
    "Pseudo Ambiguous and Clarifying Questions Based on Sentence Structures Toward Clarifying Question Answering System."
    In Proceedings of Annual Meeting of the Association for Computational Linguistics DialDoc Workshop, Dublin, Ireland, May 2022.
  4. Akishige Yuguchi, Seiya Kawano, Koichiro Yoshino, Carlos Ishi, Yasutomo Kawanishi, Yutaka Nakamura, Takashi Minato, Yasuki Saito and Michihiko Minoh.
    "Butsukusa: A Conversational Mobile Robot Describing Its Own Observations and Internal States."
    In Proceedings of 17th ACM/IEEE International Conference on Human-Robot Interaction, 2022.
  5. Koichiro Yoshino.
    "Embodied Dialogue by Autonomous Robot."
    The third workshop of natural language processing for conversational AI (NLP4ConvAI) at the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP2021); Invited Talk, 2021.
  6. Sara Asai, Seitaro Shinagawa, Koichiro Yoshino, Sakriani Sakti, Satoshi Nakamura.
    "Eliciting Cooperative Persuasive Dialogue by Multimodal Emotional Robot."
    International Workshop on Spoken Dialogue Systems Technology (IWSDS) 2021.
  7. Seiya Kawano, Koichiro Yoshino and Satoshi Nakamura.
    "Controlled Neural Response Generation by Given Dialogue Acts Based on Label-aware Adversarial Learning"
    Transaction of The Japanese Society for Artificial Intelligence,Vol.36, No.4, July 2021.
  8. Shohei Tanaka, Koichiro Yoshino, Katsuhito Sudoh and Satoshi Nakamura.
    "ARTA: Collection and Classification of Ambiguous Requests and Thoughtful Actions"
    Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL 2021), Singapore, Singapore, July 2021
  9. Seiya Kawano, Koichiro Yoshino, David Traum and Satoshi Nakamura.
    "Dialogue Structure Parsing on Multi-Floor Dialogue Based on Multi-Task Learning"
    In Proceedings of the first ROBOT-DIAL workshop at IJCAI 2020.
  10. Seiya Kawano, Masahiro Mizukami, Koichiro Yoshino and Satoshi Nakamura.
    "Entrainable Neural Conversation Model based on Reinforcement Learning"
    IEEE Access, Vol.8, pp.178283 - 178294, September 2020.
  11. Tung The Nguyen, Koichiro Yoshino, Sakriani Sakti and Satoshi Nakamura.:
    "Policy Reuse for Dialog Management Using Action-Relation Probability"
    IEEE Access, Vol.8, pp.159639-159649, 2020
  12. Luis Fernando D'Haro, Koichiro Yoshino, Chiori Hori, Tim K. Marks, Lazaros Polymenakos, Jonathan K. Kummerfeld, Michel Galley, and Xiang Gao.:
    "Overview of the seventh Dialog System Technology Challenge: DSTC7"
    Computer Speech & Language, Volume 62, 2020
  13. Seitaro Shinagawa, Koichiro Yoshino, Seyed Hossein Alavi, Kallirroi Georgila, David Traum, Sakriani Sakti, and Satoshi Nakamura.:
    "An Interactive Image Editing System using an Uncertainty-based Confirmation Strategy"
    IEEE Access, Vol.8, pp.98471 - 98480, 2020
  14. The Tung Nguyen, Koichiro Yoshino, Sakriani Sakti, Satoshi Nakamura.:
    "Dialog Management of Healthcare Consulting System by Utilizing Deceptive Information"
    Transactions of the Japanese Society for Artificial Intelligence, Vol.35, No.1, pp.DSI-C_1-12, 2020
  15. Nurul Lubis, Sakti Sakriani, Koichiro Yoshino, Satoshi Nakamura.:
    "Positive Emotion Elicitation in Chat-based Dialogue Systems"
    IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), Vol.27, Issue.4, pp.866--877, April 2019
  16. Koichiro Yoshino, Kana Ikeuchi, Katsuhito Sudoh, Satoshi Nakamura.:
    "Improving Spoken Language Understanding by Wisdom of Crowds"
    In Proceedings of The 28th International Conference on Computational Linguistics (COLING2020), Barcelona (virtual), Spain, December 2020 (accepted).
  17. Koichiro Yoshino, Kohei Wakimoto, Yuta Nishimura and Satoshi Nakamura.:
    "Caption Generation of Robot Behaviors based on Unsupervised Learning of Action Segments"
    In Proceedings of International Workshop on Spoken Dialogue Systems Technology (IWSDS) 2020, Madrid (virtual), Spain, Sep. 2020.
  18. Seiya Kawano, Koichiro Yoshino, Satoshi Nakamura.:
    "Neural Conversation Model Controllable by Given Dialogue Act Based on Adversarial Learning and Label-aware Objective"
    The 12th International Conference on Natural Language Generation (INLG2019), Tokyo, Japan, October 2019.
  19. Shohei Tanaka, Koichiro Yoshino, Katsuhito Sudoh, Satoshi Nakamura.:
    "Conversational Response Re-ranking Based on Event Causality and Role Factored Tensor Event Embedding"
    The 1st workshop on NLP for ConvAI workshop in ACL conference, Florence, Italy, July 2019


Contact Information

koichiro.yoshino [at]