BAN445 Introduction to Chatbots and Dialogue System Design for Business: Ideation, Prototyping and Evaluation (E)

• Topics

  This course teaches essential skills for integrating, supporting, and assessing chatbots and dialogue system applications in business. The ultimate goal of the course is to prepare future managers to lead projects featuring dialogue systems and to collaborate with cross-functional teams of software developers, designers, and quality assurance specialists in this regard. Along with learning the theory of dialogue systems and human-computer interaction, students will participate in practical activities, such as project planning, prototyping sessions, and hands-on exercises for chatbot evaluation and data analytics of user-chatbot interactions.

  The course provides an overview of dialogue system architectures, including rule- and frame-based approaches as well as modern transformer-based solutions. Students will learn how to select appropriate architectures by considering business objectives, domain-specific constraints, potential risks, and organizational resources. The course is particularly relevant for students aspiring to managerial roles in businesses, non-governmental, or governmental organizations where chatbot technologies are implemented. The competences developed in this course are valuable for careers in digital consulting, UX design, and product management.

  The syllabus will contain the following elements:

  1. Introduction. Definitions, types, areas of application, and system components of dialogue systems. Short history of dialogue system development. A brainstorming exercise for group projects.

  2. Understanding conversation structure for dialogue system design. Theory of human-computer interaction, usability and voice. Exercises for writing conversation scenarios and extracting insights from user-chatbot interactions.

  3. Principles and methods of chatbot prototyping. Chatbot integration. Available resources for chatbot creation in English, Norwegian, and other languages. A programming exercise.

  4. Quality assurance and ethics. Safety and privacy. Problems of LLM benchmarking. Regulation and accountability through real-world cases. Business risk assessment. An annotation exercise.

  5. Group presentations. Summary and conclusions.

• Learning outcome

  Knowledge

  Upon successful completion of the course, the students will be able to:

  • characterize the conceptualizations and the major types of chatbots and dialogue systems, including their historical development and current state-of-the-art technologies;
  • explain the core principles in dialogue system design and human-computer interaction;
  • distinguish among the common evaluation methods and metrics for dialogue systems.

  Skills

  Upon completion of the course, the students will be able to:

  • select chatbot designs according to specific business goals;
  • create conversational scenarios using established UX and dialogue design principles;
  • build and configure a simple chatbot prototype;
  • conduct basic evaluation of chatbot usability and effectiveness;
  • identify ethical, social, and organizational challenges and risks associated with different chatbot architectures.

  General competences

  Upon completion of the course, the students will have competences to:

  • assess systems based on natural language processing in business contexts;
  • collaborate effectively in interdisciplinary cross-functional teams.

• Teaching

  The theoretical part of the content will be presented through traditional lectures and reading materials. Students will be able to track their progress through short digital quizzes. Each session will contain at least one element of interactive learning, such as group project brainstorming, discussions of cases, and individual hands-on exercises for prototyping and annotation.

• Recommended prerequisites

  It is recommended, but not required, to have basic understanding of natural language processing. Completing BAN432 would be an ideal preparation.

• Credit reduction due to overlap

  None.

• Compulsory Activity

  There will be two quizzes in class and two quizzes at home. For the in-class quizzes to be approved, only participation is required. For the take-home quizzes to be approved, students have to score at least 50%. All quizzes must be approved in order to undertake final evaluation.

• Assessment

  The assessment consists of two parts:

  • A 10 minute group presentation about dialogue system applications in the areas of students' interests (customer support, EdTech, health sector, smart home devices, etc.). The group project must be prepared in groups of 3-4 students during the teaching week and presented in the last session. (50%)
  • An individual project where the students develop chatbot prototypes with the use of the suggested script templates or low-code solutions. The students will have four weeks to submit the project after the last session. (50%)

  Both parts must be submitted and passed in the same semester.

• Grading Scale

  Pass/Fail

• Computer tools

  Python (any environment for working with Jupyter Notebooks). Students might be asked to create accounts on external services for chatbot prototyping.

• Literature

  Bean, A. M., Kearns, R. O., Romanou, A., Hafner, F. S., Mayne, H., Batzner, J., Foroutan, N., Schmitz, C., Korgul, K., Batra, H., Deb, O., Beharry, E., Emde, C., Foster, T., Gausen, A., Grandury, M., Han, S., Hofmann, V., Ibrahim, L., Kim, H., Kirk, H. R., Lin, F., Liu, G. K.-M., Luettgau, L., Magomere, J., Ryström, J., Sotnikova, A., Yang, Y., Zhao, Y., Bibi, A., Bosselut, A., Clark, R., Cohan, A., Foerster, J. N., Gal, Y., Hale, S. A., Raji, I. D., Summerfield, C., Torr, P., Ududec, C., Rocher, L., & Mahdi, A. (2025). Measuring what matters: Construct validity in large language model benchmarks. In Proceedings of the Thirty-Ninth Annual Conference on Neural Information Processing Systems: Datasets and Benchmarks Track. https://openreview.net/forum?id=mdA5lVvNcU

  Cheng, Zerui & Wohnig, Stella & Gupta, Ruchika & Alam, Samiul & Abdullahi, Tassallah & Ribeiro, João & Nielsen-Garcia, Christian & Mir, Saif & Li, Siran & Orender, Jason & Bahrainian, Seyed & Kirste, Daniel & Gokaslan, Aaron & Glinka, Mikołaj & Eickhoff, Carsten & Viswanath, Pramod & Wolff, Ruben. (2025). Position: Benchmarking is Broken - Don't Let AI be its Own Judge. TechRxiv. 10.36227/techrxiv.175752188.89738992/v1

  Jurafsky, D., & Martin, J. H. (2026). Speech and language processing: An introduction to natural language processing, computational linguistics, and speech recognition, with language models (3rd ed.) [Online manuscript]. https://web.stanford.edu/~jurafsky/slp3/

  McTear, M. (2020). Conversational AI: Dialogue Systems, Conversational Agents, and Chatbots. Synthesis Lectures on Human Language Technologies. Springer Cham. https://doi.org/10.2200/S01060ED1V01Y202010HLT048