The Impact of Question Styles on Response Characteristics in Dialogue with Generative AI: A Comparative Analysis of Polite and Direct Communication Patterns
DOI:
https://doi.org/10.51094/jxiv.966Keywords:
Generative AI, question style, prompt engineering, dialogue analysisAbstract
This study employs an experimental methodology to investigate the influence of question style on discourse with generative AI. In particular, we focus on two contrasting question patterns: a polite and considerate communication style (Type A) and a direct and concise communication style (Type D). We then analyze the impact of these on the response characteristics of generative AI. In the experiment, a dialogue was conducted on the theme of technical explanations about image generation AI, targeting four major generative AI models: The four major generative AI models under consideration were ChatGPT, Claude, Command R+, and Gemini.
The analysis demonstrated that the distinction between question styles has a profound influence on the AI response generation process. It was demonstrated that inquiries incorporating consideration expressions elicit comprehensive and systematic responses, whereas direct questions tend to elicit concise and focused responses. It is noteworthy that the influence was demonstrated to extend beyond mere superficial differences in wording to encompass the manner in which information is structured and the learning process itself. Specifically, it was observed that the questions of Type A promoted a context-building information structure and inquiry-based learning patterns, while the questions of Type D promoted a core-supplement structure and focused learning patterns.
These findings have significant implications for the development of effective methods of interaction with generative AI. In particular, they highlight the importance of strategically selecting question styles according to the intended purpose, particularly in the context of AI utilization in educational and learning environments, as well as in the design of AI literacy education in organizations. This study provides fundamental insights that contribute to the qualitative enhancement of human-AI communication in the age of AI.
Conflicts of Interest Disclosure
The authors declare no conflict of interest.Downloads *Displays the aggregated results up to the previous day.
References
Rahimi, F., Talebi Bezmin Abadi, A.: Chatgpt and publication ethics. Archives of Medical Research 54(3), 272-274 (2023) https://doi.org/10.1016/j.arcmed.2023.03.004
Bick, A., Blandin, A., Deming, D.J.: The rapid adoption of generative ai. Working Paper 32966, National Bureau of Economic Research (September 2024). https://doi.org/10.3386/w32966 . http://www.nber.org/papers/w32966
Namoun, A., Ibrahim, I.A., Mustafa, E., Alrehaili, A., more: Generative artificial intelligence in education: an umbrella review of applications and challenges. Research Square (2024) https://doi.org/10.21203/rs.3.rs-8498155/v2 . This is a preprint; it has not been peer-reviewed by a journal
Bozkurt, A.: Tell me your prompts and I will make them true: The alchemy of prompt engineering and generative AI. International Council for Open and Distance Education Oslo, Norway (2024). https://doi.org/10.55982/openpraxis.16.2.661
White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., Schmidt, D.C.: A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT (2023). https://arxiv.org/abs/2302.11382
Chen, B., Zhang, Z., Langrené, N., Zhu, S.: Unleashing the potential of prompt engineering in large language models: a comprehensive review. arXiv preprint arXiv:2310.14735 (2023)
Markowitz, D.M., Boyd, R.L., Blackburn, K.: From silicon to solutions: AI's impending impact on research and discovery. Frontiers in Social Psychology 2 (2024) https://doi.org/10.3389/frsp.2024.1392128
Alafnan, M.A., Dhari, S., Jovic, M., Lomidz, K.: Chatgpt as an educational tool: Opportunities, challenges, and recommendations for communication, business writing, and composition courses. Journal of Artificial Intelligence and Technology 3(2), 60-64 (2023) https://doi.org/10.37965/jait.2023.00184
Degraa, I., Chamari, K., Zmijewski, P., Ben Saad, H.: From human writing to artificial intelligence generated text: examining the prospects and potential threats of chatgpt in academic writing. Biology of Sport 40(2), 615-622 (2023) https://doi.org/10.5114/biolsport.2023.125623
Kestin, G., Miller, K., Kiales, A., Milbourne, T., Ponti, G.: Ai tutoring outperforms active learning. Research Square (2024) https://doi.org/10.21203/rs.3.rs-4243877/v1 . Preprint
Sok, S., Heng, K.: Opportunities, challenges, and strategies for using chatgpt in higher education: A literature review. Journal of Digital Educational Technology 4(1), 2401 (2024) https://doi.org/10.30935/jdet/14027
Cotton, D.R.E., Cotton, P.A., Shipway, J.R.: Chatting and cheating: Ensuring academic integrity in the era of chatgpt. Innovations in Education and Teaching International 61(2), 228-239 (2023) https://doi.org/10.1080/14703297.2023.2190148
Crawford, J., Cowling, M., Allen, K.-A.: Leadership is needed for ethical chatgpt: Character, assessment, and learning using artificial intelligence (ai). Journal of University Teaching & Learning Practice 20(3) (2023) https://doi.org/10.53761/1.20.3.02
Jarrah, A.M., Wardat, Y., Fidalgo, P.: Using chatgpt in academic writing is (not) a form of plagiarism: What does the literature say? Online Journal of Communication and Media Technologies 13(4), 202346 (2023) https://doi.org/10.30935/ojcmt/13572
Karakose, T.: The utility of chatgpt in educational research—potential opportunities and pitfalls. Educational Process: International Journal 12(2), 7-13 (2023) https://doi.org/10.22521/edupij.2023.122.1
Rasul, T., Nair, S., Kalendra, D., Robin, M., Santini, F.d.O., Ladeira, W.J., Sun, M., Day, I., Rather, R.A., Heathcote, L.: The role of chatgpt in higher education: Benefits, challenges, and future research directions. Journal of Applied Learning & Teaching 6(1) (2023) https://doi.org/10.37074/jalt.2023.6.1.29
Elsen-Rooney, M.: NYC Education Department Blocks ChatGPT on School Devices, Networks. Chalkbeat New York. Accessed: 2023-01-04. https://www.chalkbeat.org/2023/1/4/23538888/nyc-schools-blocks-chatgpt-artificial-intelligence
Korzyński, P., Mazurek, G., Krzypkowska, P., Kurasiński, A.: Artificial intelligence prompt engineering as a new digital competence: Analysis of generative ai technologies such as chatgpt. Entrepreneurial Business and Economics Review (2023) https://doi.org/10.15678/eber.2023.110302
Wu, S., Shen, X., Xia, R.: A New Dialogue Response Generation Agent for Large Language Models by Asking Questions to Detect User's Intentions (2023). https://arxiv.org/abs/2310.03293
Tix, B.J.: Follow-Up Questions Improve Documents Generated by Large Language Models (2024). https://arxiv.org/abs/2407.12017
Sabzalieva, E., Valentini, A.: Chatgpt and artificial intelligence in higher education: quick start guide. Technical report, International Institute for Higher Education in Latin America and the Caribbean (IESALC) (2023). United Nations Educational, Scientific and Cultural Organization (UNESCO). Accessed: 2023-04. https://eduq.info/xmlui/handle/11515/38828
Malik, A., Khan, M.L., Hussain, K.: How is chatgpt transforming academia? examining its impact on teaching, research, assessment, and learning. Technical report, Sultan Qaboos University (April 9 2023). https://doi.org/10.2139/ssrn.4413516 . SSRN Electronic Journal, Last revised: November 10, 2024. https://ssrn.com/abstract=4413516
Mizumoto, T., Yamazaki, T., Yoshikawa, K., Ohagi, M., Kawamoto, T., Sato, T.: Dialogue systems can generate appropriate responses without the use of question marks? - a study of the effects of "?" for spoken dialogue systems -. In: Calzolari, N., Kan, M.-Y., Hoste, V., Lenci, A., Sakti, S., Xue, N. (eds.) Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pp. 4858-4864. ELRA and ICCL, Torino, Italia (2024). https://aclanthology.org/2024.lrec-main.435
Li, J.: Teaching Machines to Converse (2020). https://arxiv.org/abs/2001.11701
Noever, D., McKee, F.: Chatbots as Problem Solvers: Playing Twenty Questions with Role Reversals (2023). https://arxiv.org/abs/2301.01743
Oppenlaender, J., Linder, R., Silvennoinen, J.: Prompting AI Art: An Investigation into the Creative Skill of Prompt Engineering (2024). https://arxiv.org/abs/2303.13534
Boussioux, L., Lane, J.N., Zhang, M., Jacimovic, V., Lakhani, K.R.: The crowdless future? generative ai and creative problem-solving. Organization Science 35(5), 1589-1607 (2024) https://doi.org/10.1287/orsc.2023.1843 https://doi.org/10.1287/orsc.2023.1843
Sato, K.: The effect of question formats in dialogue with generative ai: A comparative analysis of open questions and prompt engineering. Jxiv (2024) https://doi.org/10.51094/jxiv.853 . Version 1
Motoki, F., Neto, V.P., Rodrigues, V.: More human than human: measuring chatgpt political bias. Public Choice 198(1), 3-23 (2024) https://doi.org/10.1007/s11127-023-01097-2
Downloads
Posted
Submitted: 2024-11-21 06:36:07 UTC
Published: 2024-11-22 10:08:20 UTC
License
Copyright (c) 2024
Keisuke Sato
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
