The Effect of Question Formats in Dialogue with Generative AI: A Comparative Analysis of Open Questions and Prompt Engineering
##article.authors##
-
佐藤, 桂輔
茨城工業高等専門学校 一般教養部
https://orcid.org/0000-0002-0469-8077
https://researchmap.jp/skeisuke
DOI:
https://doi.org/10.51094/jxiv.853キーワード:
Generative AI、 Question formats、 Open-ended questions、 Prompt engineering、 Human-AI interaction抄録
As dialogue with generative AI enters a new phase, this study compares and analyzes the effects of question formats in AI interactions, specifically comparing open-ended questions with prompt engineering (structured questions). We conducted experiments using eight AI models in four areas: the future of education, improving communication within companies, family meal planning, and new movies, and analyzed them using 20 evaluation metrics.
The results showed that the question format had a multifaceted and complex effect on the AI response. Open questions showed an advantage in terms of creativity, diversity, and thought promotion, while prompt engineering formats were effective in terms of concreteness and naturalness of dialogue. In addition, the effectiveness of the question format was highly dependent on the theme and AI model used.
These findings suggest the importance of strategic selection of question formats in designing interactions with AI. Choosing the appropriate question format according to the purpose, theme, and AI model used can lead to more effective and creative interactions.
This research provides new guidelines for effective interaction with AI and emphasizes the importance of improving AI literacy.
利益相反に関する開示
The author declares no competing interests.ダウンロード *前日までの集計結果を表示します
引用文献
Eapen, T., Finkenstadt, D.J., Folk, J., Venkataswamy, L.: How generative ai can augment human creativity. Harvard Business Review 101(4) (2023) 25
Rayan, J., Kanetkar, D., Gong, Y., Yang, Y., Palani, S., Xia, H., Dow, S.P.: Exploring the potential for generative ai-based conversational cues for real-time collaborative ideation. In: Proceedings of the 16th Conference on Creativity & Cognition, pp. 117–131 (2024)
Yang, D.: Human-ai interaction in the age of large language models. In: Proceedings of the AAAI Symposium Series, vol. 3, pp. 66–67 (2024)
Bozkurt, A.: Tell me your prompts and I will make them true: The alchemy of prompt engineering and generative AI. International Council for Open and Distance Education Oslo, Norway (2024). https://doi.org/10.55982/openpraxis.16.2.661
Korzy´nski, P., Mazurek, G., Krzypkowska, P., Kurasi´nski, A.: Artificial intelligence prompt engineering as a new digital competence: Analysis of generative ai technologies such as chatgpt. Entrepreneurial Business and Economics Review (2023) https://doi.org/10.15678/eber.2023.110302
White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., Schmidt, D.C.: A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT (2023). https://arxiv.org/abs/2302.11382
White, J., Hays, S., Fu, Q., Spencer-Smith, J., Schmidt, D.C.: ChatGPT Prompt Patterns for Improving Code Quality, Refactoring, Requirements Elicitation, and Software Design (2023). https://arxiv.org/abs/2303.07839
Wit, A.: ‘when i’m retired...’: Using topic modeling to analyze open-ended survey questions in the a broader mind longitudinal survey. (2021). https://doi.org/10.31235/osf.io/ng4m2 . https://api.semanticscholar.org/CorpusID:240821164
Haq, I.U., Pifarr´e, M., Fraca, E.: Novelty evaluation using sentence embedding models in open-ended cocreative problem-solving. International Journal of Artificial Intelligence in Education (2024) https://doi.org/10.1007/s40593-024-00392-3
Ecoffet, A., Clune, J., Lehman, J.: Open Questions in Creating Safe Open-ended AI: Tensions Between Control and Creativity (2020). https://arxiv.org/abs/2006.07495
Chen, B., Zhang, Z., Langren´e, N., Zhu, S.: Unleashing the potential of prompt engineering in large language models: a comprehensive review. arXiv preprint arXiv:2310.14735 (2023)
Boussioux, L., Lane, J.N., Zhang, M., Jacimovic, V., Lakhani, K.R.: The crowdless future? generative ai and creative problem solving. Working Paper 24-005, Harvard Business School Technology & Operations Mgt. Unit (July 2024). https://doi.org/10.2139/ssrn.4533642 . https://ssrn.com/abstract=4533642
Doshi, A.R., Hauser, O.P.: Generative ai enhances individual creativity but reduces the collective diversity of novel content. Science Advances 10(28), 5290 (2024) https://doi.org/10.1126/sciadv.adn5290 https://www.science.org/doi/pdf/10.1126/sciadv.adn5290
Perifanis, N.-A., Kitsios, F.: Investigating the influence of artificial intelligence on business value in the digital era of strategy: A literature review. Information 14(2) (2023) https://doi.org/10.3390/info14020085
Markowitz, D.M., Boyd, R.L., Blackburn, K.: From silicon to solutions: Ai’s impending impact on research and discovery. Frontiers in Social Psychology 2 (2024) https://doi.org/10.3389/frsps.2024.1392128
Anthropic: Claude prompting guide.md. Claude AI assistant. Unpublished document, Claude Example Project, as of August 18, 2024 (2024)
Shimabucoro, L., Ruder, S., Kreutzer, J., Fadaee, M., Hooker, S.: LLM See, LLM Do: Guiding Data Generation to Target Non-Differentiable Objectives (2024). https://arxiv.org/abs/2407.01490
Oppenlaender, J., Linder, R., Silvennoinen, J.: Prompting AI Art: An Investigation into the Creative Skill of Prompt Engineering (2024). https://arxiv.org/abs/2303.13534
Marrone, R., Taddeo, V., Hill, G.: Creativity and artificial intelligence—a student perspective. Journal of Intelligence 10(3), 65 (2022) https://doi.org/10.3390/jintelligence10030065
ダウンロード
公開済
投稿日時: 2024-08-20 01:13:53 UTC
公開日時: 2024-08-22 08:22:01 UTC
ライセンス
Copyright（c）2024
佐藤, 桂輔
この作品は、Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Licenseの下でライセンスされています。
