International Symposium on Telecommunication (IST)

Home / 11th International Symposium on Telecommunication (IST'2024)

Evaluating LLM-Generated Persian Questions for Teaching Conditional Programming Using Bloom's Taxonomy

Authors :

Marzieh Alidadi¹ Fattaneh Taghiyareh² Narges Shahhoseini³

1- University of Tehran 2- University of Tehran 3- University of Tehran

Keywords :

Large Language Models (LLMs)،Artificial Intelligence (AI)،Learning Objects (LOs)،Educational Contents،Persian،Question Generation،Bloom's Taxonomy

Abstract :

The use of Large Language Models (LLMs) to generate educational content is increasingly becoming popular, but we still need to learn more about their effectiveness in non-English languages, especially for professional areas such as teaching computational concepts. Using Bloom’s taxonomy as a framework of assessment, this study evaluates the ability of LLMs to author Persian (Farsi) Learning Objects (LOs) for teaching conditional programming. We provided four LLMs (BloomGPT, Code Tutor, Copilot, and LLaMa) with a prompt in Persian to create educational questions and exercises to teach conditional programming structures to novice learners. A group of experts was asked to evaluate questions generated by LLMs based on their alignment with the specified level of Bloom’s cognitive domain, suitability for teaching conditional programming structures, and clarity in using Persian language. Results show almost no agreement among experts in language clarity and fair agreement on other aspects of the study. BloomGPT proves itself to be dominant overall in Bloom’s Taxonomy, especially at the “Analyze” level. Meanwhile, Copilot closely followed Code Tutor in most aspects of the study. Our findings provide insights into the ability of LLMs to design high-quality Persian learning resources in computer programming.