The use of artificial intelligence and its challenges for academic evaluation: a literature review
DOI:
https://doi.org/10.29197/cpu.v21i41.564Keywords:
Natural language processing, ChatGPT, Artificial Intelligence, Education, Academic evaluationAbstract
Natural language processing (NLP) has been studied in the computer industry for a long time. Recent technical advances have led to the development of complex artificial intelligence (AI) models, including Chat Generative Pre-Trained Transformer (GPT chat). These have a worrying potential for academic evaluation because they can carry out a range of linguistic activities and generate human reactions. Considering this issue, the aim of this article is to examine the potential advantages and disadvantages of using Chat GPT and other NLP technologies in academic writing and their evaluation, according to the review of the literature, to draw attention to the ethical concerns raised by its use. For this purpose, a literature search was carried out that included academic articles on Google Academic, where the result of the search was 12 articles with a high number of quotes. A qualitative methodology was used for the study, which involves reading and analyzing the sources to find relevant information to meet the research objectives. Keywords such as "GPT Chat", "AI-generated content", "academic writing" and "natural language processing" "fraud (ethics) in evaluations" guided the search. The results of this preliminary analysis suggest that Chat GPT and other NLP techniques could increase the efficiency of academic writing and research. However, its use also raises concerns about how it can affect the reliability and validity of academic work. The study emphasizes the need to explore thoroughly the potential applications, dangers, and limitations of these technologies. It also emphasizes the value of in-depth discussions and moral concerns around its implementation.
Downloads
References
Ahsan, K., Akbar, S. y Kam, B. (2022). Contract cheating in higher education: A systematic literature review and future research agenda. Assessment & Evaluation in Higher Education, 47(4), 523–539. https://doi.org/10.1080/02602938.2021.1931660
AlAfnan, M. A., Dishari, S., Jovic, M. y Lomidze, K. (2023). Chatgpt as an educational tool: Opportunities, challenges, and recommendations .communication, business writing, and composition courses. Journal of Artificial Intelligence and Technology, 3(2), 60-68. https://ojs.istp-press.com/jait/article/download/184/178
Anson, C. M. y Straume, I. (2022). Amazement and Trepidation: Implications of AI-Based Natural Language Production for the Teaching of Writing. Journal of Academic Writing, 12(1), 1–9. https://doi.org/10.18552/joaw.v12i1.820
Ayd?n O?, Karaarslan E. OpenAI ChatGPT generated literature review: Digital twin in healthcare. In: Ayd?n O?, ed. Emerging computer technologies 2. I?zmir Akademi DernegiI?zmir Akademi Dernegi, I?zmir 2022.
Assaraf, N. (2022). Chatgpt: Optimizing language models for dialogue. https://blog.cloudhq.net/openais-chatgpt-optimizing- language-models-for-dialogue/
Cassidy, C. (2023). Australian universities to return to ‘pen and paper’ exams after students caught using AI to write essays. The Guardian. https://www.theguardian.com/australia-news/2023/jan/10/universities-to-return-to- pen-and-paper-exams-after-students-caught-using-ai-to-write-essays Hutson 2022
Chaudhry, I. S., Sarwary, S. A. M., El Refae, G. A. y Chabchoub, H. (2023). Time to Revisit Existing Student’s Performance Evaluation Approach in Higher Education Sector in a New Era of ChatGPT—A Case Study. Cogent Education, 10(1), 2210461. https://www.tandfonline.com/doi/pdf/10.1080/2331186X.2023.2210461
Chechitelli, A. (2023). Sneak preview of Turnitin’s AI writing and ChatGPT detection capability. Turnitin. from https://www.turnitin.com/blog/sneak-preview-of-turnitins-ai-writing-and-chatgpt-detection-capability
Crouch, A. (2023). Culture making: Recovering our creative calling. InterVarsity Press. From: http://www.christianity9to5.org/wp-content/uploads/2013/04/CultureMakingChapters3-5.pdf
Dawson, P., Sutherland-Smith, W. y Ricksen, M. (2020). Can software improve marker accuracy at detecting contract cheating? A pilot study of the Turnitin authorship investigate alpha. Assessment & Evaluation in higher education, 45(4), 473-482. https://doi.org/10.1080/02602938.2019.1662884
Dawson, P. y Sutherland-Smith, W. (2018). Can markers detect contract cheating? Results from a pilot study. Assessment & Evaluation in Higher Education, 43(2), 286–293. https://doi.org/10. 1080/02602938.2017.1336746
Crawford, J., Cowling, M. y Allen, K. A. (2023). Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI). Journal of University Teaching & Learning Practice, 20(3), 02. From: https://ro.uow.edu.au/cgi/viewcontent.cgi?article=3180&context=jutlp
Dawson, P., Sutherland-Smith, W. y Ricksen, M. (2020). Can software improve marker accuracy at detecting contract cheating? A pilot study of the Turnitin authorship investigate alpha. Assessment & Evaluation in Higher Education, 45(4), 473–482. https://doi.org/10.1080/02602938. 2019.1662884
Dergaa, I., Chamari, K., Zmijewski, P., & Saad, H. B. (2023). From human writing to artificial intelligence generated text: examining the prospects and potential threats of ChatGPT in academic writing. Biology of Sport, 40(2), 615-622. https://acortar.link/uMvQ7t
Fergus, S., Botha, M. y Ostovar, M. (2023). Evaluating academic answers generated using ChatGPT. Journal of Chemical Education, 100(4), 1672-1675. https://pubs.acs.org/doi/full/10.1021/acs.jchemed.3c00087
Firat, M. (2023). What ChatGPT means for universities: Perceptions of scholars and students. Journal of Applied Learning and Teaching, 6(1). https://acortar.link/uMvQ7t
Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y. y Pearson, A. T. (2022). Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers. bioRxiv. https://doi.org/10.1101/2022.12.23.521610
Hassani, H. y Silva, E. S. (2023). The role of ChatGPT in data science: how ai-assisted conversational interfaces are revolutionizing the field. Big data and cognitive computing, 7(2), 62. Freom https://www.mdpi.com/2504-2289/7/2/62
Hern, A. (2022) AI bot ChatGPT stuns academics with essay-writing skills and usability. The Guardian. https://www.theguardian.com/technology/2022/dec/04/ai-bot-chatgpt-stuns- academics-with-essay-writing-skills-and-usability
Hopp, C. y Speil, A. (2021). How prevalent is plagiarism among college students? Anonymity preserving evidence from Austrian undergraduates. Accountability in Research, 28(3), 133–148. https://doi.org/10.1080/08989621.2020.1804880
Hutson, M. (2022). Could AI help you to write your next paper? Nature, 611(7934), 192–193. https://doi.org/10.1038/d41586-022-03479-w
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepan ?o, C., Madriaga, M., Aggabao, R., Diaz- Candido, G. y Maningo, J. (2023). Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digital Health, 2(2), e0000198.
Lines, L. (2016). Ghostwriters guaranteeing grades? The quality of online ghostwriting services available to tertiary students in Australia. Teaching in Higher Education, 21(8), 889–914. https:// doi.org/10.1080/13562517.2016.1198759
Linn, M. C., Donnelly-Hermosillo, D. y Gerard, L. (2023). Synergies Between Learning Technologies and Learning Sci- ences: Promoting Equitable Secondary School Teaching. In Handbook of research on science education (pp. 447-498). Routledge.
Linn, M. C., Gerard, L., Ryoo, K., McElhaney, K., Liu, O. L. y Rafferty, A. N. (2014). Computer-guided inquiry to improve science learning. Science, 344(6180), 155-156.
Lo, C. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13(4), 410. From: https://www.mdpi.com/2227-7102/13/4/410
Lund, B. D. y Wang, T. (2023). Chatting about ChatGPT: how may AI and GPT impact academia and libraries?. Library Hi Tech News, 40(3), 26-29. https://acortar.link/YoxHgQ
Malinka, K., Peresíni, M., Firc, A., Hujnak, O. y Janus, F. (2023). On the educational impact of ChatGPT: Is Artificial Intelligence ready to obtain a university degree? In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1 (pp. 47-53). https://dl.acm.org/doi/pdf/10.1145/3587102.3588827
Mitchell, A. (2022, December 26). Professor catches student cheating with ChatGPT: ‘I feel abject terror’. New York Post. https://nypost.com/2022/12/26/students-using-chatgpt-to-cheat- professor-warns/
Pellegrino, J. W. y Quellmalz, E. S. (2010). Perspectives on the integration of technology and assessment. Journal of Re- search on Technology in Education, 43(2), 119-134.
QAA. (2020) . Contracting to cheat in Higher Education – How to address contract heating, the use of third-party services and essay mills. Quality Assurance Agency (QAA).
Sabzalieva, E. y Valentini, A. (2023). ChatGPT e inteligencia artificial en la educación superior: Guía de inicio rápido. UNESCO.
Sedaghat, S. (2023). Early applications of ChatGPT in medical practice, education and research. Clinical Medicine, 23(3), 278-279. From https://www.rcpjournals.org/content/clinmedicine/23/3/278
Shahriar, S., y Hayawi, K. (2023). Let's have a chat! A Conversation with ChatGPT: Technology, Applications, and Limitations. arXiv preprint arXiv:2302.13817. https://arxiv.org/pdf/2302.13817.pdf
Stokel-Walker, C. (2022). AI bot ChatGPT writes smart essays — Should professors worry? Nature, d41586-022-04397-7. https://doi.org/10.1038/d41586-022-04397-7
van Dis EAM, Bollen J, Zuidema W, van Rooij R, Bockting CL. ChatGPT: five priorities for research. Nature. 2023; 614(7947):224–226.
Yang, M. (2023). New York City schools ban AI chatbot that writes essays and answers prompts. The Guardian. https://www.theguardian.com/us-news/2023/jan/06/new-york-city- schools-ban-ai-chatbot-chatgpt
Yeadon, W., Inyang, O. O., Mizouri, A., Peach, A. y Testrow, C. (2022). The death of the short-form physics essay in the coming AI revolution. arXiv preprint arXiv:2212.11661. https://arxiv.org/abs/ 2212.11661
Zhai, X. (2021). Advancing automatic guidance in virtual sci- ence inquiry: From ease of use to personalization. Educational Technology Research and Development, 69(1), 255-258.
Zhai, X., Haudek, K. C., Shi, L., Nehm, R. y Urban-Lurain, M. (2020a). From substitution to redefinition: A framework of machine learning-based science assessment. Journal of Re- search in Science Teaching, 57(9), 1430-1459.
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Los autores/as conservan los derechos de autor y ceden a la revista el derecho de la primera publicación.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Unless otherwise indicated, all articles in this journal are published under a
Licencia Internacional Creative Commons 4.0 Atribución-NoComercial-CompartirIgual .
The authors retain the copyright and assign the right to the first publication to the magazine.
