Process-Oriented Guided-Inquiry Learning in Saudi Secondary School Chemistry Instruction

Amani Khalaf. H. Alghamdi; Fayadh Hamed Alanazi

doi:10.29333/ejmste/9278

Full Text (PDF)

Amani Khalaf. H. Alghamdi ¹ ^* , Fayadh Hamed Alanazi ²

More Detail

¹ Professor, College of Education, Imam Abdulrahman Bin Faisal University, SAUDI ARABIA
² College of Education, Jouf University, SAUDI ARABIA
^* Corresponding Author

Abstract

Students’ perceptions of the Process-Oriented Guided Inquiry Learning (POGIL) approach in Saudi Arabian public secondary school chemistry classes was investigated. The sample comprised 189 Grade 10 students (male and female) from four schools in the northern and southern regions. Student-centered (POGIL) and teacher-focused chemistry teaching approaches were applied using the same instructional materials to randomly selected gendered experimental and control groups over a four-week period in winter 2018. Results from analyzing What is Happening in This Class? (WIHIC) data revealed that POGIL helped students more actively engage in the classroom thus enabling them to perceive their chemistry learning experience as positive and academically effective. POGIL also improved students’ affective traits such as cohesiveness and personal relevance. There were no gendered differences in overall WIHIC scores. Results suggest that (a) POGIL mitigates the impact of borrowing education curricula and policies from Western countries and (b) Saudi pre-service and inservice teachers should receive POGIL training because POGIL works in the Saudi educational context.

Keywords

Process-Oriented Guided Inquiry Learning (POGIL)
secondary science education
chemistry
Saudi Arabia
gendered differences

License

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Article Type: Research Article

https://doi.org/10.29333/ejmste/9278

EURASIA J Math Sci Tech Ed, 2020 - Volume 16 Issue 12, Article No: em1920

Publication date: 28 Nov 2020

Article Views: 877

Article Downloads: 690

Open Access References How to cite this article

References

Alaudan, R. (2015). Saudi student teachers’ perceptions of formative assessment (Doctoral thesis), University of York, York, England. Retrieved from http://etheses.whiterose.ac.uk/7648/
Al-Dahmash, A. H., Mansour, N. S., Alshamrani, S. M., & Almohi, S. (2016). An analysis of activities in Saudi Arabian middle school science textbooks and workbooks for the inclusion of essential features of inquiry. Research in Science Education, 46(6), 879-900. https://doi.org/10.1007/s11165-015-9485-7
Alghamdi, A. (2013). Pedagogical Implications of Using Discussion Board to Improve Student Learning in Higher Education. Higher Education Studies, 3(5), 68-80.
Alghamdi, Hamdan, A., & Al-Salouli, M. (2013). Saudi elementary school science teachers' beliefs about teaching science in the new millennium. International Journal of Science and Mathematics Education, 11(2), 501-525.
Alhammad, K. (2015). A conceptual framework for re-shaping science education in Saudi Arabia. In N. Mansour & S. Al-Shamrani (eds.), Science education in the Arab Gulf States: Cultural and historical perspectives on science education (pp. 121-136). Rotterdam, the Netherlands: Sense. https://doi.org/10.1007/978-94-6300-049-9_6
Al-Harbi, A. (2013). E-learning in the KSA: A taxonomy of learning methods in Saudi Arabia. (Master’s thesis, Auckland University of Technology, Auckland, New Zealand). Retrieved from https://pdfs.semanticscholar.org/310f/c65ebcf97acec439e3b7cd934d75482fb054.pdf
Alrwathi, E., Almazroa, H., Alahmed, N., Scantbly, C., & Alshaye, F. (2014, December 4-7). Exploring curriculum implementation: Focus on science teaching. Paper presented at History, Philosophy, and Science Teaching Asian Regional Conference, Taipei, Taiwan.
Al-Sadaawi, A. S. (2007). An investigation of performance-based assessment in science in Saudi primary schools (Doctoral dissertation). Retrieved from http://vuir.vu.edu.au/1497/
Alshamrani, S. (2012). Priorities of research in science education in Saudi Arabia. Journal of Educational Sciences and Islamic Studies, 24(1), 199-128.
Al-Weher, M. (2004). The effect of a training course based on constructivism on student teachers’ perceptions of the teaching/learning process. Asia-Pacific Journal of Teacher Education, 32(2), 169-185. https://doi.org/10.1080/1359866042000248480
Atwater, M. M. (1996). Social constructivism: infusion into the multicultural science education research agenda. Journal of Research in Science Teaching, 33(8), 821-837. https://doi.org/10.1002/(SICI)1098-2736(199610)33:8<821::AID-TEA1>3.0.CO;2-Y
Avargil, S. (2019). Learning chemistry: Self-efficacy, chemical understanding, and graphing skills. Journal of Science Education and Technology, 28, 285-298. https://doi.org/10.1007/s10956-018-9765-x
Aydarova, O. (2013). If not “The Best of the West,” then “Look East”: Imported teacher education curricula in the Arabian Gulf. Journal of Studies in International Education, 17(3), 284-302. https://doi.org/10.1177/1028315312453742
Badri, M., Yang, G., Al Mazroui, K., Mohaidat, J., Al Rashedi, A., & Al Housani, N. (2017). Out-of-school experience categories influencing interest in biology of secondary school students by gender: Exploration on an Abu Dhabi sample. Journal of Biological Education, 51(2), 166-185. https://doi.org/10.1080/00219266.2016.1177576
Bahgat, G. (1999). Education in the Gulf monarchies: Retrospect and prospect. International Review of Education, 45(2), 127-136. https://doi.org/10.1023/A:1003610723356
Barak, M. (2017). Science Teacher Education in the Twenty-First Century: A Pedagogical Framework for Technology-Integrated Social Constructivism. Research in Science Education, 47(2), 283-303. https://doi.org/10.1007/s11165-015-9501-y
Barthlow, M. J. (2011). The effectiveness of process oriented guided inquiry learning to reduce alternate conceptions in secondary chemistry (Doctoral dissertation). Retrieved from https://core.ac.uk/download/pdf/58824070.pdf
Barthlow, M. J., & Watson, S. B. (2014). The effectiveness of process-oriented guided-inquiry learning to reduce alternative conceptions in secondary chemistry. School Science & Mathematics, 114(5), 246-255. https://doi.org/10.1111/ssm.12076
Berk, R. A. (1990). Importance of expert judgement in content-related validity evidence. Western Journal of Nursing Research, 12(5), 659-671. https://doi.org/10.1177/019394599001200507
Biklen, S., & Pollard, D. (2001). Feminist perspectives on gender in classrooms. In V. Richardson (Ed.), Handbook of research on teaching (pp. 723-747). Washington, DC: American Educational Research Association.
Boujaoude, S., & Gholam, G. (2014). The Middle East, Egypt, Lebanon, and Saudi Arabia. In B. T. Vlaardingerbroek & N. Taylor (Eds.), Issues in upper secondary science education: Comparative perspectives (pp. 243-260). New York, NY: Springer. https://doi.org/10.1057/9781137275967_15
Boz, Y., Yerdelen-Damar, S., & Belge-Can, H. (2018). Investigation of relations among middle school (junior high school) students’ gender, learning approaches, perceptions of learning environment and science achievement. Ilkogretim Online, 17(3), 1268-1282. https://doi.org/10.17051/ilkonline.2018.466342
Brandriet, A. R., Xu, X., Bretza, S. L., & Lewis, J. E. (2011). Diagnosing changes in attitude in first-year college chemistry students with a shortened version of Bauer’s semantic differential. Chemical Education Research Practices, 12, 271-278. https://doi.org/10.1039/C1RP90032C
Bransford, J. D., Brown, A. L., Cocking, R. R., Donovan, M. S., Bransford, J. D., & Pellegrino, J. W. (2000). How people learn. Washington, DC: National Academy Press.
Brown, J. P. (2015). Anatomy and physiology: A guided inquiry. Ottawa, ON: Wiley.
Bruner, J., & Haste, H. (1987). Making sense. New York, NY: Routledge.
Buchmann, C., DiPrete, T. A., & McDaniel, A. (2008). Gender inequalities in education. Annual Review of Sociology, 34, 319-337. https://doi.org/10.1146/annurev.soc.34.040507.134719
Busch, H. (2005). Is science education relevant? Europhysics News, 36(5), 162-167. https://doi.org/10.1051/epn:2005505
Chase, A., Pakhira, D., & Stains, M. (2013). Implementing Process-Oriented, Guided-Inquiry Learning for the first time: Adaptations and short-term impacts on students’ attitude and performance. Journal of Chemical Education, 90(4), 409-416. https://doi.org/10.1021/ed300181t
Chionh, Y. H., & Fraser, B. J. (1998, April). Validation of the ‘What is Happening in This Class’ questionnaire. Paper presented at the National Association for Research in Science Teaching Annual Meeting, San Diego, CA.
Cobern, W. W. (1996). Constructivism and non-western science education research. International Journal of Science Education, 4, 287-302. https://doi.org/10.1080/0950069960180303
Cohen, J. (1988). Statistical power analysis for the behavioural sciences. Hillsdale, NJ: Erlbaum. https://doi.org/10.1002/bs.3830330104
Dawson, C. (2000). Upper primary boys’ and girls’ interests in science: Have they changed since 1980? International Journal of Science Education, 22(6), 557-570. https://doi.org/10.1080/095006900289660
De Gale, S., & Boisselle, L. N. (2015). The effect of POGIL on academic performance and academic confidence. Social Science Education Journal, 26(1), 56-61. http://hdl.handle.net/2139/4004
denBrok, P., Telli, S., Cakiroglu, J., Taconis, R., &Tekkaya, C. (2010). Learning environment profiles of Turkish secondary biology classrooms. Learning Environments Research, 13(3), 187-204. https://doi.org/10.1007/s10984-010-9076-5
Dimopoulos, K., Koulaidis, V., & Sklaveniti, S. (2005). Towards a framework of socio-linguistic analysis of science textbooks: The Greek case. Research in Science Education, 35, 173-195. https://doi.org/10.1007/s11165-004-8162-z
Eberlein, T., Kampmeier, J., Minderhout, V., Moog, R. S., Platt, T., Varma-Nelson, P., & White, H. B. (2008). Pedagogies of engagement in science: A comparison of PBL, POGIL, and PLTL. Biochemistry and Molecular Biology Education, 36(4), 262-273. https://doi.org/10.1002/bmb.20204
Elliott, S. N., Kratochwill, T. R., Littlefield Cook, J., & Travers, J. (2000). Educational psychology (3rd ed.). Boston, MA: McGraw-Hill College.
Emilov, I., & Tafrova-Gergorova, A. (2016). A cross-national study of the learning environment in chemistry classes at Albanian, Kosovar, Romanian and Turkish secondary schools. International Journal of Science and Mathematics Education, 14(1), 107-123. https://doi.org/10.1007/s10763-014-9591-4
Fakayode, S. O. (2014). Guided-inquiry laboratory experiments in the analytical chemistry laboratory curriculum. Analytical and Bioanalytical Chemistry, 406, 1267-1271. https://doi.org/10.1007/s00216-013-7515-8
Fraser, B, J. (1998). Classroom environment instruments: Development, validity and applications. Learning Environments Research, 1, 7-33. https://doi.org/10.1023/A:1009932514731
Freedman, M. P. (2002). The influence of laboratory instruction on science achievement and attitude toward science across gender differences. Journal of Women and Minorities in Science and Engineering, 8(2), 191-200. https://doi.org/10.1615/JwomenMinorScienEng.v8.i2.50
George R. (2000). Measuring change in students’ attitudes toward science over time: An application of latent variable growth modeling. Journal of Science Education and Technology, 9(3), 213-225. https://doi.org/10.1023/A:1009491500456
Ghasemi, E., & Burley, H. (2019). Gender, affect, and math: A cross-national meta-analysis of trends in International Mathematics and Science Study 2015 outcomes. Large Scale Assessments in Education, 7(10). https://doi.org/10.1186/s40536-019-0078-1
G-Mrabet, J. (2012, February 23). Western education in the Arabian Gulf: The costs and benefits of reform [Web log post]. Retrieved from http://www.mei.edu/content/western-education-arabian-gulf-costs-and-benefits-reform
Guo, C.-J. (2007). Issues in science learning: An international perspective. In S. K. Abell & N. G. Lederman (Eds.), Handbook of research on science education (pp. 227-256). Mahwah, NJ: Lawrence Erlbaum.
Hale, D., & Mullen, L. G. (2009). Designing Process-Oriented Guided-Inquiry Activities: A new innovation for marketing classes. Marketing Education Review, 19(1), 73-80. https://doi.org/10.1080/10528008.2009.11489063
Hamdan, A. (2005). Women and education in Saudi Arabia: Challenges and achievements. International Education Journal, 6(1), 42-64.
Hamdan, A. (2006). Arab women's education and gender perceptions: An insider analysis. Journal of International Women's Studies, 8(1), 52-64.
Hanson, D. M. (2014). POGIL: Instructor’s guide to process-oriented guided-inquiry. Hampton, NH: Pacific Crest.
Howe, C. (1997). Gender and classroom interaction: A research review. Edinburgh, Scotland: Scottish Council for Research in Education.
Imel, S. (2000). Contextual learning in adult education: Practice application brief No. 12. ERIC Digest No. ED448304. https://eric.ed.gov/?id=ED448304
International Association for Educational Advancement. (2019). Trends in International Mathematics and Science Study (TIMSS). Retrieved from IEA website: https://www.iea.nl/studies/iea/timss
Jarvis T., & Pell A., (2004). Factors influencing elementary school children’s attitudes toward science before, during, and after a visit to the UK National Space Centre. Journal of Research in Science and Teaching, 42(1), 53-83. https://doi.org/10.1002/tea.20045
Jones, M. G., Howe, A., & Rua, H. J. (2000). Gender differences in students’ experiences, interests, and attitudes toward science and scientists. Science Education, 84, 180-192. https://doi.org/10.1002/(SICI)1098-237X(200003)84:2<180::AID-SCE3>3.0.CO;2-X
Jost, M. B. (1999). Theory into practice: Social constructivism and technology in instructional technology and teacher education (Doctoral dissertation). Retrieved from https://lib.dr.iastate.edu/rtd/12202
Kahveci, M. (2015a). Assessing high school students’ attitudes toward chemistry with a shortened semantic differential. Chemistry Education Research Practices, 16, 283-292. https://doi.org/10.1039/C4RP00186A
Kahveci, M. (2015b). Majors’ gender-based affective states toward learning physical chemistry. In M. Kahveci & M. Orgill (Eds.), Affective dimensions in chemistry education (pp. 297-318). New York, NY: Springer. https://doi.org/10.1007/978-3-662-45085-7_15
Kan, A., & Akbas A. (2006). Affective factors that influence chemistry achievement (attitude and self-efficacy) and the power of these factors to predict chemistry achievement. Journal of Turkish Science Education, 3(1), 30-39. Retrieved from https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.629.6460&rep=rep1&type=pdf
Kind P., Jones K., & Barmby, P. (2007). Developing attitudes towards science measures. International Journal of Science Education, 29(7), 871-893. https://doi.org/10.1080/09500690600909091
Kingdom of Saudi Arabia. (2016). Vision 2030. Retrieved from Vision 2030 website: https://vision2030.gov.sa/sites/default/files/report/Saudi_Vision2030_EN_2017.pdf
Kousa, P., Kavonius, R., & Aksela, M. (2018). Low-achieving students’ attitudes towards learning chemistry and chemistry teaching methods. Chemistry Education Research and Practice, 19(2), 431-441. https://doi.org/10.1039/C7RP00226B
Kurniawan, D. A., Darmaji, A., Putri, Y. E., Jannah, N., & Puspitasari, T. O. (2019). Perception and attitudes toward science: Condition of students in learning natural sciences in Indonesia. International Journal of Scientific and Technology Research, 8(10), 2293-2298. Retrieved from http://www.ijstr.org/final-print/oct2019/Perception-And-Attitudes-Toward-Science-Condition-Of-Students-In-Learning-Natural-Sciences-In-Indonesia.pdf
Lee, J. F. K., & Collins, P. (2009). Australian English-language textbooks: The gender issues. Gender and Education, 21(4), 353-370. https://doi.org/10.1080/09540250802392257
Litvin, K. (2010). Understanding the cultural dimensions of Gulf Arab students. Doha, Qatar: Qatar University, Foundation Programme English Department.
Mayr, A. (2015). Institutional discourse. In D. Tannen, H. E. Hamilton, & D. Chiffrin (Eds.), The handbook of discourse analysis (2^nd ed.) (pp. 755-774). Hoboken, NJ: Wiley Blackwell. https://doi.org/10.1002/9781118584194.ch35
Mercer, N., & Littleton, K. (2007). Dialogue and the development of children’s thinking. New York, NY: Routledge. https://doi.org/10.4324/9780203946657
Moog, R. S., Creegan, J. F., Hanson, M. D., Spencer, N. J., Straumanis, A., & Bunce, M. D. (2009). POGIL: Process-oriented guided-inquiry learning. In N. Pienta, M. M. Cooper, & T. J. Greenbowe (Eds.), Chemists’ guide to effective teaching (Vol. 2) (pp. 90-101). Upper Saddle River, NJ: Prentice Hall.
Ormerod, M., & Duckworth, D. (1975). Pupils’ attitudes to science. Slough, England: National Foundation for Educational Research.
Osborne J., Simon, S., & Collins, S. (2003). Attitudes towards science: A review of the literature and its implications. International Journal of Science Education, 25(9), 1049-1079. https://doi.org/10.1080/0950069032000032199
Papanastasiou, C., & Papanastasiou, E. C. (2004). Major influences on attitudes toward science. Educational Research and Evaluation, 10(3), 239-257. https://doi.org/10.1076/edre.10.3.239.30267
Pedretti, E. (1999). Decision making and STS education: Exploring scientific knowledge and social responsibility in schools and science centers through an issues-based approach. School Science and Mathematics, 99, 174-181. https://doi.org/10.1111/j.1949-8594.1999.tb17471.x
Pickens, J. (2005). Attitudes and perceptions. In N. Borkowski (Ed.), Organizational behavior in health care (pp. 43-76). New York, NY: Jones & Barlett.
Potvin, P., & Hasni, A. (2014). Interest, motivation and attitude towards science and technology at K-12 levels: A systematic review of 12 years of educational research. Studies in Science Education, 50(1), 85-129. https://doi.org/10.1080/03057267.2014.881626
Rahayu, S. (2015). Evaluating the affective dimension in chemistry education. In N. Kahveci & M. Orgill (Eds.), Affective domains in chemistry education (pp. 29-49). Berlin, Germany: Springer. https://doi.org/10.1007/978-3-662-45085-7_2
Rummel, R. J. (1981). Understanding conflict and war, Volume 2. San Francisco, CA: Sage. Retrieved from https://www.hawaii.edu/powerkills/TCH.CHAP6.HTM
Said, Z., Summers, R., Abd-El-Khalick, F., & Wang, S. (2016) Attitudes toward science among grades 3 through 12 Arab students in Qatar: Findings from a cross-sectional national study. International Journal of Science Education, 38(4), 621-643. https://doi.org/10.1080/09500693.2016.1156184
Selim, M., & Shrigley, R. (1983). The group dynamics approach: A sociopsychological approach for testing the effect of discovery and expository teaching on the science achievement and attitude of young Egyptian students. Journal of Research in Science Teaching, 20, 213-224. https://doi.org/10.1002/tea.3660200305
Şen, Ş., Yılmaz, A., & Geban, Ö. (2015). The effects of process oriented guided inquiry learning environment on students’ self-regulated learning skills. Problems of Education in the 21st Century, 66, 54-66. Retrieved from http://journals.indexcopernicus.com/abstract.php?icid=1167872
Şen, Ş., Yılmaz, A., & Geban, Ö. (2016). The effect of process oriented guided inquiry learning (POGIL) on 11th graders’ conceptual understanding of electrochemistry. Asia Pacific Forum on Science Learning and Teaching, 17(2), Article 5. https://www.eduhk.hk/apfslt/v17_issue2/sen/index.htm#con
Silfver, E. (2019). Gender performance in an out-of-school science context. Cultural Studies of Science Education, 14, 139-155. https://doi.org/10.1007/s11422-017-9851-z
Smith, L., & Abouammoh, A. (Eds.). (2013). Higher education in Saudi Arabia. New York, NY: Springer. https://doi.org/10.1007/978-94-007-6321-0
Spencer, J. N., & Moog, R. S. (2008). POGIL in the physical chemistry classroom. In R. S. Moog & J. N. Spencer (Eds.), Process-oriented guided-inquiry learning (pp. 148-156). Washington, DC: American Chemical Society. https://doi.org/10.1021/bk-2008-0994.ch013
Tranter, P., & Warn, J. (2008). Relationships between interest in motor racing and driver attitudes and behaviour amongst mature drivers: An Australian case study. Accident Analysis and Prevention, 40, 1683-1689. https://doi.org/10.1016/j.aap.2008.06.003
Treagust, D. F., Qureshi, S. S., Ojeil, J., Vishnumolakala, V. R., Mocerino, M., & Southan, D. C. (2018). Process-oriented guided inquiry learning (POGIL) as a culturally relevant pedagogy (CRP) in Qatar: A perspective from grade 10 chemistry classes. Research in Science Education. https://doi.org/10.1007/s11165-018-9712-0
Trumper, R. (2006). Factors affecting junior high school students’ interest in physics. Journal of Science Education and Technology, 15(1), 47-58. https://doi.org/10.1007/s10956-006-0355-6
Valdez, J. E., & Bungihan, M. E. (2019). Problem-based learning approach enhances the problem solving skills in chemistry of high school students. Journal of Technology and Science Education, 9(3), 282-294. https://doi.org/10.3926/jotse.631
Villagonzalo, E. C. (2014, March 6-8). Process-oriented guide-inquiry learning: An effective approach in enhancing students’ academic performance. In Proceedings of the DLSU Research Congress. Manila, Philippines.
Walker, L., & Warfa, A. R. M. (2017). Process oriented guided inquiry learning (POGIL®) marginally effects student achievement measures but substantially increases the odds of passing a course. PLoS One, 12(10), Article No. e0186203. https://doi.org/10.1371/journal.pone.0186203
Whiteley, P. (2007). The ‘gender fairness’ of integrated science textbooks used in Jamaican high schools. International Journal of Science Education, 18(8), 969-976. https://doi.org/10.1080/0950069960180808
Wiseman, A., & Al-Bakr, F. (2012). The elusiveness of teacher quality: A comparative analysis of teacher certification and student achievement in Gulf Cooperation Council (GCC) countries. Prospects, 43, 289-309. https://doi.org/10.1007/s11125-013-9272-z
Zgraggen, S. (2018). Comparing the Process Oriented Guided Inquiry Learning (POGIL) method to an Independently Developed Guided Inquiry Method (InDGIM) in a high school academic chemistry course (Doctoral dissertation). Retrieved from https://scholarworks.arcadia.edu/grad_etd/15/

How to cite this article

APA

Alghamdi, A. K. H., & Alanazi, F. H. (2020). Process-Oriented Guided-Inquiry Learning in Saudi Secondary School Chemistry Instruction. Eurasia Journal of Mathematics, Science and Technology Education, 16(12), em1920. https://doi.org/10.29333/ejmste/9278

Vancouver

Alghamdi AKH, Alanazi FH. Process-Oriented Guided-Inquiry Learning in Saudi Secondary School Chemistry Instruction. EURASIA J Math Sci Tech Ed. 2020;16(12):em1920. https://doi.org/10.29333/ejmste/9278

AMA

Alghamdi AKH, Alanazi FH. Process-Oriented Guided-Inquiry Learning in Saudi Secondary School Chemistry Instruction. EURASIA J Math Sci Tech Ed. 2020;16(12), em1920. https://doi.org/10.29333/ejmste/9278

Chicago

Alghamdi, Amani Khalaf. H., and Fayadh Hamed Alanazi. "Process-Oriented Guided-Inquiry Learning in Saudi Secondary School Chemistry Instruction". Eurasia Journal of Mathematics, Science and Technology Education 2020 16 no. 12 (2020): em1920. https://doi.org/10.29333/ejmste/9278

Harvard

Alghamdi, A. K. H., and Alanazi, F. H. (2020). Process-Oriented Guided-Inquiry Learning in Saudi Secondary School Chemistry Instruction. Eurasia Journal of Mathematics, Science and Technology Education, 16(12), em1920. https://doi.org/10.29333/ejmste/9278

MLA

Alghamdi, Amani Khalaf. H. et al. "Process-Oriented Guided-Inquiry Learning in Saudi Secondary School Chemistry Instruction". Eurasia Journal of Mathematics, Science and Technology Education, vol. 16, no. 12, 2020, em1920. https://doi.org/10.29333/ejmste/9278