Design of instruments for scientific creative thinking skills and creative thinking digital skills: Rasch models and confirmatory factor analysis

Firdaus Firdaus; Wiyanto Wiyanto; Ngurah Made Darma Putra; Wiwi Isnaeni

doi:10.29333/ejmste/16310

Full Text (PDF)

Firdaus Firdaus ¹ ² , Wiyanto Wiyanto ¹ ^* , Ngurah Made Darma Putra ¹ , Wiwi Isnaeni ¹

More Detail

¹ Universitas Negeri Semarang, Semarang, INDONESIA² Universitas Sains Al-Qur'an, Wonosobo, INDONESIA^* Corresponding Author

Abstract

This study aims to design a valid and reliable instrument to measure scientific creative thinking skills (SCTS) and creative thinking digital skills (CTDS) for undergraduates majoring in physics education. The research employs the research and development approach, using an adapted Borg & Gall model. The instrument is designed based on the three-dimensional scientific structure creativity model for measuring SCTS and an adapted van Laar’s model for CTDS. Validity and reliability testing is conducted through empirical testing using the Rasch model and confirmatory factor analysis (CFA). The study’s results indicate that the developed instrument has met high standards of validity and reliability. The main finding of this study is the availability of an effective instrument to measure creative thinking skills in scientific and digital contexts. This research contributes to the development of instruments and encourages innovation in learning. The results can help educators provide targeted feedback and design learning that strengthens students’ creative thinking skills.

Keywords

CFA
CTDS
instruments
Rasch models
SCTS

License

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Article Type: Research Article

EURASIA J Math Sci Tech Ed, Volume 21, Issue 5, May 2025, Article No: em2632

https://doi.org/10.29333/ejmste/16310

Publication date: 01 May 2025

Article Views: 235

Article Downloads: 172

Open Access References How to cite this article

References

Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45(1), 131-142. https://doi.org/10.1177/0013164485451012
Andrich, D., & Marais, I. (2019). A course in Rasch measurement: Measuring in the educational, social, and health sciences. Springer. https://doi.org/10.1007/978-981-13-7496-8
Aschauer, W., Haim, K., & Weber, C. (2022). A contribution to scientific creativity: A validation study measuring divergent problem solving ability. Creativity Research Journal, 34(2), 195-212. https://doi.org/10.1080/10400419.2021.1968656
Bartlett, J. W., & Frost, C. (2008). Reliability, repeatability and reproducibility: Analysis of measurement errors in continuous variables. Ultrasound in Obstetrics and Gynecology, 31(4), 466-475. https://doi.org/10.1002/uog.5256
Benbow, R. J., Lee, C., & Hora, M. T. (2021). Exploring college faculty development in 21st-century skill instruction: An analysis of teaching-focused personal networks. Journal of Further and Higher Education, 45(6), 818-835. https://doi.org/10.1080/0309877X.2020.1826032
Berkowitz, D., Caner, M., & Fang, Y. (2012). The validity of instruments revisited. Journal of Econometrics, 166(2), 255-266. https://doi.org/10.1016/j.jeconom.2011.09.038
Binkley, M., Erstad, O., Herman, J., Raizen, S., Ripley, M., Miller-Ricci, M., & Rumble, M. (2012). Defining twenty-first century skills. In P. Griffin, B. McGaw, & E. Care (Eds.), Assessment and teaching of 21^st century skills (pp. 17-66). Springer. https://doi.org/10.1007/978-94-007-2324-5_2
Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences. Routledge. https://doi.org/10.4324/9781315814698
Boone, W. J. (2016). Rasch analysis for instrument development: Why, when, and how? CBE–Life Sciences Education, 15(4). https://doi.org/10.1187/cbe.16-04-0148
Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch analysis in the human sciences. Springer. https://doi.org/10.1007/978-94-007-6857-4
Cojocariu, V. M., & Boghian, I. (2024). A literature review on digital creativity in higher education–Toward a conceptual model. Education Sciences, 14(11), Article 1189. https://doi.org/10.3390/educsci14111189
Comrey, A. L., & Lee, H. B. (2013). A first course in factor analysis. Psychology Press. https://doi.org/10.4324/9781315827506
Daker, R. J., Colaizzi, G. A., Mastrogiannis, A. M., Sherr, M., Lyons, I. M., & Green, A. E. (2022). Predictive effects of creative abilities and attitudes on performance in university-level computer science courses. Translational Issues in Psychological Science, 8(1), 104-124. https://doi.org/10.1037/tps0000293
Davis, G. A. (2012). Gifted children and freedom education. PT Index.
de Alencar, E. M. L. S., Fleith, D. D. S., & Pereira, N. (2017). Creativity in higher education: Challenges and facilitating factors. Trends in Psychology, 25(61), 553-561. https://doi.org/10.9788/TP2017.2-09
DeVellis, R. F., & Thorpe, C. T. (2003). Scale development theory and applications. SAGE.
Dollinger, S. J., Urban, K. K., & James, T. A. (2004). Creativity and openness: Further validation of two creative product measures. Creativity Research Journal, 16(1), 35-47. https://doi.org/10.1207/s15326934crj1601_4
Falcão, F., Jalali, C., & Costa, P. (2023). Empirical validation study and psychometric evaluation of the properties of the populist attitudes scale for the Portuguese population. BMC Psychology, 11, Article 93. https://doi.org/10.1186/s40359-023-01118-1
Ferrari, A. (2012). Digital competence in practice: An analysis of frameworks. Publications Office of the European Union. https://doi.org/10.2791/82116
Forum, W. E. (2023). Future of jobs report 2023. World Economic Forum. https://doi.org/10.1142/11458
Gable, R. K., & Wolf, M. B. (1993). The validity of effective instruments. In Instrument development in the affective domain. Evaluation in education and human services, vol 36 (pp. 95-200). Springer. https://doi.org/10.1007/978-94-011-1400-4_4
Georgiou, H., Turney, A., Matruglio, E., Jones, P., Gardiner, P., & Edwards-Groves, C. (2022). Creativity in higher education: A qualitative analysis of experts’ views in three disciplines. Education Sciences, 12(3), Article 154. https://doi.org/10.3390/educsci12030154
Glerum, A., Atasoy, B., & Bierlaire, M. (2014). Using semi-open questions to integrate perceptions in choice models. Journal of Choice Modelling, 10(1), 11-33. https://doi.org/10.1016/j.jocm.2013.12.001
Gough, H. G. (1979). A creative personality scale for the adjective check list. Journal of Personality and Social Psychology, 37(8), 1398-1405. https://doi.org/10.1037/0022-3514.37.8.1398
Gruys, M. L., Munshi, N. V., & Dewett, T. C. (2011). When antecedents diverge: Exploring novelty and value as dimensions of creativity. Thinking Skills and Creativity, 6(2), 132-137. https://doi.org/10.1016/j.tsc.2011.01.005
Guilford, J. P. (1957). Creative abilities in the arts. Psychological Review, 64(2), 110-118. https://doi.org/10.1037/h0048280
Hair Jr, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis. Cengage Learning. https://doi.org/10.3390/polym12123016
Haleem, A., Javaid, M., Qadri, M. A., & Suman, R. (2022). Understanding the role of digital technologies in education: A review. Sustainable Operations and Computers, 3, 275-285. https://doi.org/10.1016/j.susoc.2022.05.004
Hennessey, B. A., & Amabile, T. M. (2010). Creativity. Annual Review of Psychology, 61, 569-598. https://doi.org/10.1146/annurev.psych.093008.100416
Herlinawati, H., Marwa, M., Ismail, N., Junaidi, Liza, L. O., & Situmorang, D. D. B. (2024). The integration of 21^st century skills in the curriculum of education. Heliyon, 10(15), Article e35148. https://doi.org/10.1016/j.heliyon.2024.e35148
Hong, E. (2013). Creative thinking abilities: Measures for various domains. In H. O’Neil, R. Perez, & E. Baker (Eds.), Teaching and measuring cognitive readiness (pp. 201-222). Springer. https://doi.org/10.1007/978-1-4614-7579-8_11
Hooper, D., Coughlan, J., & Mullen, M. R. (2008). Structural equation modelling: Guidelines for determining model fit. Electronic Journal of Business Research Methods, 6(1), 53-60. https://doi.org/10.21427/D7CF7R
Hu, W., & Adey, P. (2002). A scientific creativity test for secondary school students. International Journal of Science Education, 24(4), 389-403. https://doi.org/10.1080/09500690110098912
Isnaeni, W., Budi Prasetyo, A. P., Ristianafi, D. P., & Setiawan, H. (2019). Analyzing students’ critical thinking, communication, collaboration, and creativity skills during animal physiology courses. Unnes Science Education Journal, 8(2), 168-175. https://doi.org/10.15294/usej.v8i2.34047
Jones, C., & Townley, B. (2016). Misfits, mavericks and mainstreams: Drivers of innovation in the creative industries. Organization Studies, 37(6), 751-768. https://doi.org/10.1177/0170840616647671
Kubiszyn, T., & Borich, G. D. (2013). Educational testing and measurement: Classroom application and practice. Wiley Global Education.
Lehmann, J., & Gaskins, B. (2019). Learning scientific creativity from the arts. Palgrave Communications, 5(1), 4-8. https://doi.org/10.1057/s41599-019-0308-8
León-Pérez, F., Bas, M.-C., & Escudero-Nahón, A. (2020). Self-perception about emerging digital skills in higher education students. Comunicar, 28(62), 91-101. https://doi.org/10.3916/C62-2020-08
Lin, C. Y., Hwang, J. S., Wang, W. C., Lai, W. W., Su, W. C., Wu, T. Y., Yao, G., & Wang, J. Der. (2019). Psychometric evaluation of the WHOQOL-BREF, Taiwan version, across five kinds of Taiwanese cancer survivors: Rasch analysis and confirmatory factor analysis. Journal of the Formosan Medical Association, 118(1 Part 2), 215-222. https://doi.org/10.1016/j.jfma.2018.03.018
Lin, C.-Y., & Pakpour, A. H. (2017). Using hospital anxiety and depression scale (HADS) on patients with epilepsy: Confirmatory factor analysis and Rasch models. Seizure, 45, 42-46. https://doi.org/10.1016/j.seizure.2016.11.019
Linacre, J. M. (1994). Sample size and item calibration stability. Rasch Measurement Transactions, 7(4), Article 328.
Mahama, I., Yusuf Dramanu, B., & Asamoah-Gyimah, K. (2023). Predictive abilities of curiosity, creativity, and motivation on academic performance of high school students in Ghana. Education Research International. https://doi.org/10.1155/2023/4626945
Marbach-Ad, G., Hunt, C., & Thompson, K. V. (2019). Exploring the values undergraduate students attribute to cross-disciplinary skills needed for the workplace: An analysis of five stem disciplines. Journal of Science Education and Technology, 28(5), 452-469. https://doi.org/10.1007/s10956-019-09778-8
Mark, K. P., So, J. C. H., Chan, V. C. W., Luk, G. W. T., & Ho, W. T. (2018). Surviving in the gig economy: Change of stem students’ perceptions on the generic skills for the workplace. In Proceedings of the 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (pp. 1085-1090). IEEE. https://doi.org/10.1109/TALE.2018.8615414
Marsh, H. W., Hau, K. T., Balla, J. R., & Grayson, D. (1998). Is more ever too much? The number of indicators per factor in confirmatory factor analysis. Multivariate Behavioral Research, 33(2), 181-220. https://doi.org/10.1207/s15327906mbr3302_1
Martinková, P., & Drabinová, A. (2019). ShinyItemAnalysis for teaching psychometrics and to enforce routine analysis of educational tests. The R Journal, 10(2), 503-515. https://doi.org/10.32614/RJ-2018-074
Martinková, P., Drabinová, A., Liaw, Y. L., Sanders, E. A., McFarland, J. L., & Price, R. M. (2017). Checking equity: Why differential item functioning analysis should be a routine part of developing conceptual assessments. CBE–Life Sciences Education, 16(2). https://doi.org/10.1187/cbe.16-10-0307
Milfont, T. L., & Fischer, R. (2010). Testing measurement invariance across groups: Applications in cross-cultural research. International Journal of Psychological Research, 3(1), 111-130. https://doi.org/10.21500/20112084.857
Mueller, R. O., & Hancock, G. R. (2015). Factor analysis and latent structure analysis: Confirmatory factor analysis. In N. J. Smelser, & P. B. Baltes (Eds.), International encyclopedia of the social & behavioral sciences (pp. 686-690). Elsevier. https://doi.org/10.1016/B978-0-08-097086-8.25009-5
O’Leary-Kelly, S. W., & Vokurka, R. J. (1998). The empirical assessment of construct validity. Journal of Operations Management, 16(4), 387-405. https://doi.org/10.1016/s0272-6963(98)00020-5
OECD. (2017). Pisa 2015 assessment and analytical framework: Science, reading, mathematic, financial literacy and collaborative problem solving. OECD Publishing. https://doi.org/10.1787/9789264281820-en
Partnership for 21st-Century Skills. (2015). Partnership for 21st-century skills-core content integration. battelle for kids. https://www.battelleforkids.org/insights/p21-resources/
Ploj Virtič, M. (2022). Teaching science & technology: Components of scientific literacy and insight into the steps of research. International Journal of Science Education, 44(12), 1916-1931. https://doi.org/10.1080/09500693.2022.2105414
Prinsley, R., & Baranyai, K. (2015). STEM skills in the workforce: What do employers want? Occasional Paper Series, Number 9. https://doi.org/10.13140/RG.2.2.12120.60167
Robson, S. (2014). The analyzing children’s creative thinking framework: Development of an observation-led approach to identifying and Analyzing young children’s creative thinking. British Educational Research Journal, 40(1), 121-134. https://doi.org/10.1002/berj.3033
Rodríguez, G., Díez, J., Pérez, N., Baños, J. E., & Carrió, M. (2019). Flipped classroom: Fostering creative skills in undergraduate students of health sciences. Thinking Skills and Creativity, 33, Article 100575. https://doi.org/10.1016/j.tsc.2019.100575
Runco, M. A., & Jaeger, G. J. (2012). The standard definition of creativity. Creativity Research Journal, 24(1), 92-96. https://doi.org/10.1080/10400419.2012.650092
Said-Metwaly, S., Van den Noortgate, W., & Kyndt, E. (2017). Approaches to measuring creativity: A systematic literature review. Creativity. Theories–Research-Applications, 4(2), 238-275. https://doi.org/10.1515/ctra-2017-0013
Sak, U., & Ayas, M. B. (2013). Creative scientific ability test (C-SAT): A new measure of scientific creativity. Psychological Test and Assessment Modeling, 55(3), 316-329. https://doi.org/10.1037/t72324-000
Shi, Y., Yuan, T., Bell, R., & Wang, J. (2020). Investigating the relationship between creativity and entrepreneurial intention: The moderating role of creativity in the theory of planned behavior. Frontiers in Psychology, 11. https://doi.org/10.3389/fpsyg.2020.01209
Srikongchan, W., Kaewkuekool, S., & Mejaleurn, S. (2021). Backward instructional design based learning activities to developing students’ creative thinking with lateral thinking technique. International Journal of Instruction, 14(2), 233-252. https://doi.org/10.29333/iji.2021.14214a
Sternberg, R. J., & Lubart, T. I. (2014). The concept of creativity: Prospects and paradigms. In R. J. Sternberg (Ed.), Handbook of creativity (pp. 3-15). Cambridge University Press. https://doi.org/10.1017/CBO9780511807916
Stoetzler, M., & Yuval-Davis, N. (2002). Standpoint theory, situated knowledge and the situated imagination. Feminist Theory, 3(3), 315-333. https://doi.org/10.1177/146470002762492024
Sullivan, G. M. (2011). A Primer on the validity of assessment instruments. Journal of Graduate Medical Education, 3(2), 119-120. https://doi.org/10.4300/jgme-d-11-00075.1
Sun, M., Wang, M., & Wegerif, R. (2020). Effects of divergent thinking training on students’ scientific creativity: The impact of individual creative potential and domain knowledge. Thinking Skills and Creativity, 37, Article 100682. https://doi.org/10.1016/j.tsc.2020.100682
Taber, K. S. (2018). The use of Cronbach’s alpha when developing and reporting research instruments in science education. Research in Science Education, 48(6), 1273-1296. https://doi.org/10.1007/s11165-016-9602-2
Thornhill-Miller, B., Camarda, A., Mercier, M., Burkhardt, J.-M., Morisseau, T., Bourgeois-Bougrine, S., Vinchon, F., El Hayek, S., Augereau-Landais, M., Mourey, F., Feybesse, C., Sundquist, D., & Lubart, T. (2023). Creativity, critical thinking, communication, and collaboration: Assessment, certification, and promotion of 21^st century skills for the future of work and education. Journal of Intelligence, 11(3), Article 54. https://doi.org/10.3390/jintelligence11030054
Torrance, E. P. (2018). Torrance tests of creative thinking. APA PsycTests. https://doi.org/10.1037/t05532-000
Valtonen, T., Hoang, N., Sointu, E., Näykki, P., Virtanen, A., Pöysä-Tarhonen, J., Häkkinen, P., Järvelä, S., Mäkitalo, K., & Kukkonen, J. (2021). How pre-service teachers perceive their 21st-century skills and dispositions: A longitudinal perspective. Computers in Human Behavior, 116, Article 106643. https://doi.org/10.1016/j.chb.2020.106643
van Laar, E., van Deursen, A. J. A. M., Dijk, J. A. G. M., & de Haan, J. (2019). The sequential and conditional nature of 21st-century digital skills. International Journal of Communication, 13(409), 3462-3487. https://doi.org/10.1037/t73734-000
van Laar, E., van Deursen, A. J. A. M., van Dijk, J. A. G. M., & de Haan, J. (2018). 21st-century digital skills instrument aimed at working professionals: Conceptual development and empirical validation. Telematics and Informatics, 35(8), 2184-2200. https://doi.org/10.1016/j.tele.2018.08.006
van Laar, E., van Deursen, A. J. A. M., van Dijk, J. A. G. M., & de Haan, J. (2020). Determinants of 21st-century skills and 21st-century digital skills for workers: A systematic literature review. SAGE Open, 10(1). https://doi.org/10.1177/2158244019900176
Wallach, M. A., & Kogan, N. (1965). Modes of thinking in young children: A study of the creativity-intelligence distinction. Rinehart & Winston.
Willemsen, R. H., de Vink, I. C., Kroesbergen, E. H., & Lazonder, A. W. (2023). The role of creative thinking in children’s scientific reasoning. Thinking Skills and Creativity, 49, Article 101375. https://doi.org/10.1016/j.tsc.2023.101375
Williams, F. (1980). Creativity assessment packet. DOK.
Wiyanto, W., & Hidayah, I. (2021). Review of a scientific creativity test of the three-dimensional model. Journal of Physics: Conference Series, 1918, Article 052088. https://doi.org/10.1088/1742-6596/1918/5/052088
Yildiz, C., & Guler Yildiz, T. (2021). Exploring the relationship between creative thinking and scientific process skills of preschool children. Thinking Skills and Creativity, 39, Article 100795. https://doi.org/10.1016/j.tsc.2021.100795

How to cite this article

APA

Firdaus, F., Wiyanto, W., Putra, N. M. D., & Isnaeni, W. (2025). Design of instruments for scientific creative thinking skills and creative thinking digital skills: Rasch models and confirmatory factor analysis. Eurasia Journal of Mathematics, Science and Technology Education, 21(5), em2632. https://doi.org/10.29333/ejmste/16310

Vancouver

Firdaus F, Wiyanto W, Putra NMD, Isnaeni W. Design of instruments for scientific creative thinking skills and creative thinking digital skills: Rasch models and confirmatory factor analysis. EURASIA J Math Sci Tech Ed. 2025;21(5):em2632. https://doi.org/10.29333/ejmste/16310

AMA

Firdaus F, Wiyanto W, Putra NMD, Isnaeni W. Design of instruments for scientific creative thinking skills and creative thinking digital skills: Rasch models and confirmatory factor analysis. EURASIA J Math Sci Tech Ed. 2025;21(5), em2632. https://doi.org/10.29333/ejmste/16310

Chicago

Firdaus, Firdaus, Wiyanto Wiyanto, Ngurah Made Darma Putra, and Wiwi Isnaeni. "Design of instruments for scientific creative thinking skills and creative thinking digital skills: Rasch models and confirmatory factor analysis". Eurasia Journal of Mathematics, Science and Technology Education 2025 21 no. 5 (2025): em2632. https://doi.org/10.29333/ejmste/16310

Harvard

Firdaus, F., Wiyanto, W., Putra, N. M. D., and Isnaeni, W. (2025). Design of instruments for scientific creative thinking skills and creative thinking digital skills: Rasch models and confirmatory factor analysis. Eurasia Journal of Mathematics, Science and Technology Education, 21(5), em2632. https://doi.org/10.29333/ejmste/16310

MLA

Firdaus, Firdaus et al. "Design of instruments for scientific creative thinking skills and creative thinking digital skills: Rasch models and confirmatory factor analysis". Eurasia Journal of Mathematics, Science and Technology Education, vol. 21, no. 5, 2025, em2632. https://doi.org/10.29333/ejmste/16310