Investigating the Effects of Flipped Learning, Student Question Generation, and Instant Response Technologies on Students’ Learning Motivation, Attitudes, and Engagement: A Structural Equation Modeling
Chien-Yuan Su 1,  
More details
Hide details
Department of Curriculum and Learning Science, College of Education, Zhejiang University, Hangzhou, CHINA
Graduate Institute of Information and Computer Education, National Taiwan Normal University, Taipei, TAIWAN
Online publish date: 2018-04-14
Publish date: 2018-04-14
EURASIA J. Math., Sci Tech. Ed 2018;14(6):2453–2466
In recent years, flipped learning has grown in popularity and been more widely adopted as a mechanism of enabling active learning, which is used in different educational scenarios. This paper describes a novel extension of flipped learning by integrating student question-generation and an instant response system into the higher education curriculum and examining the impacts of this extension on students’ learning motivation, attitudes, and engagement. Data were obtained from 54 sophomores at Zhejiang University, China, and the data were tested using the partial least squares structural equation modeling approach. The results indicated that this research model predicted 47.3% of the variance of learning motivation, 78.6% of the variance of attitudes toward learning, and 62.4% of the variance of learning engagement. Also, the results showed that the constructs of flipped learning and student question generation have a positive impact on the students’ learning motivation, attitudes, and engagement. In contrast, though the instant response system also has a positive impact on students’ engagement, it does not influence motivation or attitudes. Instructional implications and research suggestions are provided based on the results of the study.
1. Abeysekera, L., & Dawson, P. (2015). Motivation and cognitive load in the flipped classroom: Definition, rationale and a call for research. Higher Education Research & Development, 34(1), 1–14.
2. Bates, S. P., Galloway, R. K., & McBride, K. L. (2012). Student-generated content: Using PeerWise to enhance engagement and outcomes in introductory physics courses. In N. S. Rebello, P. V. Engelhardt, & C. Singh (Eds.), AIP Conference Proceedings (Vol. 1413, pp. 123–126).
3. Beatty, I. D. (2005). Transforming student learning with classroom communication systems. EDUCAUSE Center for Analysis and Research, 2004(3), 1–13.
4. Berg, C. A. R. (2005). Factors related to attitude change toward learning chemistry among university students. Chemistry Education Research and Practice, 6(1), 1–18.
5. Bergmann, J., & Sams, A. (2012). Flip your classroom: Reach every student in every class every day: International Society for Technology in Education (1st ed.). Eugene, OR: International Society for Technology in Education.
6. Bergmann, J., & Sams, A. (2014). Flipped learning: Gateway to student engagement (1st ed.). Eugene, OR: International Society for Technology in Education.
7. Bishop, J. L., & Verleger, M. A. (2013, June). The flipped classroom: A survey of the research. Paper presented at the 120th ASEE Annual Conference & Exposition, Atlanta, GA.
8. Brewer, C. A. (2004). Near real-time assessment of student learning and understanding in biology courses. BioScience, 54(11), 1034–1039.
9. Briggs, L. (2006, November). Response devices keep FSU students focused. Campus Technology, 19. Retrived from
10. Brown, S. I., & Walter, M. I. (2005). The art of problem posing (3rd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.
11. Bruff, D. (2009). Teaching with classroom response systems: Creating active learning environments (1st ed.). San Francisco, CA: Jossey-Bass.
12. Bullock, D., LaBella, V., Clingan, T., Ding, Z., Stewart, G., & Thibado, P. (2002). Enhancing the student‐instructor interaction frequency. The Physics Teacher, 40(9), 535–541.
13. Caldwell, J. E. (2007). Clickers in the large classroom: Current research and best-practice tips. CBE—Life Sciences Education, 6(1), 9–20.
14. Campbell, J., & Mayer, R. E. (2009). Questioning as an instructional method: Does it affect learning from lectures? Applied Cognitive Psychology, 23(6), 747–759.
15. Chang, C.-Y. (2016). The marriage of interdisciplinary research: The old, the new, the borrowed, and the blue. Journal of Science and Innovation, 6(1), 29–39.
16. Chang, R.-C., Chung, L.-Y., & Huang, Y.-M. (2016). Developing an interactive augmented reality system as a complement to plant education and comparing its effectiveness with video learning. Interactive Learning Environments, 24(6), 1245–1264.
17. Chao, C.-Y., Chen, Y.-T., & Chuang, K.-Y. (2015). Exploring students’ learning attitude and achievement in flipped learning supported computer aided design curriculum: A study in high school engineering education. Computer Applications in Engineering Education, 23(4), 514–526.
18. Chen Hsieh, J. S., Huang, Y.-M., & Wu, W.-C. V. (2017). Technological acceptance of LINE in flipped EFL oral training. Computers in Human Behavior, 70, 178–190.
19. Chen, C.-H., Chiu, C.-H., & Wu, C.-Y. (2012). Effects of shared note-taking and questioning review in elementary school computer classes. Contemporary Educational Research Quarterly, 20(2), 47–91.
20. Chen, Y., Wang, Y., Kinshuk, & Chen, N.-S. (2014). Is FLIP enough? Or should we use the FLIPPED model instead? Computers & Education, 79, 16–27.
21. Chi, M. T. H. (2009). Active‐constructive‐interactive: A conceptual framework for differentiating learning activities. Topics in Cognitive Science, 1(1), 73–105.
22. Chien, Y.-T., Chang, C.-Y. (2015). Supporting socio-scientific argumentation in the classroom through automatic group formation based on students’ real-time responses. In M. S. Khine (Ed.), Science education in East Asia (pp. 549–563). Cham, Switzerland: Springer.
23. Chien, Y.-T., Lee, Y.-H., Li, T.-Y., & Chang, C.-Y. (2015). Examining the effects of displaying clicker voting results on high school students’ voting behaviors, discussion processes, and learning outcomes. EURASIA Journal of Mathematics, Science and Technology Education, 11(5), 1089–1104.
24. Chin, C. & Kayalvizhi, G. (2005). What do pupils think of open science investigations? A study of Singaporean primary 6 pupils. Educational Research, 47(1), 107–126.
25. Chin, C., & Brown, D. E. (2000). Learning deeply in science: An analysis and reintegration of deep approaches in two case studies of grade 8 students. Research in Science Education, 30(2), 173–197.
26. Chin, C., & Brown, D. E. (2002). Student-generated questions: A meaningful aspect of learning in science. International Journal of Science Education, 24(5), 521–549.
27. Chin, C., & Osborne, J. (2008). Students’ questions: A potential resource for teaching and learning science. Studies in Science Education, 44(1), 1–39.
28. Chin, W. W., & Newsted, P. R. (1999). Structural equation modeling analysis with small samples using partial least square. In R. Hoyle (Ed.), Statistical strategies for small-sample research (pp. 307–341). Thousand Oaks, CA: Sage Publications.
29. Coley, K., Hantla, B., & Cobb, C. (2013, October). Best practices for beginning a flipped classroom in the humanities. Paper presented at the 2013 NAPCE Annual Conference, Rosemont, IL.
30. Cubric, M., & Jefferies, A. (2015). The benefits and challenges of large-scale deployment of electronic voting systems: University student views from across different subject groups. Computers & Education, 87, 98–111.
31. Davies, R. S., Dean, D. L., & Ball, N. (2013). Flipping the classroom and instructional technology integration in a college-level information systems spreadsheet course. Educational Technology Research & Development, 61(4), 563–580.
32. De Jesus, H. P., Teixeira-Dias, J. J., & Watts, M. (2003). Questions of chemistry. International Journal of Science Education, 25(8), 1015–1034.
33. Donaldson, M. (1987). Children’s minds (New ed.). New York, NY: Harper Perennial.
34. Dori, Y. J., & Herscovitz, O. (1999). Question-posing capability as an alternative evaluation method: Analysis of an environmental case study. Journal of Research in Science Teaching, 36(4), 411–430.<411::AID-TEA2>3.0.CO;2-E.
35. Drake, J. M., & Barlow, A. T. (2007). Assessing students’ levels of understanding multiplication through problem writing. Teaching Children Mathematics, 14(5), 272–277.
36. El-Rady, J. (2006). To click or not to click: That’s the question. Innovate: Journal of online education, 2(4), Article 6.
37. Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50.
38. Gilboy, M. B., Heinerichs, S., & Pazzaglia, G. (2015). Enhancing student engagement using the flipped classroom. Journal of nutrition education and behavior, 47(1), 109–114.
39. Hair, J. F., Jr., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2017). A primer on partial least squares structural equation modeling (PLS-SEM) (2nd ed.). Thousand Oaks, CA: Sage Publications.
40. Hair, J. F., Jr., Tatham, R. L., Anderson, R. E., & Black, W. (1998). Multivariate data analysis (5th ed.). Upper Saddle River, NJ: Prentice Hall.
41. Hamdan, N., McKnight, P., McKnight, K., & Arfstrom, K. M. (2013). The flipped learning model: A white paper based on the literature review titled a review of flipped learning. Arlington, VA: Flipped Learning Network. Retrieved from
42. Han, J. H., & Finkelstein, A. (2013). Understanding the effects of professors’ pedagogical development with Clicker Assessment and Feedback technologies and the impact on students’ engagement and learning in higher education. Computers & Education, 65, 64–76.
43. Harlen, W., Elstgeest, J., & Jelly, S. (2001). Primary science: Taking the plunge (2nd ed.). London, England: Heinemann.
44. Heiner, C. E., Banet, A. I., & Wieman, C. (2014). Preparing students for class: How to get 80% of students reading the textbook before class. American Journal of Physics, 82(10), 989–996.
45. Hung, H.-T. (2015). Flipping the classroom for English language learners to foster active learning. Computer Assisted Language Learning, 28(1), 81–96.
46. Hwang, G.-J., Lai, C.-L., & Wang, S.-Y. (2015). Seamless flipped learning: A mobile technology-enhanced flipped classroom with effective learning strategies. Journal of Computers in Education, 2(4), 449–473.
47. Ikuta, J., & Maruno, S. (2005). Student question generation in the classroom: A review. Kyushu University Psychological Research, 6, 37–48.
48. Jensen, J. L., Kummer, T. A., & Godoy, P. D. d. M. (2015). Improvements from a flipped classroom may simply be the fruits of active learning. CBE—Life Sciences Education, 14, 1–12.
49. Kay, R. H., & Lesage, A. (2009). Examining the benefits and challenges of using audience response systems: A review of the literature. Computers & Education, 53(3), 819–827.
50. Kennedy, G., Cutts, Q., & Draper, S. W. (2006). Evaluating electronic voting systems in lectures: Two innovative methods. In D. A. Banks (Ed.), Audience response rystems in higher education. Hershey, PA: Information Science Publishing.
51. Kim, M. K., Kim, S. M., Khera, O., & Getman, J. (2014). The experience of three flipped classrooms in an urban university: An exploration of design principles. The Internet and Higher Education, 22, 37–50.
52. Koballa, T. R., Jr. (1988). Attitude and related concepts in science education. Science Education, 72(2), 115–126.
53. Lai, C.-Y., & Wu, C.-C. (2006). Using handhelds in a Jigsaw cooperative learning environment. Journal of Computer Assisted Learning, 22(4), 284–297.
54. Lantz, M. E. (2010). The use of ‘Clickers’ in the classroom: Teaching innovation or merely an amusing novelty? Computers in Human Behavior, 26(4), 556–561.
55. Latessa, R., & Mouw, D. (2005). Use of an audience response system to augment interactive learning. Family Medicine, 37(1), 12–14.
56. Lee, B. (2017). TELL us ESP in a flipped classroom. EURASIA Journal of Mathematics, Science and Technology Education, 13(8), 4995–5007.
57. Lin, P.-C., & Chen, H.-M. (2016). The effects of flipped classroom on learning effectiveness: using learning satisfaction as the mediator. World Transactions on Engineering and Technology Education, 14(2), 231–244.
58. Madsen, H. S. (1983). Techniques in Testing (1st ed.) New York, NY: Oxford University Press.
59. Maskill, R., & Pedrosa de Jesus, H. (1997a). Asking model questions. Education in Chemistry, 34(5), 132–134.
60. Maskill, R., & Pedrosa de Jesus, H. (1997b). Pupils’ questions, alternative frameworks and the design of science teaching. International Journal of Science Education, 19(7), 781–799.
61. Mok, M. M. C. (2005, December). A conceptual framework for the design of web-based self-directed learning curriculum. Paper presented at the 2005 International Conference on Education and Information Technology, Keelung, Taiwan.
62. Moredich, C., & Moore, E. (2007). Engaging students through the use of classroom response systems. Nurse Educator, 32(3), 113–116.
63. Murphy, B., & Smark, C. J. (2006). Convergence of learning experiences for first year tertiary commerce students–Are personal response systems the meeting point? The Journal of American Academy of Business, Cambridge, 10(1), 186–191.
64. Ogden, L., & Shambaugh, N. (2016). Best teaching and technology practices for the hybrid flipped college classroom. In P. Vu, S. Fredrickson, & C. Moore (Eds.), Handbook of research on innovative pedagogies and technologies for online learning in higher education (pp. 281–303). Hershey, PA: IGI Global.
65. Pedrosa de Jesus, H., Neri de Souza, F., Teixeira‐Dias, J. J. C., & Watts, M. (2005). Organising the chemistry of question‐based learning: A case study. Research in Science & Technological Education, 23(2), 179–193.
66. Penuel, W. R., Boscardin, C. K., Masyn, K., & Crawford, V. M. (2007). Teaching with student response systems in elementary and secondary education settings: A survey study. Educational Technology Research and Development, 55(4), 315–346.
67. Perez, J. A. (1986). Effects of student-generated problems on problem solving performance. Unpublished doctoral dissertation, Teachers College, Columbia University, New York, NY.
68. Pierce, R., & Fox, J. (2012). Vodcasts and active-learning exercises in a “flipped classroom” model of a renal pharmacotherapy module. American Journal of Pharmaceutical Education, 76(10), Article 196.
69. Pizzini, E. L., & Shepardson, D. P. (1991). Student questioning in the presence of the teacher during problem solving in science. School Science and Mathematics, 91(8), 348–352.
70. Preszler, R. W., Dawe, A., Shuster, C. B., & Shuster, M. (2007). Assessment of the effects of student response systems on student learning and attitudes over a broad range of biology courses. CBE—Life Sciences Education, 6(1), 29–41.
71. Rifai, N. A. (2010). Attitude, motivation, and difficulties involved in learning the English language and factors that affect motivation in learning it. Procedia - Social and Behavioral Sciences, 2(2), 5216–5227.
72. Rosenshine, B., Meister, C., & Chapman, S. (1996). Teaching students to generate questions: A review of the intervention studies. Review of Educational Research, 66(2), 181–221.
73. Sabri, M., Khalid, F. K., & Li, L. K. (2016). Assessing students engagement in an online student question-generation activity towards their learning motivation. International Journal of Languages, Literature and Linguistics, 2(1), 23–31.
74. Saulnier, B. (2015). The flipped classroom in systems analysis & design: Leveraging technology to increase student engagement. Information Systems Education Journal, 13(4), 33–40.
75. Simpson, V., & Oliver, M. (2007). Electronic voting systems for lectures then and now: A comparison of research and practice. Australasian Journal of Educational Technology, 23(2), 187–208.
76. Sohrabi, B., & Iraj, H. (2016). Implementing flipped classroom using digital media: A comparison of two demographically different groups perceptions. Computers in Human Behavior, 60, 514–524.
77. Song, D., Oh, E. Y., & Glazewski, K. (2017). Student-generated questioning activity in second language courses using a customized personal response system: A case study. Educational Technology Research and Development, 65(6), 1425–1449.
78. Stowell, J. R., & Nelson, J. M. (2007). Benefits of electronic audience response systems on student participation, learning, and emotion. Teaching of Psychology, 34(4), 253–258.
79. Strayer, J. F. (2012). How learning in an inverted classroom influences cooperation, innovation and task orientation. Learning Environments Research, 15(2), 171–193.
80. Trees, A. R., & Jackson, M. H. (2007). The learning environment in clicker classrooms: Student processes of learning and involvement in large university‐level courses using student response systems. Learning, Media and Technology, 32(1), 21–40.
81. Tu, K., & Conover, G. (2010, January). Student participation in question generation. Poster session presented at the 2010 Health Professions Educational Research Symposium, Fort Lauderdale, FL.
82. van Dijk, L. A., van der Berg, G. C., & van Keulen, H. (2001). Interactive lectures in engineering education. European Journal of Engineering Education, 26(1), 15–28.
83. Volet, S. (2001). Understanding learning and motivation in context: A multi-dimensional and multi-level cognitive-situative perspective. In S. Volet & S. Järvelä (Eds.), Motivation in learning contexts: Theoretical advances and methodological implications (pp. 57–82). Elmsford, NY: Pergamon Press.
84. Wang, F. H. (2017). An exploration of online behaviour engagement and achievement in flipped classroom supported by learning management system. Computers & Education, 114, 79–91.
85. Watts, M., Gould, G., & Alsop, S. (1997). Questions of understanding: Categorising pupils’ questions in science. School Science Review, 79(286), 57–63.
86. Wei, C.-W., Lin, Y.-C., & Lin, Y.-T. (2016). An interactive diagnosis approach for supporting clinical nursing courses. Interactive Learning Environments, 24(8), 1795–1811.
87. Yang, C. C. R. (2017). An investigation of the use of the ‘flipped classroom’ pedagogy in secondary English language classrooms. Journal of Information Technology Education: Innovations in Practice, 16, 1–20.
88. Yilmaz, R. (2017). Exploring the role of e-learning readiness on student satisfaction and motivation in flipped classroom. Computers in Human Behavior, 70, 251–260.
89. Yu, F.-Y. (2005). Promoting metacognitive strategy development through online question-generation instructional approach. In C.-K. Looi, D. Jonassen, & M. Ikeda (Eds.), Towards sustainable and scalable educational innovations informed by the learning sciences (pp. 564–571). Amsterdam, The Netherlands: IOS Press.
90. Yu, F.-Y. (2009). Scaffolding student-generated questions: Design and development of a customizable online learning system. Computers in Human Behavior, 25(5), 1129–1138.
91. Yu, F.-Y. (2011). Multiple peer-assessment modes to augment online student question-generation processes. Computers & Education, 56(2), 484–494.
92. Yu, F.-Y., & Hung, C.-C. (2006). An empirical analysis of online multiple-choice question-generation learning activity for the enhancement of students’ cognitive strategy development while learning science. In T. Simos & G. Maroulis (Series Eds.), Lecture Series on Computer and Computational Sciences: Vol. 7A. Recent progress in computational sciences and engineering (pp. 585–588). Boca Raton, FL: CRC Press.
93. Yu, F.-Y., & Liu, Y.-H. (2008). The comparative effects of student question-posing and question-answering strategies on promoting college students’ academic achievement, cognitive and metacognitive strategies use. Journal of Education & Psychology, 31(3), 25–52.
94. Yu, F.-Y., & Wu, C.-P. (2012). Student question-generation: The learning processes involved and their relationships with students’ perceived value. Journal of Research in Education Sciences, 57(4), 135–162.
95. Yu, F.-Y., Chang, Y.-L., & Wu, H.-L. (2015). The effects of an online student question-generation strategy on elementary school student English learning. Research and Practice in Technology Enhanced Learning, 10(1), Article 24.
96. Yu, F.-Y., Tsai, H.-C., & Wu, H.-L. (2013). Effects of online procedural scaffolds and the timing of scaffolding provision on elementary Taiwanese students’ question-generation in a science class. Australasian Journal of Educational Technology, 29(3), 416–433.